We all know that python is slower than compiled languages like C. But what can you do about it? Jake VanderPlas, director of research in the physical sciences for the university of Washington's eScience institute, digs into python's internals to explain how it works and what program design choices you can make to use python efficiently.
Related Stories
MATLAB and Python are both rather popular languages. Real Python has an overview of the two with an eye towards encouraging use of Python. There is a lot to say when comparing languages, so this is a long read.
MATLAB® is widely known as a high-quality environment for any work that involves arrays, matrices, or linear algebra. Python is newer to this arena but is becoming increasingly popular for similar tasks. As you’ll see in this article, Python has all of the computational power of MATLAB for science tasks and makes it fast and easy to develop robust applications. However, there are some important differences when comparing MATLAB vs Python that you’ll need to learn about to effectively switch over.
In this article, you’ll learn how to:
- Evaluate the differences of using MATLAB vs Python
- Set up an environment for Python that duplicates the majority of MATLAB functions
- Convert scripts from MATLAB to Python
- Avoid common issues you might have when switching from MATLAB to Python
- Write code that looks and feels like Python
Earlier on SN:
Python's Guido van Rossum Steps Down (2018)
What's Today's Top Language? Python... no, Wait, Java... no, C (2017)
GNU Octave - Open Source Answer to Matlab - Hits 4.0.0 (2015)
You Want MatLab on Your Resume to Get a Job at Google (2014)
Why Python is Slow: Looking Under the Hood (2014)
(Score: -1) by Anonymous Coward on Tuesday May 13 2014, @02:16AM
Remember the days when you could still get FROTHY PISS on (SoylentNews||newsitename) long after the story had been posted? Oh, what heady, foamy times those were.
(Score: 2) by bugamn on Tuesday May 13 2014, @02:19AM
I like programming in Python for it simplicity, but I know I have to pay something for that. Fortunately for most almost everything I need that cost isn't high, or there are modules that help, so Python is good enough for the job. It is also quite a nice tool to make quick programs and test ideas before a serious implementation.
(Score: 2) by Angry Jesus on Tuesday May 13 2014, @02:24AM
Python gets a lot of serious use though. As I recall, Dropbox is almost 100% python (so much so that they are developing their own JIT [dropbox.com]). Youtube is also mostly python (with ffmpeg for the video processing).
(Score: 4, Informative) by physicsmajor on Tuesday May 13 2014, @03:11AM
For good reason.
The same reasons, in fact, why you don't waste time optimizing an inefficient string processing routine which, when you profile, gets used 0.01% of your runtime. To turn that around: if Python lets your team hack out 80% of your application in 5% of the time, thanks to these features, that is an obvious massive win.
In my experience (warning: Python dev, I'm biased but also not sugarcoating) this is true. Python results in clearer, easier to maintain code. When I need serious performance there's always Cython or the various methods to hook into lower level libraries, but the remaining >= 80% of the time using Python was a clear and objective win for efficiency and maintainability.
(Score: 2) by kaszz on Tuesday May 13 2014, @04:01AM
Dito for Perl. What I really like is the powerful and easy pattern matching capability. One can mix huge binaries with bit operations and laissez faires operands that stretch arithmetic to string matching without boundaries.
As always if you code to shoot yourself in the foot. The code will deliver your misery.
(Score: 2) by Nerdfest on Tuesday May 13 2014, @10:56AM
Clear, easy to maintain Python code? So, you're the guy.
(Score: 2) by Nerdfest on Tuesday May 13 2014, @10:58AM
Damn, meant perl code. Doh.
(Score: 2) by LaminatorX on Tuesday May 13 2014, @04:21AM
The raw number crunching is slow, but the things I write in Python are network or disk access bound, not CPU bound. I can imagine a sufficiently large data set where that would shift, but in practice it's never happened.
(Score: 3, Interesting) by bill_mcgonigle on Tuesday May 13 2014, @05:46AM
I have Pythonista friends who tell me that Python is a transformable dialect of LISP. I ask them why they don't transform it to LISP and run it through the well-proven LISP to C compilers (to juice their runtime performance).
They don't answer then question and we eat pizza instead; wait about three months and they tell me that Python is a dialect of LISP.
(Score: 2) by maxwell demon on Tuesday May 13 2014, @08:11AM
I have no idea whether your friends are right, but even if they are, actually writing a Python-to-Lisp translator would still be a lot of work. Probably more work than justified by the gain it would give them. Especially if their programs just don't need that runtime optimization. For example, speeding up code that spends most of its time waiting for I/O is generally useless.
The Tao of math: The numbers you can count are not the real numbers.
(Score: 1) by Wootery on Tuesday May 13 2014, @11:16AM
I've never heard this claim (but I don't know many proper Python fans).
There's a lot of good stuff in Python, but transformability? No. LISP-dialect? No, of course not.
Next time they try it, point them at the failed Unladen Swallow project.
(Score: 3, Insightful) by maxwell demon on Tuesday May 13 2014, @08:04AM
I don't get why they characterize "forgiving" as a virtue of dynamic typing. I don't want my language to be forgiving. I prefer to be told quite from the start that I'm trying to do nonsense, not some time later when my code fails at run time.
Dynamic typing has advantages, but "forgiving" is not one of them. Forgivingness is a disadvantage. It's a price you pay for the flexibility dynamic typing gives you. It's often a price worth to pay, but it's still a cost, not a benefit.
The Tao of math: The numbers you can count are not the real numbers.
(Score: 0) by Anonymous Coward on Tuesday May 13 2014, @08:30AM
Exactly. Add on to that the fact if you write 'forgiving' code, you might get to meet your unforgiving boss or his boss!
(Score: 0) by Anonymous Coward on Tuesday May 13 2014, @10:24AM
But ... I'm so proud of my forgiving password check. It not only tolerates typos, it even tells you how to correct them! ;-)
(Score: 1) by romlok on Tuesday May 13 2014, @09:03AM
On that specific point, they don't. The comment about "forgiving" is a separate sentence to that regarding dynamic typing. They are saying that Python in general is "forgiving".
Whether "forgiving" is a good thing or not I couldn't really say in this case, since I'm not sure in what ways the author means that Python is forgiving (and can't really think of any myself).
(Score: 0) by Anonymous Coward on Tuesday May 13 2014, @01:51PM
Well, it certainly isn't forgiving about changes of indentation. ;-)
(Score: 3, Insightful) by wonkey_monkey on Tuesday May 13 2014, @08:28AM
It's pining. For the fjords.
systemd is Roko's Basilisk
(Score: 3, Interesting) by hankwang on Tuesday May 13 2014, @09:00AM
One problem of Python that I didn't see mentioned is that of parallellization to make use of multiple cores. See Jeff Knupp, Python's Hardest Problem [jeffknupp.com] for a thorough explanation.
TL;DR: a fundamental issue with the Python interpreter is that it is not suitable for parallel execution. Only data crunching that does not involve running the interpreter (e.g., data crunching by compiled code) can be executed effectively in parallel.
Avantslash: SoylentNews for mobile [avantslash.org]
(Score: 3, Informative) by skullz on Tuesday May 13 2014, @04:39PM
This really is the largest problem with the CPython implementation. The JIL doesn't exist in Jython and I don't think in IronPython (.NET) but lingers on in the C interpreter.
The Multiprocessing package gets around this by spawning a real live Python process and then lets the OS handle the parallelism and that works very well. Python doesn't yet have a nice proper threading package without the lock.
(Score: 2) by TheLink on Tuesday May 13 2014, @06:35PM
I figure the real reason why Python is slow is that nobody is throwing lots of decent programmers and resources at making it fast:e rformance_evolution [wikipedia.org]
https://en.wikipedia.org/wiki/HipHop_for_PHP [wikipedia.org]
https://en.wikipedia.org/wiki/JavaScript_engine#P
http://benchmarksgame.alioth.debian.org/u64/benchm ark.php?test=all&lang=v8&lang2=python3&data=u64 [debian.org]
That said it's strange it's slower than normal PHP: http://benchmarksgame.alioth.debian.org/u64/benchm ark.php?test=all&lang=php&lang2=python3&data=u64 [debian.org]
The benchmarks are old but I think my point still remains true :).