Benchmarking Parallel Python Against Threading
Follow-up to Benchmarking Parallel Python from The Utovsky Bolshevik Show
Having had it pointed out to me that my last benchmarking post is fairly useless without a comparison to threading by a couple of people, I now have such a comparison. The numbers for PP are those used in the last blog post.
For threads I initially tried using Christopher Arndt's threadpool module to make my life easier. I've included these results in the table below and, looking at them, you can see why I thought had to find a different way of testing threads.
I decided to use Tim Lesher's cookbook recipe to retest threads.
The function used by all the methods is identical, so this should just be a measure of their performance.
Without further ado, the results:
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
Obviously these results don't reflect brilliantly on threads. What I did notice is that it was only Parallel Python that used more than 1 of my processors, which I presume is something GIL related.
Either Parallel Python is an excellent improvement over threads, or I'm doing something stupid regarding threads. If the latter, please let me know and I'll run the benchmarks again.
11 Sep 2007 21:47
| 
Tags: Computer Free Multiprocessor Parallel Python Software
| 
Comments (8)
| 
Trackbacks (1)
| 
Report a problem
Loading…
8 comments by 1 or more people
[Skip to the latest comment]Asd
Python only allows one thread to run at once. That is what the GIL does. So this test is entirely pointless.
11 Sep 2007, 22:11
This is true, but someone claimed in my earlier benchmarking of Parallel Python that it was meaningless without a threading comparison. So there you go. :p
11 Sep 2007, 22:32
Bill Mill
The GIL only allows one pure-python thread to run at once, this is correct. However, C extensions can release the GIL (for example, socket.recv does). Quoting the FAQ:
> Judicious use of C extensions will also help; if you use a C extension to perform a time-consuming task, the extension can release the GIL while the thread of execution is in the C code and allow other threads to get some work done.
11 Sep 2007, 22:48
Drew Vogel
“a comparison with threading” does not mean threads of a python script. Rather it means with a multi-threaded python interpreter. The GIL in the stock python interpreter prevents any two threads of a python script from being executed simultaneously—regardless of how many CPUs or CPU cores in the hardware and regardless of how many threads the python script creates. Therefore python scripts run on the stock python interpreter are multi-threaded only in semantics, not in execution. Other python interpreters (e.g. JPython) are multi-threaded and thus will execute different threads of a python script simultaneously.
Guido’s contention is that removing the GIL introduces complexity and locking overhead in the interpreter for potential performance increases that can be achieved by using separate processes that use some sort of IPC (instead of locks) for synchronization. Therefore the benchmark you should probably be running is the stock python interpreter versus JPython (or some other truly multi-threaded interpreter) executing this CPU-bound script and another script that is IO-bound (but be sure to prime the IO caches first).
11 Sep 2007, 22:49
Drew Vogel
Btw, when I wrote this:
I meant that you should run the two scripts using the threading module on JPython while using Parallel Python (which implements a form of what Guido advocates) on the stock interpreter.
11 Sep 2007, 22:53
Obviously I should have realised this first time.
embarassed face
I shall now benchmark properly. :D
11 Sep 2007, 23:10
There is now an updated benchmark comparing CPython, Parallel Python and threaded Jython at http://programming.reddit.com/info/2ocxx/comments
11 Sep 2007, 23:58
Drew Vogel
The reason I suggest running an IO-bound test is because in theory the threads running on the stock python interpreter could see a performance increase due to one thread sleeping while the IO blocks, allowing the other thread to execute. This is the approach of the State Threads C library. I’m curious whether the stock python interpreter is able to facilitate this type of “parallelism” or whether the OS will put the entire python interpreter to sleep while the script waits on IO. Therefore be sure to publish what OS you run the benchmarks on. Also post the scripts so that other people can run them on other OSes.
12 Sep 2007, 00:03
Add a comment
You are not allowed to comment on this entry as it has restricted commenting permissions.