September 11, 2007

Benchmarking Parallel Python Against Threading

Follow-up to Benchmarking Parallel Python from The Utovsky Bolshevik Show

Having had it pointed out to me that my last benchmarking post is fairly useless without a comparison to threading by a couple of people, I now have such a comparison.  The numbers for PP are those used in the last blog post.

For threads I initially tried using Christopher Arndt's threadpool module to make my life easier.  I've included these results in the table below and, looking at them, you can see why I thought had to find a different way of testing threads.

I decided to use Tim Lesher's cookbook recipe to retest threads.

The function used by all the methods is identical, so this should just be a measure of their performance.

Without further ado, the results: 


1 Worker
2 Workers
4 Workers
Parallel Python
1153s
601s
582s
threadpool
1176s
1246s
1254s
Cookbook Recipe
1175s
1238s
1362s


Obviously these results don't reflect brilliantly on threads.  What I did notice is that it was only Parallel Python that used more than 1 of my processors, which I presume is something GIL related.

Either Parallel Python is an excellent improvement over threads, or I'm doing something stupid regarding threads.  If the latter, please let me know and I'll run the benchmarks again.


- 8 comments by 1 or more people Not publicly viewable

[Skip to the latest comment]
  1. Asd

    Python only allows one thread to run at once. That is what the GIL does. So this test is entirely pointless.

    11 Sep 2007, 22:11

  2. This is true, but someone claimed in my earlier benchmarking of Parallel Python that it was meaningless without a threading comparison. So there you go. :p

    11 Sep 2007, 22:32

  3. Bill Mill

    The GIL only allows one pure-python thread to run at once, this is correct. However, C extensions can release the GIL (for example, socket.recv does). Quoting the FAQ:

    > Judicious use of C extensions will also help; if you use a C extension to perform a time-consuming task, the extension can release the GIL while the thread of execution is in the C code and allow other threads to get some work done.

    11 Sep 2007, 22:48

  4. Drew Vogel

    “a comparison with threading” does not mean threads of a python script. Rather it means with a multi-threaded python interpreter. The GIL in the stock python interpreter prevents any two threads of a python script from being executed simultaneously—regardless of how many CPUs or CPU cores in the hardware and regardless of how many threads the python script creates. Therefore python scripts run on the stock python interpreter are multi-threaded only in semantics, not in execution. Other python interpreters (e.g. JPython) are multi-threaded and thus will execute different threads of a python script simultaneously.

    Guido’s contention is that removing the GIL introduces complexity and locking overhead in the interpreter for potential performance increases that can be achieved by using separate processes that use some sort of IPC (instead of locks) for synchronization. Therefore the benchmark you should probably be running is the stock python interpreter versus JPython (or some other truly multi-threaded interpreter) executing this CPU-bound script and another script that is IO-bound (but be sure to prime the IO caches first).

    11 Sep 2007, 22:49

  5. Drew Vogel

    Btw, when I wrote this:

    “Therefore the benchmark you should probably be running is the stock python interpreter versus JPython (or some other truly multi-threaded interpreter) executing this CPU-bound script and another script that is IO-bound (but be sure to prime the IO caches first).”

    I meant that you should run the two scripts using the threading module on JPython while using Parallel Python (which implements a form of what Guido advocates) on the stock interpreter.

    11 Sep 2007, 22:53

  6. I meant that you should run the two scripts using the threading module on JPython while using Parallel Python (which implements a form of what Guido advocates) on the stock interpreter.

    Obviously I should have realised this first time.

    embarassed face

    I shall now benchmark properly. :D

    11 Sep 2007, 23:10

  7. There is now an updated benchmark comparing CPython, Parallel Python and threaded Jython at http://programming.reddit.com/info/2ocxx/comments

    11 Sep 2007, 23:58

  8. Drew Vogel

    The reason I suggest running an IO-bound test is because in theory the threads running on the stock python interpreter could see a performance increase due to one thread sleeping while the IO blocks, allowing the other thread to execute. This is the approach of the State Threads C library. I’m curious whether the stock python interpreter is able to facilitate this type of “parallelism” or whether the OS will put the entire python interpreter to sleep while the script waits on IO. Therefore be sure to publish what OS you run the benchmarks on. Also post the scripts so that other people can run them on other OSes.

    12 Sep 2007, 00:03


Add a comment

You are not allowed to comment on this entry as it has restricted commenting permissions.

Trackbacks

  1. Benchmarking Parallel Python Against Jython Threading (Benchmarks Take 3)

    Having had it pointed out to me that benchmarking against CPython threading is pointless, I am now going to do what I should have done originally (third time's the charm, right?) and benchmark Parallel CPython against threaded Java, in the hopes I will fa…

    The Utovsky Bolshevik Show - 11 Sep 2007, 23:55

September 2007

Mo Tu We Th Fr Sa Su
Aug |  Today  | Oct
               1 2
3 4 5 6 7 8 9
10 11 12 13 14 15 16
17 18 19 20 21 22 23
24 25 26 27 28 29 30

Search this blog

Tags

Galleries

Most recent comments

  • I should note that the talk info is available at http://linux.conf.au/programme/detail?TalkID=293 an… by on this entry
  • "I X'd you a Y, but I eated it by Lamby on this entry
  • Nice. Did not know it was that easy, I had a few problems to get it working some time ago. Have you … by mats on this entry
  • You can't make progress if you just argue, I try to be constructive. Cheers! by James on this entry
  • Thanks a bunch for doing this. I think that both Guido and Bruce Eckel are right; we really need peo… by Eli on this entry

Blog archive

Loading…
Not signed in
Sign in

Powered by BlogBuilder
© MMXIV