All entries for Tuesday 11 September 2007

September 11, 2007

Benchmarking Parallel Python Against Jython Threading (Benchmarks Take 3)

Follow-up to Benchmarking Parallel Python Against Threading from The Utovsky Bolshevik Show

Having had it pointed out to me that benchmarking against CPython threading is pointless, I am now going to do what I should have done originally (third time's the charm, right?) and benchmark Parallel CPython against threaded Java, in the hopes I will fail less at producing something useful.

Each of these results is the time it takes to sum the prime numbers below each multiple of 10000 between 100000 and 1000000 (i.e. perform the operation 90 times on numbers increasing by 10000 each time). 

I'm reusing the Parallel Python results from previously.

I decided to use Tim Lesher's cookbook recipe to test threads, as I already have a script which doesn't require a great deal of rewriting to make it Jython (i.e. CPython 2.2 or so) compatible.

Now, the results:


1 Worker
2 Workers
4 Workers
Vanilla CPython
1195s
N/A
N/A
Parallel CPython
1153s
601s
582s
Jython Threads
442s
241s
254s

As can be seen here, Jython threads by far and away beat Parallel CPython.  This does not, however, take into account the fact that Parallel Python can use several machines at once, which Jython threading obviously cannot do.

What's interesting to note is that Parallel CPython on one worker is roughly the same as standard GIL'd CPython (slightly faster, in fact, in this case).  If you need to write and deploy CPython as opposed to Jython, then there's no performance cost in writing parallelisable code to use Parallel Python regardless of end user (as PP, by default, spawns a number of workers equal to the available CPUs).

These statistics were taken on an IBM Thinkpad T60 with a Core Duo T2400 running Ubuntu Feisty GNU/Linux (using the standard packages where available) using the scripts found under http://oddbloke.uwcs.co.uk/parallel_benchmarks/ . 

Hopefully these are useful statistics and conclusions, as opposed to my previous efforts to produce such. :) 


Benchmarking Parallel Python Against Threading

Follow-up to Benchmarking Parallel Python from The Utovsky Bolshevik Show

Having had it pointed out to me that my last benchmarking post is fairly useless without a comparison to threading by a couple of people, I now have such a comparison.  The numbers for PP are those used in the last blog post.

For threads I initially tried using Christopher Arndt's threadpool module to make my life easier.  I've included these results in the table below and, looking at them, you can see why I thought had to find a different way of testing threads.

I decided to use Tim Lesher's cookbook recipe to retest threads.

The function used by all the methods is identical, so this should just be a measure of their performance.

Without further ado, the results: 


1 Worker
2 Workers
4 Workers
Parallel Python
1153s
601s
582s
threadpool
1176s
1246s
1254s
Cookbook Recipe
1175s
1238s
1362s


Obviously these results don't reflect brilliantly on threads.  What I did notice is that it was only Parallel Python that used more than 1 of my processors, which I presume is something GIL related.

Either Parallel Python is an excellent improvement over threads, or I'm doing something stupid regarding threads.  If the latter, please let me know and I'll run the benchmarks again.


The Bible In A Year: Day 11 (January 11th)

Follow-up to The Bible In A Year: Day 10 (January 10th) from The Utovsky Bolshevik Show

The reading for January 11th is Genesis 27-28 and Romans 9:16-33.

Genesis

To get you up to speed, Abraham has died and been buried with his wife, Sarah. Isaac and Rebekah have had male twins, Jacob and Esau. Esau, the elder, has sold his birthright to Jacob in exchange for some food. The birthright includes God’s promise to Abraham.

Chapter 27

In this passage, Jacob steals Esau’s ‘blessing’ from him by fooling a now blind Isaac into bestowing it upon him. The ‘blessing’ was an actual binding legal statement at the time, so Jacob has now stolen both Esau’s birthright and his blessing. As such, he will be the master of Isaac’s tribe, Esau included. Esau is, unsurprisingly, none too pleased about this. Rebekah tells Jacob to run away and tells Isaac that she doesn’t want him to marry a non-Israelite.

Chapter 28

1 So Isaac called for Jacob and blessed him and commanded him: “Do not marry a Canaanite woman.

Despite the trick Jacob has played on Isaac, Isaac will still bless him.

20 Then Jacob made a vow, saying, “If God will be with me and will watch over me on this journey I am taking and will give me food to eat and clothes to wear 21 so that I return safely to my father’s house, then the LORD will be my God 22 and this stone that I have set up as a pillar will be God’s house, and of all that you give me I will give you a tenth.”

Jacob finally stops referring to God as his father’s God (having just had God’s promise re-iterated to him). Also another instance of tithing.

Romans

Chapter 9

18 Therefore God has mercy on whom he wants to have mercy, and he hardens whom he wants to harden.
19 One of you will say to me: “Then why does God still blame us? For who resists his will?”

If God has already decided who will be saved and who will not, how can He blame those who will not be saved?

20 But who are you, O man, to talk back to God? “Shall what is formed say to him who formed it, ‘Why did you make me like this?’ ” 21 Does not the potter have the right to make out of the same lump of clay some pottery for noble purposes and some for common use?

Paul answers, somewhat unsatisfactorily, that we are God’s to do with as He will.

22 What if God, choosing to show his wrath and make his power known, bore with great patience the objects of his wrath—prepared for destruction? 23 What if he did this to make the riches of his glory known to the objects of his mercy, whom he prepared in advance for glory— 24 even us, whom he also called, not only from the Jews but also from the Gentiles?

Paul, however, goes on to say that God is a merciful God and so we should trust in what he is doing, even if it doesn’t make a great deal of sense to us.


Benchmarking Parallel Python

Writing about web page http://www.artima.com/weblogs/viewpost.jsp?thread=214303

This post is Bruce Eckel’s follow-up to his previous post which covered, among other things, concurrency within Python. Basically, CPython has the Global Interpreter Lock (GIL) which makes life very awkward for those wanting to run Python on more than one processor.

Anyhow, in this post Bruce points to Parallel Python as an add-on module which is a potential solution. I had a look at this and thought it was pretty cool. However, bearing in mind Guido van Rossum’s post about the performance implications of removing the GIL last time it was attempted I thought I’d see if this actually did provide a speed-up and benchmark it.

The following stats are for calculating the sum of primes below every multiple of 10000 between 105 and 106 (including the lower bound and excluding the upper). The first set uses only one working thread0 of my Core Duo laptop and the second set uses two (as I have two processors).

It should be noted that the code snippet being used is provided as an example on the Parallel Python website and so is probably one of their most optimal cases. Regardless, I think the numbers are helpful.

One Processor

Real Time Taken: 1153.53128409 s
Number of jobs: 90
Total Job Time: 1153.53128409 s
Time/Job: 12.816742

Two Processors

Real Time Taken: 601.201694012 s
Number of jobs: 90
Total Job Time: 1180.9738
Time/Job: 13.121931

It can be seen that running two worker threads increases the actual CPU time used by around 30 seconds but the fact that two processors are being used leads to a total speed up factor of 1.918709304, which is pretty impressive.

0 I’m not sure of the internals, so I don’t know if it is technically a thread. Regardless, only one calculation will happen at a time.


The Bible In A Year: Day 10 (January 10th)

Follow-up to The Bible In A Year: Day 9 (January 9th) from The Utovsky Bolshevik Show

Once again, I fail to read on time. I don’t even have the excuse of sleeping patterns this time, I was up a normal amount. Just lazy.

The reading for January 10th is Genesis 25-26 and Romans 9:1-15. As ever, today’s reading proper will follow later.


September 2007

Mo Tu We Th Fr Sa Su
Aug |  Today  | Oct
               1 2
3 4 5 6 7 8 9
10 11 12 13 14 15 16
17 18 19 20 21 22 23
24 25 26 27 28 29 30

Search this blog

Tags

Galleries

Most recent comments

  • I should note that the talk info is available at http://linux.conf.au/programme/detail?TalkID=293 an… by on this entry
  • "I X'd you a Y, but I eated it by Lamby on this entry
  • Nice. Did not know it was that easy, I had a few problems to get it working some time ago. Have you … by mats on this entry
  • You can't make progress if you just argue, I try to be constructive. Cheers! by James on this entry
  • Thanks a bunch for doing this. I think that both Guido and Bruce Eckel are right; we really need peo… by Eli on this entry

Blog archive

Loading…
Not signed in
Sign in

Powered by BlogBuilder
© MMXIV