Ghpythonlib paralell weird behavior (not paralelling the first thread)


(Daud) #1

Hey everyone,

when working with the ghpythonlib.paralell component I have found some weird behaviour. For me, the component does not run any threads paralell to the first thread. From the second Thread on, it perfectly parallels among my CPU power.

In the below example you can see, that no Thread is started in paralell to Thread 1, but after Thread1 has finished, there are always 5 thread running parallely.

Any help would be appreciated! Thank you!

(Graham) #2

I get the same printout (with minor changes each time after line 3). When I monitor the CPU usage on a more computationally intensive task I get the impression that my 4 cores are reaching full capacity at the same time but there is a lag between starting the script and all 4 going full speed so that could well be the time it takes for the first one to finish

“knock knock”
“race condition”

                           "who's there?"

(Daud) #3

Hi Graham,

thank you for your answer! It is nice, that you can confirm these results. I guess the minor changes after line 3 are exactly what we should expect; they even appear on my side if I recompute the script a few times. To be honest, I am not exactly sure, if I correctly understand your example. It would be awesome if you could explain a bit about it!

The component I am trying to paralell right now takes 5.2s and computes results for 9 branches. The result looks something like this:

Started Calculation on Branch 0
Finished Calculation on Branch 0
Started Calculation on Branch 1
Started Calculation on Branch 3

Basically these are in line with the results from the firstFurther evidence, that the first Branch isn’t getting computed parallely. Computing one branch in my example takes ~700ms. Calculating all nine Branches right now takes 5.2 second, which means I got some improvements through paralleling. However, if I enter these lines of code into my calculation function

if branchNumber == 0:
    print "Cancelled Thread 0!"

My calculation becomes exactly 700ms faster (exaclty the time it takes to calculate the first branch)! To my understanding this means, that nothing is getting computed paralelly to the first branch.

However, I am unsure if this is due to my bad usage of the component, due to a bug in the integration of the package, if it just is the behavior to be expected (for whatever reasons) or I am misinterpreting the results.

Thank you for any further advice on this!

(Graham) #4

That’s a nice workaround but not very satisfying in terms of understanding what’s going on.
The below script is a little old but if it stilll works then you could check whether it shows the same behaviour?

(Graham) #5

Hmm maybe that’s not helpful as it seems to be running a very large number of threads so the time for the first one is insignificant

(Daud) #6


I have already read Steve Baer’s blog post aswell. I will try out the example file though, to spot potential differences. Thank you!

(Daud) #7

Just an update from my side in case I get lucky and somebody finds this. I tested this in RH6 and the behavior is exactly the same.

Also, in practice you can sort of “fix” the impact of this bug(?) by including a dummy entry in your list and then skipping over that one in your calculations. Say you are having 10 entries that need to be calculated in your function calculateSolution(entry), that needs a minute to calculate each time. Then add a dummy in front of your list:
entries.insert(0, "dummy")
And then in your calculateSolution(entry) you just immediately return and waste no time on the first entry.
if(entry=='dummy') return

So that can potentially save you a minute. Anyways, this does not seem to be a proper solution and I am still looking for help on this!