Parallelising DA.SetData

#1

Hi all,

This is a very specific question regarding parallelisation of the SetDataList method on the DA interface, so, can it be done or is this a stupid question?

As it stands, I’m getting good results on parallelising a number of methods on my components, often achieving 3-4x increase in performance, then the bottleneck becomes DA.SetDataList/Tree.

It is not uncommon that DA.SetData runtime gets 5-10x longer compared to the rest of the component. Are there any options to address this?

1 Like
(Steve Baer) #2

DA is not thread safe. We worked around this by breaking the data collection and data setting routines into two separate phases.

#3

Thanks, Steve.
I already knew about the multi-threaded components, but the TaskCapableComponent is news to me, I’ll definitely take a serious look into it to understand if it handles the SetData bottleneck efficiently.

(Dale Fugier) #4

Hi @aristocompasso,

A couple of more formal links:

https://developer.rhino3d.com/guides/grasshopper/multi-treaded-components/

https://developer.rhino3d.com/guides/grasshopper/programming-task-capable-component/

And some sample source:

https://github.com/mcneel/rhino-developer-samples/tree/6/grasshopper/cs/SampleGhTaskCapable

– Dale

#5

Thanks, Dale.

Nice to see you guys working on it. In my opinion, as Grasshopper is expected to perform ever more complex computations, multithreading is going to be a major requirement.

Maybe, there could be a multithreading component to confer inline multithreading capability to any component used in the definition ad-hoc :sunglasses:

Now, isn’t this a good idea?

(Cameron Newnham) #6

Hi @aristocompasso,
If outputting is taking a long time, I’m assuming that you’re outputting a large number of objects (thousands of points, etc).

You’ll likely get much better performance by directly setting the Grasshopper type instead of outputting the Rhino type. For example, instead of DA.SetData(0, MyPoint3dList); you could use DA.SetData(0, MyPoint3dList.Select(x => new GH_Point(x));

This avoids the Cast functions, which are very expensive when called that many times.

As an example, generating and outputting 100,000 random points takes 565ms using Point3d, or 19ms when using the Linq conversion to GH_Point. Discussion here.

From memory this applies to compiled components in just the same way as it does to Script_Instance, but please correct me if I’m wrong.

1 Like
#7

Hi Cameron, thanks for your input.
I was already aware that the Grasshopper.Kernel types have better performance than the Rhino.Geometry types.

It does apply, but I’m not sure that the difference is as noticeable because the compile components already offer better performance than the script components. To give you an idea on this specific component I tested the second case on your screenshot.

Rhino.Geometry Grasshopper.Kernel.Types
SetRgVector3d() completed 1915201 in 903ms SetGhVector() completed 1915201 in 356ms 39,4% of Rhino.Geometry
SetRgPoint3d() completed 1915201 in 942ms SetGhPoint() completed 1915201 in 401ms 42,5% of Rhino.Geometry

So, an improvement, but nowhere near the 28.5x magnitude. The good news is that the compiled component sets +/- 10x more points than the script component in the same amount of time.
I’m wondering, though, if using the Grasshopper.Kernel.Types throughout the component (calculation stages) has advantages over using the Rhino.Geometry types in terms of the overall performance of the component… I have to check it out.

(David Rutten) #8

Very unlikely. The Goo types just wrap around the base types so if you want to ‘use’ GH_Point instead of Point3d in your code, you really just have to call Goo.Value every. Single. Time.

#9

Hi David,
If that’s the case, you’re making good sense.

Now, here’s a fun fact for you guys… it’s only about 1300x difference!!

List-float- List-double-
SetFloatValues() completed 16384 in 1292ms SetDoubleValues() completed 16384 in 1ms 0.077% of List-float-
(Cameron Newnham) #10

That’s pretty interesting!

Another thing that I’ve found is that sometimes the perceived performance of components is not related to the IO directly, but rather in adding to the display pipeline (regardless of whether the component is hidden/visible).

One thing that might be worth trying (for testing only, to isolate issues) would be Rhino.RhinoDoc.ActiveDoc.Views.RedrawEnabled = false. Presumably this wouldn’t have any effect on numeric outputs, but on GeometryBase types I get a pretty major ‘performance’ increase.

Out of curiosity, how are you benchmarking these?

#11

Using standard System.Diagnostics.Stopwatch class;
I catch the specific runtime by isolating the DA.SetData instruction inside the Stopwatch start and stop.

        ...
        var watchSetData = Stopwatch.StartNew();
        DA.SetDataList(3, intersectionL.Select(x => new GT.GH_Point(x)));
        watchSetData.Stop();
        var stopwatch = watchSetData.ElapsedMilliseconds;
        m.Add($"SetGhPoint() completed {intersectionL.Count} in {stopwatch} ms");
        DA.SetDataList(0, m);