I´m unsuccessfully trying to parallelize your “Analyze” component to speedup the analysis of a list of hundreds (or even thousands) of simple structures by following your scripting example (2.4.1: Cross section Optimization | Scripting Guide 2.2.0) and using Parallel.For from System.Threading and System.Threading.Tasks
Despite I apparently allocate or copy all the variables required inside the Parallel.For loop, the C# script crashes when I use more than “1” in “N threads” slider… in fact even Rhino closes dumping RhinoCrashDump files.
The most relevant part of the script ParallelAnalyzeThI.gh (658.5 KB) is the following:
// Maximum number of concurrent threads (running at the same time)
var options = new ParallelOptions() { MaxDegreeOfParallelism = Npar };
// for(int i = 0; i < Model.Count; i++)
Parallel.For(0, Model.Count, options, i =>
{
int id = Thread.CurrentThread.ManagedThreadId;
Print("Model " + i + " computed by thread " + id);
var k3d = new KarambaCommon.Toolkit();
List<double> max_disp;
List<double> out_g;
List<double> out_comp;
string message;
var myModel = Model[i] as Model;
// avoid side effects
var model = myModel.Clone();
model.cloneElements();
model = k3d.Algorithms.AnalyzeThI(model, out max_disp, out out_g, out out_comp, out message);
Print("Thread " + id + " max_disp= " + max_disp[0].ToString(prec) + " out_g= " + out_g[0].ToString(prec) + " out_comp= " + out_comp[0].ToString(prec));
});
Hello @Vigardo,
I think you need to make sure that the C++ model is copied as well. Try to add “model.buildFEModel();” in your code after cloning the C# model.
The C++ routines of K3D are parallelized already, so adding another parallel loop will probably not make it work faster.
– Clemens
I added buildFEModel after cloning and before AnalyzeThI but it did not work… ParallelAnalyzeThIv2.gh (659.3 KB)
// avoid side effects
var model = myModel.Clone();
model.cloneElements();
model.buildFEModel(); // Clemens´ suggestion
model = k3d.Algorithms.AnalyzeThI(model, out max_disp, out out_g, out out_comp, out message);
Even the LineToBeam, Assembly and AnalyzeThI components? When I feed them (with a tree of thousands of configurations) the CPU utilization barely reaches one core (using Windows Task Manager), i.e. a 25 % in my 4 core old i7-6700K.
Perhaps parallelization within each model calculations (parallelizing stuff for each model individually) may not be very effective for very small systems as mine. This is why I´m trying to parallelize the set of thousands of models as a whole, e.g. splitting the 100 models task into 4 batches of 25. This way parallelization effectiveness should be close to 100 %.
I agree that LineToBeam and model assembly could profit from processing multiple models. The AnalyzeThI component probably not much for the reasons explained above.
The internalization of the models in your script does not work between different K3D versions. Could you add the gh-part that produces them? Then it is easier to find out what the problem might be.
– Clemens
I´m sorry, but I can´t share the gh-part that produces the models. They are just very simple ones. They are made of just a few bars (e.g. 5-10) as you can see in the following example:
I´ve internalised again a few (one hundred) models but using the latest versions of Karamba3D and GH. (i.e. using Karamba3D_2_2_0_180_RH7.msi file and GH version 7 SR33 7.33.23248.13001, 2023-09-05). ParallelAnalyzeThIv3.gh (877.2 KB) I hope this helps!! Sorry for the inconveniences.
I´ll try to take a look to your AnalyzeThI code using “ILSpy”, but I bet it is parallelised either on the linear solver side and/or the for loops where system matrices are assembled. In my opinion (which may be wrong), despite such parallelisation is perfect for analysing large systems (ideally reaching 100 % CPU utilisation), for analysing many smaller ones (just a few bars but thousands of models) may not be so good (low CPU utilisation e.g. 25 %). I apologize if I´m wrong.
This is why I´m trying to parallelise your component in such a way that each thread fully processes one model (not just a part of it). This way several models are fully processed at the same time, but just one per thread, ideally using all available threads.
Back to the Rhino crash problem, would it be related to some kind of collision between both parallelisation approaches? In such case, would it be possible to disable the internal parallelisation of k3d.Algorithms.AnalyzeThI? Perhaps some boolean variable may be added in future versions to deliberately disable it.
Hi @Vigardo,
I agree, for small structures your approach may result in a speedup.
In theory the internal parallelization of K3D should no impact any outer parallel loops.
I will have a look what causes the problem.
– Clemens
Hi @cp1 or @Vigardo did either of you ever have luck finding out what makes Rhino crash? At least in version 2.2 of Karamba this still seems to be a problem.