Compiled ghpython script including treehelpers functions: RAM issue during optimisation process

Grasshopper Developer
1

I’ve compiled a set of components for acoustics analysis via ghpythonlib. So far so good. The components worked as expected until I attempted running an optimisation process. I’ve noticed that the RAM got devoured by Grasshopper after the very first iterations.

After debugging I’ve discovered that a call of ghpythonlib.treehelpers.list_to_tree() was the culprit.
For some reason, it seems that .list_to_tree() creates multiple instances of the same data tree object rather than overriding the last one. Weirdly, this happens only with the compiled version of the script. That is, if list_to_tree() is called within a GhPython component that has not been previously compiled, there is no RAM issue.

Any clue?

2

Strange : are you able to delete the previous object with del my_tree_object after you have used it as a workaround ?

Graham

3

Hi Graham. The list_to_tree function manages Data Trees internally, so it was impossible for me to explictly create an empty DataTree before populating it with data. However, I’ve also tried with recreating the list_to_tree function by using the Grasshopper modules, and the issue persists. There must be a problem with how compiled components manage the RAM.

4

@mirrag We’ve seen such a type of problem with some earlier versions of IronPython – this does not have anything to do with Rhino, but it’s something internal to the Python interpreter. We have also never see that in that particular function. Can you send a sample that gives you trouble? Will any sample repeat the issue?

Thanks,

Giulio

–
Giulio Piacentino
for Robert McNeel & Associates
giulio@mcneel.com

PS: The source is here: Transforms DataTrees in Grasshopper to nestings of lists, and vice versa · GitHub
as well as in:
%appdata%\McNeel\Rhinoceros\6.0\Plug-ins\IronPython (814d908a-e25c-493d-97e9-ee3861957f49)\settings\lib\ghpythonlib\ on your system.

6

@piac I apologise for getting back on this so late. In the past couple of days, I had a chance to dig a bit deeper into this issue and found out that the memory leak was not caused by the list_to_tree() function. Rather, it was exclusively triggered by the iterative process itself (recompute with Timer or Galapagos/Octopus). By trial and error, I found that by placing an empty Ghpython component in the Grasshopper definition and by rerunning an iterative process two things happen: (1) the RAM is emptied and (2) no more memory leaks happen.

I’ve tried literally everything to solve the issue with no success. Any suggestion would be extremely appreciated.

Unfortunately, I wasn’t able to reproduce the issue with a sample code, as the scripts that were compiled are quite complex. However, I’m willing to privately send you the scripts.

Thanks,
Gabriele

7

Hi @mirrag I am quite certain that this does not have anything to do with Rhino, but it’s part of the intrinsic problems of using a Garbage Collector (GC), which is internal to IronPython.
In order to address this, one way could be to invoke the GC manually. Before returning from the most affected components, try adding this:

import System
System.GC.Collect(System.GC.MaxGeneration, System.GCCollectionMode.Forced, True, True)
8

Hi @piac, thanks for the prompt reply. I’ve already tried invoking GC from the python gc module. I’m not sure if using System.GC.Collect would make any difference. In my attempt, I forced garbage collection in one of the component’s solveinstance function (before compiling), and was able to maintain the RAM within reasonable limits, yet the execution time skyrocketed. Of course, invoking gc at every iteration is computationally expensive.

I’ll give it a try anyway. Where do you suggest invoking System.GC.Collect? before building the component classes?

9

It doesn’t really matter. Just make sure that it runs only once per entire solution.

10

Hi @piac, I tried using GC.Collect with no success. Here are (some of) the scripts used for the compiling process. It would be great if you could give them a look.
Test.rar (16.0 KB)

11

Hi Giulio, I was wondering if you’ve had a chance to look at the code. I keep on thinking about why placing a ghpyton component within the grasshopper canvas solves the RAM issue. Is there anything that might be triggered by an empty ghpython component in terms of GC that does not happen automatically with the compiled components? Any suggestion would be appreciated.

12

@piac I am posting here after more than a year since I’ve been running into the same issue and couldn’t find a clean way to fix the memory leak with compiled components. I cannot figure out why putting an empty ghpython component in the grasshopper document solves the RAM problem. Something must be happening under the hoods that I’m utterly unaware of. Can you please provide me with any feedback? Thanks.