Rhino Compute service memory leak

Rhino Developer compute.rhino3d
1

Hi, I’m developing a application service using Rhino.Compute
and reference : https://github.com/mcneel/compute.rhino3d/blob/master/src/compute.geometry/ResthopperEndpoints.cs

During service startup, it reads grasshopper file and extract definition (type : GH_Document).
Then when api request occurs, it receives input data and compute related objects using input.
Since the service reuses same definition without calling Dispose() method, memory increases for each time when api request is handled.

As a temporary solution, I made my service restart after certain number of api request calls.

From memory profiling result, it says Grasshopper.Kernel.GH_DocumentServer.DocumentRemovedEventHandler
object is not released, which causes memory increase.
Is there a way to reuse same definition object without calling Dispose()?
Or any way to release object causing memory leak?

3

Hey @evan12,

Dropping in to say I’m experiencing similar issues. I’m putting a test project together for @stevebaer or whoever else to look at. Attempting to resuse a single GH_Document instance hasn’t quelled the problem.

@evan12 It would be helpful to compare notes. Could you upload a test project as well here?

4

So it appears that GH_Document isn’t able to be fully disposed of because of events that haven’t been unregistered. I used ANTS to profile:

image

I also saw a malignant event GumballPreview something something.

5

Thanks Leland, I’ll start digging into this.

6

I’m having trouble figuring out how to replicate this so if you have any sort of test project I would appreciate it.

7

@stevebaer:

Put this project together that simulates the key aspects of how we use resthopper:

  • sending a single large json payload and injecting it into the definition
  • sending bundles of payloads which is simulated by the console app

The big string sent here is a dummy lorem ipsum passage. Every time a new request is sent, memory builds and builds and seems not to be released. Memory allocation does wind down eventually, but when a new request is made it balloons back up to essentially where it left off.

To run:

  • build the geo service and the test app (send-json-gh-request)
  • start the geometry service, then run the test app
  • you can configure the test app to send as many serial requests you want.

Amendments to canonical copy of resthopper:

  • fixed string input conversion in ResthopperEndpoints
  • changed location of Rhino WIP from Rhino 7 WIP to Rhino WIP. You may need to change this back on your end.

The gh document is embedded in the test request as b64 string and packaged with the attached project.

compute.rhino3d.zip (1.3 MB)

8

Thanks; I’ll take a look

9

Hey @Leland_Jobson,
I think I found where the memory is stacking up. Try placing the var definition = new GH_Document(); in a using statement

This fix has been added to the compute repo

10

Steve,

Appreciate the look - thank you! I have tried this solution in our own implementation with no luck…are you seeing memory being released on your end when you try this?

11

Yes, I was just watching task manager and it looked like explicitly forcing a Dispose() call through the using statement was keeping memory usage stable. I could crank up the number of tests with your sample app (I had it go through 10 calls).

I wouldn’t be surprised if memory was being kept around for your more “real world” use case, but at least this fix is a step in the right direction.

12

Hi, as Leland described, applying fixed code doesn’t seem to solve the problem.
(Actually, I was already calling GH_Document.Dispose() explicitly at the end of resthopper call, which didn’t solve the problem)

GC takes memory down to certain level, but this level slightly increases for each api call.

Executing resthopper api call 200 times makes execution time twice longer in average.

13

Are you passing curves, surfaces, breps, or meshes as input ? Or having these returned as output? I see another spot where we aren’t properly disposing GeometryBase instances when the grasshopper endpoint is complete. I can try to fix that today.

14

Hey @Leland_Jobson, maybe try running ants profiler again with the change to Dispose() the document to see if you get different results.

I’m also planning on visiting you guys next week. Maybe we can spend a few minutes on this topic.

15

Steve,

Great news - thanks for looking into this. When I get a chance to load test the server I will be sure to make a profile. I’ll try to have something to look at for when you get here. Cheers!

16

Hi @stevebaer,

We are still having the same problem with running resthopper on our server. Each new command (call for a grasshopper script) adds more memory use. Is there a standard method to solve this problem?

17

Are you running out of memory? Compute caches solve results to try and improve performance when the same input is sent. The cache uses a standard memory cache class in .NET that should remove items as memory gets constrained. This caching is only performed when you pass “cachesolve=true” as part of your input json for a solve.

18

My colleague is busy with trying to understand what is going on concerning the memory issue. Het made an issue on Github: Memory leak/issue · Issue #396 · mcneel/compute.rhino3d · GitHub

This is his question:

There seems to be a big memory leak in compute.geometry relating to the inputs. I use Python to run Grasshopper scripts on a local compute server. I used this code to test it:

import compute_rhino3d
import compute_rhino3d.Util
import compute_rhino3d.Grasshopper as gh
import os from tqdm import tqdm file_directory = os.path.dirname(os.path.realpath(file))

GrasshopperScript_filepath = os.path.join(file_directory,“Grasshopper_Library/test.gh”)

rhino_URL = “http://localhost:8081/”
compute_rhino3d.Util.url = rhino_URL

Grasshopper_Inputs = {
“InputString”: ‘’.join([‘a’]*1000000),
“InputNumber” : 5 } variable_trees = [ ]
for

  • variable_name in Grasshopper_Inputs:
  • variable_value = Grasshopper_Inputs[variable_name]
  • variable_tree = gh.DataTree(variable_name)
  • variable_tree.Append([0],[variable_value])
  • variable_trees.append(variable_tree)
    for x in tqdm(range(1000)):
  • compute_output = gh.EvaluateDefinition(GrasshopperScript_filepath,variable_trees)`

It doesn’t really do much besides just running a GH script over and over again. The test.gh script is just:

image

As you can see, nothing happening except getting and outputting variables.

After I run this I see:

image

Which is quite hefty! We like leaving our compute server up and running continuously, so this becomes a problem after a while.
It seems the effect is directly related to the size of the inputs. In my Python example you can adjust the size of the inputs to see how it affects the memory. Small inputs have basically no effect, while large inputs can make it balloon rather quickly. So I’m assuming the problem is there.

Perhaps the inputs aren’t being cleared? I prodded around in the source code to see if I can find an obvious mistake, but unfortunately I’m no C# expert and couldn’t find anything.

Perhaps I’m doing something wrong on my side?

19

The previous input is cached on the server to help decide if parameters need to be reset and completely recomputed. If you call grasshopper again with a smaller input string, the previous massive string should get discarded and made available to the GC.

20

Hi @stevebaer, would there be a way to disable caching the input? Would this completely prevent Erik’s problem of increased memory usage over time? (at the cost of computing every iteration, even if the same parameters are used)

21

Hi, you found any solution? because I have the same problem I want to implement a solver like galapagos but out of grasshopper and when I using the Rhino Compute server when I do a lot of calls the memory of server increments a lot.