Hops running on headless Grasshopper instances - Problem with Memory

Hello Everybody,
maybe @stevebaer or @AndyPayne ?

I managed to run a bunch of headless Grasshopper solutions on async Tasks so they are solved without blocking the main UI. :sweat_smile: Now I’m trying to take Hops into this (poisonous) mix to solve long running solutions on additional Threads and possibly use network machines in future.

When using the Grasshopper Document without Hops in it, I’m expiring the solution before its Task is finished. The Document is being collected by Garbage collector - and memory is being freed up.
When moving the heavy part of the same Document into a Hops Component, it seems that something keeps the Document from being put to garbage - memory isn’t freed up.

Following is my Test Command which asks for the number of tasks to run and whether tese should use the Grasshopper Document with Hops or without. By running the Command a few times I can see my memory getting blocked up successively when using the Hops solution.

Probably it’s visible that I’m not a very seasoned programmer - and especially with this async stuff I’m sure I’m making quite a few mistakes - probably this is the reason why my system’s memory keeps getting filled up.

I would be very glad if someone could point me in the right direction on how to solve this issue!

using System;
using System.Collections.Generic;
using System.Threading.Tasks;
using Rhino;
using Rhino.Commands;
using Rhino.Input;
using Grasshopper.Kernel;
using GH_IO.Serialization;

namespace GrasshopperInRhino
{
    public class GHInRhinoTest : Command
    {
        public GHInRhinoTest()
        {
            Instance = this;
        }

        public static GHInRhinoTest Instance { get; private set; }

        public override string EnglishName => "GHInRhinoTest";

        protected override Result RunCommand(RhinoDoc doc, RunMode mode)
        {
            WaitingTask();
            return Result.Success;
        }

        static async Task WaitingTask()
        {
            int numberOfTasks = 2;
            bool useHops = false;
            RhinoGet.GetInteger("How many parallel tasks?", false, ref numberOfTasks);
            RhinoGet.GetBool("Use solution with Hops?", false, "No", "Yes", ref useHops);

            List<Task> tasklist = new List<Task>();

            for (int i = 0; i < numberOfTasks; i++)
            {
                tasklist.Add(WorkerTaskAsync("Task" + i.ToString(), useHops));
                System.Threading.Thread.Sleep(200);
            }

            await Task.WhenAll(tasklist);
        }

        static async Task WorkerTaskAsync(string name, bool useHops)
        {
            var start = DateTime.Now;
            RhinoApp.WriteLine("Enter {0}", name);

            string path = "GHInRhinoTest.gh";

            if (useHops)
            {
                path = "GHInRhinoTest_Hops.gh";
            }

            await Task.Run(() => SolveGrasshopper(path));

            RhinoApp.WriteLine("Exit {0}, Elapsed Time: {1}", name, (DateTime.Now - start).TotalMilliseconds);
        }

        public static void SolveGrasshopper(string grasshopperFilePath)
        {
            // Set GH File and get Definition
            GH_Document definition = new GH_Document();
            GH_Archive archive = new GH_Archive();
            archive.ReadFromFile(grasshopperFilePath);
            archive.ExtractObject(definition, "Definition");

            // Loop trough Grasshopper objects and compute all Context Bake Components
            foreach (GH_DocumentObject ghObj in definition.Objects)
            {
                if (ghObj is IGH_Component comp)
                {
                    if (comp.Name == "Context Bake")
                    {
                        IGH_Param contextBakeInput = comp.Params.Input[0];
                        contextBakeInput.CollectData();
                        contextBakeInput.ComputeData();
                    }
                }
            }
            definition.ExpireSolution();
        }
    }
}

I’m using Mass Addition of a series of 5.000.000 small floats to simulate a heavy calculation in my Grasshopper definitions.

Here are my files:
GHInRhinoTest.gh (8.0 KB)
GHInRhinoTest_Hops.gh (8.4 KB)
GHInRhinoTest_HopsContent.gh (4.8 KB)

GHInRhinoTest.cs (2.8 KB)

Thank you!

Hello again,

I think I am one step further as I tried to kick all async processes out of my solution - but I’m still seeing the same loss of free memory as before. Hence the change of this thread’s title.

Here is my adapted code and once again my files - I also had to adjust GHInRhinoTest_Hops.gh as my fancy auto-pathfinder for the Hops path didn’t seem to work the way I intended.

Once more I would be very happy if someone could guide me the right direction on how to keep system memory from blocking up…

Thanks a lot!

using System;
using System.Collections.Generic;
using System.Threading.Tasks;
using Rhino;
using Rhino.Commands;
using Rhino.Input;
using Grasshopper.Kernel;
using GH_IO.Serialization;

namespace GrasshopperInRhino
{
    public class GHInRhinoTestSync : Command
    {
        public GHInRhinoTestSync()
        {
            Instance = this;
        }

        public static GHInRhinoTestSync Instance { get; private set; }

        public override string EnglishName => "GHInRhinoTestSync";

        protected override Result RunCommand(RhinoDoc doc, RunMode mode)
        {
            //WaitingTaskAsync();
            WaitingTask();
            return Result.Success;
        }

        static void WaitingTask()
        {
            int numberOfTasks = 2;
            bool useHops = false;
            RhinoGet.GetInteger("How many parallel tasks?", false, ref numberOfTasks);
            RhinoGet.GetBool("Use solution with Hops?", false, "No", "Yes", ref useHops);

            List<Task> tasklist = new List<Task>();

            for (int i = 0; i < numberOfTasks; i++)
            {
                string name = "Task" + i.ToString();
                var start = DateTime.Now;
                RhinoApp.WriteLine("Enter {0}", name);

                string path = "GHInRhinoTest.gh";

                if (useHops)
                {
                    path = "GHInRhinoTest_Hops.gh";
                }

                SolveGrasshopper(path);

                RhinoApp.WriteLine("Exit {0}, Elapsed Time: {1}", name, (DateTime.Now - start).TotalMilliseconds);
                System.Threading.Thread.Sleep(200);
            }
        }

        public static void SolveGrasshopper(string grasshopperFilePath)
        {
            // Set GH File and get Definition
            GH_Document definition = new GH_Document();
            GH_Archive archive = new GH_Archive();
            archive.ReadFromFile(grasshopperFilePath);
            archive.ExtractObject(definition, "Definition");

            // Loop trough Grasshopper objects and compute all Context Bake Components
            foreach (GH_DocumentObject ghObj in definition.Objects)
            {
                if (ghObj is IGH_Component comp)
                {
                    if (comp.Name == "Context Bake")
                    {
                        IGH_Param contextBakeInput = comp.Params.Input[0];
                        contextBakeInput.CollectData();
                        contextBakeInput.ComputeData();
                    }
                }
            }
            definition.ExpireSolution();
        }
    }
}

GHInRhinoTest.gh (8.0 KB)
GHInRhinoTest_Hops.gh (7.1 KB)
GHInRhinoTest_HopsContent.gh (4.8 KB)

GhInRhinoTestSync.cs (2.6 KB)

Have you tried this with all of the caching turned off in hops? Hops tries to store results of calculations for future use so you can get quicker results when doing something like moving a slider back and forth.

Hi @stevebaer ,

I did indeed switch off both “Chache In Memory” and “Cache On Server” on the Hops Component.
Is there any other place where cache could be turned off?

Thank you!

That should be enough. You can look in preferences to see if anything is actually getting cached

“0 items in cache” is what I also get when opening GHInRhinoTest_Hops.gh
So caching should not be the cause of my issue, right?

Sounds like that isn’t the issue then. I was hoping this would be the cause as I think this will take a lot of effort to get to the bottom of.

Are you crashing due to running out of memory?

Thank you @stevebaer, good point!

As each Grasshopper solution takes around 5 seconds to solve and shovels a few hundred MB on the memory stack, it takes quite a while for my 64 GB to fill up. And as soon as that happens, Windows memory management seems to kick in and probably uses Virtual Memory to fill the gap (at least that’s how I interpret it from watching the memory stats on the task manager…)

But around 99% of physical memory keep being filled up until I quit the Rhino Instance - no matter how long I wait. I would interpret that as an evidence that none of those headless Grasshopper instances I opened to calculate all those solutions gets garbage collected and it’s memory gets freed up…

That’s quite different with the solution without Hops, where a few moments after one solution is finished and the next one started, the memory used for this solution is freed up again (probably as soon as the solution is expired and garbage collector “arrived” :slight_smile: ).

If that helps anything, here is a package with my complete solution, the compiled plugin and once again the three Grasshopper files. To make it run, you would have to place the Grasshopper files on the Desktop (for simplicity’s sake I search for them on Environment.SpecialFolder.Desktop). Befor starting up the plugin, you would have to change the file path of the Hops component in GHInRhinoTest_Hops.gh - I didn’t manage to get Hops working with a relative path based on the solution it is placed in (… yet? :wink:).

GrasshopperInRhino.zip (73.8 KB)

Thanks a lot for your efforts, Steve!

Hey Romio82. Just chiming in here. I just downloaded the zip file. Installed the plugin and set the Hops path for the GHInRhinoTest_Hops.gh file. I then ran the GHInRhinoTestAsync command and selected Yes to use Hops and set it to run 2 parallel tasks. This is what was returned:

Enter Task0
Enter Task1
Exit Task0, Elapsed Time: 413.7022
Exit Task1, Elapsed Time: 206.3948

Is that what you expected?

Hi Andy,

thanks for chiming in!

That actually is what’s expected… The Plugin starts two new headless instances, tries to solve the definition and reports the time (in ms) it took for solution when each task is finished.

For now I would actually want to solve my issue using the GHInRhinoSync command as I’m not completely sure if the async one is running correcly (Async = more difficult debugging - at least for me)… Sorry for not stating this more clearly before… :sweat:

If you have a look at your Task Managers Memory graph while executing GHInRhinoSync with Hops, you will see that each task shovels a few hundred MB of Memory on the stack, without releasing it afterwards. If you run 20 or 30 Tasks, it piles up quite a bit.

In contrast when using GHInRhinoSync without Hops, Memory gets freed up shortly after each task finishes.

Do you see that on your side as well?

Thank you very much and sorry for the inconvenience with the two commands…

Ok. Yes, when I run GHInRhinoSync with caching turned off for Hops, I did notice a steady increase in memory usage which doesn’t go back down after the command is run. So clearly something is happening here. I don’t exactly know when Steve or I will be able to jump into your code but we’ll try to do our best.

All right. In the meantime, thanks a lot to both of you for getting involved!
Looking forward to hearing from you whenever you find some time…
Thank you!

Hey @romio82 I did look a little more at your code. I’m not much closer to figuring out why Hops is retaining so much memory, however, I did modify your SolveGrasshopper function to look like this:

public static void SolveGrasshopper(string grasshopperFilePath)
{
    var io = new GH_DocumentIO();
    if (System.IO.File.Exists(grasshopperFilePath))
    {
        io.Open(grasshopperFilePath);
       var ghFile = io.Document;
       if (ghFile == null) return;

       Result rc = Result.Success;
       using (ghFile)
       {
           var script = new GH_RhinoScriptInterface();
           rc = script.RunAsCommand(ghFile, null, RhinoDoc.ActiveDoc, RunMode.Scripted);
        }
        RhinoDoc.ActiveDoc.Views.Redraw();
    }
}

Like I said, I don’t think this fixes your issue, but I think this is closer to how you might script Rhino to solve a Grasshopper player (similarly to the GrasshopperPlayer command).
Also, simply for documentation sake, I’ve attached a screenshot of the memory usage while running the GHInRhinoSync command. You can see there’s a steady increase in memory usage while it’s calculating each iteration. However, once I closed the rhino.compute console, that memory is freed up again. Perhaps @stevebaer has some insight here.
Memory

1 Like

Hey @AndyPayne,

thank you for the update!
That’s a very nice way of handling a headless Grasshopper solution indeed!
I guess you can use script.AssignDataToParameter to edit inputs and script.BakeDataInObject to retrieve outputs, right? That’s very elegant!

Of course I had to try and see now if using script.CloseDocument() to force Grasshopper to free up the memory used for the solution, but with no success so far…

But thank you for the tip with GH_RhinoScriptInterface, it’s in any case great to know how to do headless Grasshopper solutions properly!

Rhino instances take up a lot of memory. I would assume a lot of that memory is just due to running of headless Rhinos.

Hey Steve,

probably I’m missing something, but as far as I understand it, I’m just running headless Grasshoppers on my Tasks… Are those always starting new Rhinos in the Background?

Anyway, I’m running exactly the same headless Task when using the Grasshopper file without Hops - and in that case, the headless instance is closed after finishing calculation and memory is getting freed up again. When there is a Hops component in there, that is not the case.

And that seems to be the core of my question: Why doesn’t that happen when a Hops component is in use?

Hops launches and uses new Rhinos in the background

Ah, I see, thanks for clarification!
So the problem seems to be that those Rhinos don’t close down after finishing their respective solution?