Is Disposing Rhino.Geometry worth it?

Hello everyone, here is a story: @MateuszZwierzycki and myself had a discussion a couple of days ago about whether it makes sense to Dispose Rhino.Geometry objects, that do implement IDisposable
There is a school of thought that says, since GC takes such good care of everything, one is never obligated to cleanup after themselves (when working with managed resources).
There is a descent discussion here, but in order to decide in a real world situation, whether to implement IDisposable or not, we figured we should do some testing and see for ourselves what the difference is and is there any at all?
We were quite shocked, to be honest, to find such huge contrast in performance!
It was very unexpected and with this post we hope to make this topic less ambiguous, so it may influence design decisions and help do proper refactor and optimisation.

So, here is the scenario: you have a class that takes a mesh and performs some operations on it, but without affecting the input Mesh, in other words it duplicates the Mesh, which is the only thing that we care about in that example and it is the only thing we’re going to do here:

    public class MeshDuplicator
    {
        public Mesh Mesh { get; private set; }

        public MeshDuplicator(Mesh mesh)
        {
            this.Mesh = mesh.DuplicateMesh();
        }
    }

Now let’s have the same class, but implementing IDisposable:

public class MeshDuplicatorDisposable : MeshDuplicator, IDisposable
{
    public MeshDuplicatorDisposable(Mesh mesh) : base(mesh)
    { }

    public void Dispose()
    {
        base.Mesh?.Dispose();

        GC.SuppressFinalize(this);
    }

    // Finalizer (Destructor)
    ~MeshDuplicatorDisposable()
    {
        Dispose();
    }
}

Now let’s have some client code that creates a lot of instances of those two classes, we’ll have a small GH component that will allow us to control the amount of objects and carve various configurations in which we’ll be measuring computation speed and memory consumption:

    protected override void SolveInstance(IGH_DataAccess DA)
    {
        Mesh mesh = null;
        int iterations = default;
        bool dispose = default;

        if (!DA.GetData(0, ref mesh)) return;
        if (!DA.GetData(1, ref iterations)) return;
        if (!DA.GetData(2, ref dispose)) return;

        long startMemory = GC.GetTotalMemory(true);
        var sw = new Stopwatch();
        sw.Start();

        if (dispose)
        {
            for (int i = 0; i < iterations; i++)
            {
                using (var meshDup = new MeshDuplicatorDisposable(mesh))
                {
                    var facesCount = meshDup.Mesh.Faces.Count;
                    Console.WriteLine(facesCount);
                }
            }
        }
        else
        {
            for (int i = 0; i < iterations; i++)
            {
                var meshDup = new MeshDuplicator(mesh);
                var facesCount = meshDup.Mesh.Faces.Count;
                Console.WriteLine(facesCount);
            }
        }

        sw.Stop();
        DA.SetData(0, sw.ElapsedMilliseconds);

        long endMemory = GC.GetTotalMemory(true);
        long memoryUsed = endMemory - startMemory;

        DA.SetData(1, memoryUsed.ToString());
    }
}

There are two main factors at play here: the size of the Mesh and the amount of iterations. For each configuration there will be 1000 measurements, so the results actually mean something.
Since we are interested in a real world scenario I built the plugin in Release configuration (should help GC to perform at its’ best :wink: )
Setup: 32 GB of RAM, 64-bit Windows, Rhino 7


Configuration 1 (small Mesh, many iterations)

  • Mesh : Vertex count: 1724, Face count: 1764
  • Iterations count per measurement: 100000

Speed: With IDisposable was faster 100% of measurements

Results Dispose No Dispose
Best speed (milliseconds) 918 2311
Worst speed (milliseconds) 1004 4186
Average speed(milliseconds) 952.84 2651.449

With IDisposable Memory consumption was smaller in 96% of measurements

Results Dispose No Dispose
Smallest difference round (bytes) 8 40
Largest difference round (bytes) 32 88176
Average consumption (bytes) 281 3910.5

On average implementing IDisposable resulted in 2.78 times faster computation and almost 14 times smaller memory consumption.


Configuration 2 (small Mesh, few iterations)

  • Mesh : Vertex count: 1724, Face count: 1764
  • Iterations count per measurement: 100

Speed: With IDisposable was faster 100% of measurements

Results Dispose No Dispose
Best speed (milliseconds) 0 3
Worst speed (milliseconds) 1 31
Average speed(milliseconds) 0.211 3.988

With IDisposable Memory consumption was smaller in 100% of measurements

Results Dispose No Dispose
Smallest difference round (bytes) 872 3824
Largest difference round (bytes) 3952 9120
Average consumption (bytes) 1222.67 5288

On average implementing IDisposable resulted in 18.9 times faster computation and 4.32 times smaller memory consumption.


Configuration 3 (large Mesh, few iterations)

  • Mesh : Vertex count: 999002, Face count: 1000000
  • Iterations count per measurement: 100

Speed: With IDisposable was slower 100% of measurements

Results Dispose No Dispose
Best speed (milliseconds) 2118 1927
Worst speed (milliseconds) 2445 2072
Average speed(milliseconds) 2220.99 1996.94

With IDisposable Memory consumption was larger in 84.6% of measurements

Results Dispose No Dispose
Smallest difference round (bytes) 16 312
Largest difference round (bytes) 9464 1784
Average consumption (bytes) 2605.87 2531.73

On average implementing IDisposable resulted in 1.112 times slower computation and 1.03 times larger memory consumption.


The conclusion on implementing IDisposable when working with Meshes are:

  • it can boost your plugins’ performance
  • Disposing very large Meshes is a little less optimal than letting GC take care of that
  • testing Release might surprise you

We only did such thorough testing on Meshes, my guess is other types might be affected differently?
@DavidRutten @stevebaer @menno are there any advises you’d give to those of us, who are trying to achieve maximum performance, but in a sane sort of way?
And in real life scenarios, dear reader :upside_down_face:, do you tend to test the living hell out of your code or follow some best practices?

Best regards,
Kiryl

1 Like

I’d say, if possible to re-use a mesh, do it. Same for other geometry types.

A possible reason your code with IDisposable is faster, is that memory can be re-used that may still partially be in cache. I’d be interested to see this test but with random meshes being created, instead of meshes being duplicated.

I’m a bit puzzled by your Console.WriteLine statements. If you’re outputting stuff to any console, or writing to file or whatever, this may impact the overall execution time significantly. I was recently tripped up by this when it turned out that writing debug information to the Rhino command history window was the rate-limiting step in the code that I was writing :woozy_face:

3 Likes

interesting, I added Console.WriteLine there, because the compiler is pretty smart some people say and I wanted to make sure it doesn’t outsmart the test - the Mesh was supposed to be used for something

I thought that was the goal, and you’re right if you don’t have anything to do with the new mesh, the compiler can remove any code that does not accomplish anything. I’d probably just do something like:

int[] facecounts = new int[iterations];
for(int i = 0; i < iterations; ++i) {
  using(var meshDup = new MeshDuplicatorDisposable(mesh))
  {
    facecounts[i] = meshDup.Mesh.Faces.Count;
  }
}

just to prevent the compiler from removing any code that does not do anything.

1 Like

I guess I’ll run more tests later in that case :wink:
ForumNewbie question: Would you suggest I edit the post, use the comments or some other way to add new data?
Also when you mentioned random Meshes, did you mean random amount of vertices and faces, or random shapes (as far as I know the scale can make a computational difference)?

Garbage collection is a hard problem for language and compiler designers: if a framework provides a mechanism to explicitly mark things for disposal I’d do it when convenient even if it’s optional. It could do anything from free actual unmanaged resources to cutting some links to make the GC spend less time trying to figure out if connections for a cycle of garbage objects or are ready for reclamation.

Doubly so if it turns out to matter: I believe that .NET GC is still nondeterministic, so telling McNeel that they can free all of those unmanaged resources even if it takes the GC a while to clean up the managed wrapper could and seems to turn out to be a big win.

I’m not sure why that would be an either/or in this case:

Best practices: The library vendor provided IDisposable for a reason. Use it unless you have a good reason not to.

Testing: Results indicate that using it helps performance.

So my answer is ‘both’, not one or the other.

2 Likes

Thanks a lot for your reply - it made me realise I formed questions in somewhat naive manner :face_with_monocle:
In terms of testing though, what I meant was testing for speed, cause unit and integration tests check for proper result, but not for performance. In real life there are deadlines, and due to the fact that testing for performance takes time, it’s good to know which practices and techniques are helpful and what are not, which I think you explained well :wink:

Visual Studio Debugger has a Process Memory graph that you can use to check if Disposing is worth it or not. If your plugin causes this to increase significantly, and dispose helps to decrease, then it is certainly worthwhile.

image

Duplicate a mesh doesn’t sounds like the better test case for this. In case you don’t know, GH inputs already duplicate its source data to prevent changes be propagated on source components, so if it somehow cached somewhere it may affect to your metrics.

I wouldn’t focus on execution speed but on resource usage (CPU, memory…), because I don’t think that the speed measurements you get from an arbitrary A/B test can be extrapolated to a generic case. On the other hand, I think that the resources metrics can be associated quite well with design patterns, which in the end what you are looking for is to know in which patterns it is convenient to use it and in which it is not, rather than knowing whether it is always convenient to use it or not.

1 Like

the Release build results were actually quite a bit different than Debug and no wonder - Release is optimised, so I guess GC should have it easier in the Release build
In any case due to the difference, I prefered to measure for Release, because that is the real life build scenario


I am aware of that, that’s why there was an iteration input and everything was running in a loop


Great point!