How to optimise GHPython OBJ export?

Hi,

I’ve written this function to be able to export OBJs directly from GHPython inside Grasshopper. So far everything works fine, however dense meshes take quite some time to export. I’m thus looking for ways to optimise the code!

import os

def export_obj(mesh, path, filename):
    
    obj_fname = "%s.%s" %(filename, "obj")
    mtl_fname = "%s.%s" %(filename, "mtl")
    obj_fpath = os.path.join(path, obj_fname)
    
    header = "# Rhino\n\nmtllib %s\ng %s\nusemtl diffuse_0_0_0_255\n" \
        %(mtl_fname, "object_1")
    
    with open(obj_fpath, 'w') as f:
        f.write(header)
        
        for vtx in mesh.Vertices:
            f.write("v %.4f %.4f %.4f\n" %(vtx.X, vtx.Y, vtx.Z))
        
        for uv in mesh.TextureCoordinates:
            f.write("vt %.2f %.2f\n" %(uv[0], uv[1]))
        
        for vec in mesh.Normals:
            f.write("vn %.4f %.4f %.4f\n" %(vec.X, vec.Y, vec.Z))
        
        for face in mesh.Faces:
            f.write("f")
            for i in face:
                f.write(" %d/%d/%d" % (i + 1, i + 1, i + 1))
            f.write("\n")

Any suggestions are welcome. :slight_smile:

One thing that comes to mind is using multi-threading.

Another is apparently, thanks to @Terry_Chappell’s work, create a C/C++ algorithm and invoke it inside ghpython. From Terry’s measurement it is x300 faster.

Check this thread:

Some math operations inside a DLL programmed in C++ run up to 300X faster than in Python.

I find that binary reads run about 3X faster in a C++ DLL vs Python. For example, the DLL used to help import point clouds reads a 4GB file in 4 sec on my computer which uses M.2 SSD with peak read speed of 3.2 GB/sec.

1 Like

Thanks, @Terry_Chappell and @ivelin.peychev.
I’ll look into it, but I only know the very basics of C++, so I’m not terribly optimistic.
Also how would you access rhinocommon in C++ in Mac Rhino?

You need the C++ only for computations and reading/writing lines/strings, not for RhinoCommon stuff.

Hi @p1r4t3b0y,

According to this stackoverflow post it is much quicker to build a big list of strings and then write them in one go eg write(“\n”.join(mylist)) rather than making so many individual write() calls.

Note that it would be a bad idea to progressively build a huge Python string and then write that, as Python strings are immutable and not optimised for concatenation.

I read also about

https://docs.microsoft.com/en-us/dotnet/api/system.io.streamreader.readlineasync?view=netframework-4.8
https://docs.microsoft.com/en-us/uwp/api/windows.storage.fileio.writelinesasync

Alas, I haven’t had time to test them.

Yes, I saw that earlier and tried it, but it added to the recompute time, compared to the above solution. I also tried writelines() with generators and the performance was even more terrible.

Yes, thanks for pointing that out. Furthermore, for string concatenation adding strings is more costly than building them with format or the % operator, at least for big strings.

Hmm… Have you tried list comprehensions, eg:

f.write(["v %.4f %.4f %.4f\n" %(vtx.X, vtx.Y, vtx.Z) for vtx in mesh.Vertices])

Yes, also a little worse in terms of performance. Weird right?

Does the profile module work in Rhino 6? It doesn’t in v5 for Windows

Yup. Does your script go on to do other things or is that the end? You could output your list of strings to another component for writing in parallel with other operations…

What do you mean by profile module?

My ultimate goal is to output sequences of OBJs from GHPython. I wrote it as a function to work as a part of a larger script that animates a mesh. I plan to call the function multiple times during the animation. Another thing that I still have to implement is support for multiple mesh objects inside one OBJ file, but that’s kind of straightforward.

https://docs.python.org/2/library/profile.html

It sounds like you are hitting the limit of how quickly Ironpython can write text to a file on your system so there is not much optimisation to be done other than doing other computational work in parallel. If I understand correctly then GH is multi threaded so you can have one component writing to a file whilst another computes, provided they are wired up in parallel and not in series…?

Yes, it seems to work.

OK, that’s a shame.

This I don’t know.

Great! Can you try to run it on your code?

Are you sure about that?
As far as I know only a few components are multithreaded.

I remember also a while ago, when I asked @DavidRutten if components are executed/expired in the order they are placed/connected, he mentioned that it is not following how they are positioned on the canvas, nor how they are connected. They are simply waiting in line, and whichever is next gets executed.

I hope I got it wrong.

Nope. :slight_smile: no idea. I’ve seen a few discussions of threading so I assumed…

No it’s not true, not even in GH2 in which all components are multithreaded by default. Only ever one component is calculating at any given time*, it’s just that those calculations are distributed amongst multiple threads.

* In GH2 it’s possible that more than one component are calculating simultaneously, however only when those calculations are part of different solutions. Either because two documents are both running at once, or because some component is still busy doing stuff and hasn’t realised yet it’s been cancelled because a new solution has already begun.

This is possible already actually. If the file writing process is slow, a component could choose to start the writing process on a separate thread but then immediately return control to the caller. This way the time taken to write to file isn’t spend ‘inside’ the solution. There are no such components in the standard GH1 library though.

1 Like