Python component in multiple GH files: how to keep updated?

I have a Python component that I use in multiple GH files. Every time I update the component to fix a bug or whatever, I need to (somewhat laboriously) go to each GH file that uses it and update it to use the new version.

This isn’t a show stopper, just a bit of a pain. But like any programmer I worry (especially when trying to sleep at night) about source control…did I remember to update all the right files?

Is there a cleaner/more reliable way to do this? Can I somehow call the component in the “master” GH file from another file? Convert it to a plugin, perhaps?

Any ideas welcome, thanks!

It is possible to load a Python script from an external file to simplify the process of updating your Python component across several GH files. Here’s how to do it:

  1. First, enable script input on your component by holding Shift and right-clicking.
  2. Next, add a Read File component to your GH file.
  3. To specify the programming language, add the following line at the beginning of your code:
#! python2
#! python3
//#! csharp

This method allows you to centralize your script updates, so that any changes are reflected across all GH files that use this script.

8 Likes

Hops?

I have a script that can distribute the latest C# code in one GH definition to other GH definitions that contain C# nodes with the same name. I don’t know the same approach works for Python, but should be the same principle.

//Inputs: file (string), name (List<string>), target (List<string>)

    var io = new Grasshopper.Kernel.GH_DocumentIO();
    io.Open(file);
    string dir = System.IO.Path.GetDirectoryName(file);

    var doc = io.Document;
    doc.Enabled = true;
    Dictionary<string,Tuple<string,string>> dat = new Dictionary<string,Tuple<string,string>>();
    foreach(var obj in doc.Objects )
    {
      if(obj.GetType().ToString().Contains("CSNET_Script"))
      {
        if(name.Contains(obj.NickName))
        {
          int index = name.IndexOf(obj.NickName);
          string tt = target[index];
          var _source = Grasshopper.Utility.InvokeGetterSafe(obj, "ScriptSource");
          string source = (string) Grasshopper.Utility.InvokeGetterSafe(_source, "ScriptCode");
          string additionalsource = (string) Grasshopper.Utility.InvokeGetterSafe(_source, "AdditionalCode");
          dat[tt] = new Tuple<string,string>(source, additionalsource);
        }
      }
    }

    var ff = System.IO.Directory.EnumerateFiles(dir, "*.gh");
    foreach(var f in ff)
    {
      var _io = new Grasshopper.Kernel.GH_DocumentIO();
      _io.Open(f);
      var _doc = _io.Document;
      _doc.Enabled = true;

      foreach(var obj in _doc.Objects )
      {
        if(obj.GetType().ToString().Contains("CSNET_Script"))
        {
          if(target.Contains(obj.NickName))         
          {
              var elem = dat[obj.NickName];
              var source = elem.Item1;
              var additionalsource = elem.Item2;

              var _source = Grasshopper.Utility.InvokeGetterSafe(obj, "ScriptSource");
              Grasshopper.Utility.InvokeSetterSafe(_source, "ScriptCode", source);
              Grasshopper.Utility.InvokeSetterSafe(_source, "AdditionalCode", additionalsource);
              _io.Save();
          }
        }
      }
   }
1 Like

It depends a bit on whether you’re referring to the existing GHPython component or the CPython/IronPython modes of the new Rhino 8 Grasshopper script editor. And on whether or not this is just for you or for many users (and if those users share a server etc.). Either way, in addition to the methods proposed above, here are some options to consider:


1) The most general, simple and Pythonic approach that should work no matter what:
Structuring all the relevant classes and functions into a Python module and importing/calling these within your component. Meaning that you would just need to update a Python file that lives locally or on a server. Here’s a slide from the in-house Python course I teach demonstrating this:


2) Automating reading and overwriting the component code:
The GHPython component (and presumably the new script editor component) has a Code property, which one can systemically read/overwrite in order to update the code within the component. Meaning that one can develop functions that both “push” and “pulls” code from a central folder structure on e.g. a server, a DropBox, or a GitHub. I’ve used this approach quite extensively in my old research groups, when running workshops, and in-house at BIG. Have a look at the two first code blocks for some examples:


3) Compiling GHPython components to a plugin:
I have never had much luck with and find the workflow pretty cumbersome, but it might work for your needs:


4) Using the plugin options provided by the new script editor:
Again I have not used these new features, but they might be relevant to your goals:

6 Likes

I have had good luck with @AndersDeleuran 's solution #1 (packaging and importing python modules) for plugins. However, just to add a few small personal preference items in case its helpful:

  1. I prefer to separate the actual source code and the modules that Rhino is accessing. This makes it easier for me to create git repository for the package, include tests, virtual-environments, manage package versions, include docs, experiment, etc… without having all of that right in the Rhino scripts directory, or messing with the path in Rhino to point it at some other folder. But in order to shorten the iteration time (ie: make a change in the code, test the code in Rhino, …) I use a free VSCode plugin called ‘fsdeploy’ which watches and copies changed files into the deployment directory as I am editing them. This helps make it much easier to write and then test the edited code in Rhino.


    Note that one big plus to writing in an editor like VSCode is that if you type-hint your functions and methods using the Python 2.7 MyPy style and also use the Rhino-stubs + Grasshopper-stubs, it will recognize the types and you get nice autocomplete and type-checking, if you like that kind of thing.

  2. When writing code and testing in Rhino, using the python ‘reload’ function on your imported module can help shorten the development cycle a lot as well. This reloads the module every GH-component execution, so you can see any newly written / edited code working without rebooting Rhino.


    There are some ‘gotchas’ with reload however, especially if you or any of the packages you interact with use isinstance checks - so I usually put the reload under a ‘Dev-mode’ flag, so I can turn it on to write and test, then turn if off to deploy.

  3. As just a general suggestion, I have been very happy with the pattern where the GH-component functions as only a facade/wrapper to collect the inputs, and display user-messages, but passes off all the actual work to an outside function or, as I mostly prefer, a class for each component. All input validation, work, and output filtering/sorting happen in this module outside GH. The less ‘real’ code in the GH-Component, the easier it is to manage versions and updates, IMO.

best of luck with it,
@ed.p.may

4 Likes

Can someone please make this easy for those of us who don’t want to know the entire Python “stack”?

I’m looking for the equivalent of a simple include file to reuse a bit of code in multiple Python components in a GH file. So far, I see no way to specify a path when importing a module? Where does it look? Or how do I tell it where to look?

No reply… I’m not sure if that’s because few people know the answer or because the few who do are offended by my use of the word “simple” in my question?

Somewhere on this page:

I got the idea that locating the module file in the same directory as the GH file might work. So I tried that using the fibo.py example on this page:

# Fibonacci numbers module

def fib(n):    # write Fibonacci series up to n
    a, b = 0, 1
    while a < n:
        print(a, end=' ')
        a, b = b, a+b
    print()

def fib2(n):   # return Fibonacci series up to n
    result = []
    a, b = 0, 1
    while a < n:
        result.append(a)
        a, b = b, a+b
    return result

But that produced this mysterious error:

Runtime error (SyntaxErrorException): unexpected token '='

Traceback:
  line 1, in script

Which was confusing… and made sense only when I pasted the entire fibo.py file into my Python GH component, which generated a more informative error message:

Runtime error (SyntaxErrorException): unexpected token '='

File "", line 6
    print(a, end=' ')
                ^
SyntaxError: unexpected token '='

So without bothering to figure out why this statement fails: print(a, end=' '), I simply deleted the first def fib(n) and left only def fib2(n) in the fibo.py file. So my GH Python component has this:

import fibo

a = fibo.fib2(x)

And the fibo.py file in the same folder has this:

# Fibonacci numbers module

def fib2(n):   # return Fibonacci series up to n
    result = []
    a, b = 0, 1
    while a < n:
        result.append(a)
        a, b = b, a+b
    return result


fibo_2024Mar12a.gh (8.3 KB)
fibo.py (198 Bytes)

There is much more to the Python “import system” but for now, this is good enough for me.

Hey @Joseph_Oster

maybe this is what you are looking for.

You can have your python scripts in a random folder and import it into several grasshopper scripts. Only editing 1 file is necessary. Let me know if you need additional help.

Thanks but that doesn’t work for me. When I run this loop:

for sp in sps:
    print(sp)

I get only this:

C:\Program Files\Rhino 7\Plug-ins\IronPython\IronPython.dll\Lib\site-packages

But there is no \IronPython\IronPython.dll\ folder, only \IronPython\Lib\site-packages

And I can’t find this file anywhere: distutils-precedence.pth

The procedure you describe sounds like what I was looking for but… :question: It doesn’t work for me.

No worries, using the same directory as the GH file serves my immediate purpose.

P.S. I do have this folder: C:\Users\Joseph\.rhinocode\py39-rh8 but it contains only an empty \cache\ folder.

I never checked, if this works in Rhino 7. Can you try with R8?

I believe I checked R8 yesterday before I replied. Checking again now, using IPy2 I see:

C:\Program Files\Rhino 8\System\netcore\IronPython.dll\Lib\site-packages

But C:\Program Files\Rhino 8\System\netcore has no \IronPython.dll\Lib\site-packages, only a \runtimes folder with no distutils-precedence.pth file anywhere.

Trying ‘Python 3 Script’ instead (Py3) causes Rhino to freeze so hard I have to kill it. Multiple tries, all the same result. Completely useless. R8 has been “official” for a long time, yet is still infested with bugs.

P.S. In this folder: C:\Program Files\Rhino 8\System - among many other files - I find these two:

  • IronPython.dll
  • Rhino.Runtime.Code.Languages.IronPython2.dll

Files, not folders.

You might implement the first method I posted up here:

In Rhino 8 as well, where GHPython, IronPython, and CPython all have the standard Rhino script folder hardcoded:

You can use Everything by voidtools.com. It indexes your drive and you can find anything with it.

I’ll spare you the gory details I went through to update R8. It was a dysfunctional mess, far from automatic. If McNeel doesn’t already know about these difficulties, they should.

Finally I managed to download rhino_en-us_8.5.24072.13001.exe and execute it. Py3 in R8 worked this time and showed this when I ran your `for sp in sps: loop:

C:\Users\Joseph\.rhinocode\py39-rh8
C:\Users\Joseph\.rhinocode\py39-rh8\lib\site-packages

Those folders exist and I found the distutils-precedence.pth file and edited it as directed:

import os; var = 'SETUPTOOLS_USE_DISTUTILS'; enabled = os.environ.get(var, 'stdlib') == 'local'; enabled and __import__('_distutils_hack').add_shim(); 
import sys; sys.path.append(r"C:\Users\Joseph\Documents\python")

Moved the fibo.py file, restarted Rhino (R8), copied yesterday’s fibo_2024Mar12a.gh to a Py3 component and it fails:

I re-booted my computer and tried again. Failed again. I’m done for now banging my head against this wall. What I got yesterday in R7 works fine for me. Thanks anyway, it was a wild goose chase.

I couldn’t get this to work (putting the module in the same directory as the Grasshopper file). Maybe because I’m on a Mac? Too bad, what a nice simple solution that would be. I guess I could mess with path variables somewhere in the Rhino files…but nah :slight_smile:

Putting the module in the scripts folder, as AndersDeleuran suggested, seems to work fine. It’s still a bit of a pain - there are separate scripts folders for different Rhino versions, and it makes it tough to keep the module in GitHub with all the grasshopper scripts that use it - but I’ll muddle through.

I’m happy that others are trying to do the same thing, and thanks a bunch, all of you, for posting your thoughts. I’ll experiment a bit with aliases to see if I can keep the code centralized, but even if not, I think I’m in pretty good shape.

I know there have been some updates to this. But currently in my Rhino 8.6 running this codes:

import sys

for p in sys.path:
    print(p)

Here are the default paths environment paths, seems like a wide range:

C:\Users\Scott Davidson\.rhinocode\py39-rh8\site-envs\default-SVYlOV1X
C:\Users\Scott Davidson\AppData\Roaming\McNeel\Rhinoceros\8.0\scripts
C:\Users\Scott Davidson\.rhinocode\py39-rh8\site-rhinoghpython
C:\Users\Scott Davidson\.rhinocode\py39-rh8\site-rhinopython
C:\Users\Scott Davidson\.rhinocode\py39-rh8\site-interop
C:\Users\Scott Davidson\.rhinocode\py39-rh8\python39.zip
C:\Users\Scott Davidson\.rhinocode\py39-rh8\DLLs
C:\Users\Scott Davidson\.rhinocode\py39-rh8\lib
C:\Program Files\Rhino 8\System
C:\Users\Scott Davidson\.rhinocode\py39-rh8
C:\Users\Scott Davidson\.rhinocode\py39-rh8\lib\site-packages
C:\Windows\Microsoft.NET\Framework64\v4.0.30319\

Then at the tope of the component we can add any path on the fly:

- Use env directive to add an environment path to sys.path automatically
    # env: /path/to/your/site-packages/

# env: F:\Users\scottd\Documents\python

That would be a way to configure a library location?

Or I guess a 3rd way is to add the paths to the module search paths in the Python engine?

image

1 Like

I did that last week and saw only one path, which was invalid:

Is you Rhino 8.6? That is the Release Candidate.

This is the Python 2 component default:

C:\Users\Scott Davidson\AppData\Roaming\McNeel\Rhinoceros\8.0\scripts
C:\Users\Scott Davidson\.rhinocode\py27-rh8\site-envs\default-FjcsR4md
C:\Users\Scott Davidson\.rhinocode\py27-rh8\site-rhinoghpython
C:\Users\Scott Davidson\.rhinocode\py27-rh8\site-rhinopython
C:\Users\Scott Davidson\.rhinocode\py27-rh8\Lib\site-packages
C:\Users\Scott Davidson\.rhinocode\py27-rh8\Lib

And the #env is working on that component also. So more can be added quickly.