Transfer data to Grasshopper from python

No I don’t really do any GrassHopper, just python…

1 Like

1 Like

Thanks for the advice. That definitely helps!

I have imported the SQLite package into the C# component. Let’s see if that works properly.

No, the data is just generated locally by another external python program on the same device. As diff-arch suggested, I might try using another file format like SQLite rather than plain CSV as windows won’t allow writing and reading at the same time

hey, that sounds promising! I wonder if this will cause any loss in data, as the data is generated in real-time.

One could implement FileSystemWatcher to only read the input file when it changes and thus trigger downstream computation. I’m pretty sure that’s what the native Read File component uses, which one could also use to dynamically read in e.g. a CSV or JSON file.

1 Like

Yes, or use a contextmanager or semaphores, but I think that using a database is way easier than dealing with all the other stuff. SQLite is well documented and there is lots of sample code online that can be quickly adapted and it’s always nice to have your data in well-structured database any way.

I’m trying to import the System.Data.SQLite into the C# but I don’t know which version of .NET Framework the C# is working with. I tried .NET 2.0 but it seems not working with a warning like this:

Any ideas?

Why not Python, like you mentioned in your initial post? I don’t use C#.

Hi, thanks for the help! I’m trying your approach, but I’m stuck here on creating GH_FileWatcher.

I’m confused about how to create this instance.
Any thoughts?

Thanks again! It’s working well. While sometimes reading and writing may lead to some conflicts and result in errors in python, it is working. I guess using some kind of watcher might be helpful to solve it.

1 Like

As @Dancergraham suggested above, you could catch those errors and simply ignore them, meaning if a sqlite3 exception ever gets raised, you simply skip it and try again by updating the GHPython component.

import Grasshopper as gh
import sqlite3


def update_component():
    """Updates this component, similar to using a Grasshopper timer."""
    # written by Anders Deleuran (andersholdendeleuran.com)
    def call_back(e):
        """Defines a callback action"""
        ghenv.Component.ExpireSolution(False)
    # Get the Grasshopper document
    ghDoc = ghenv.Component.OnPingDocument() 
    # Schedule this component to expire
    ghDoc.ScheduleSolution(
        1, gh.Kernel.GH_Document.GH_ScheduleDelegate(call_back)
    )


db_path = "/home/marshall/db_path.db"

try:
    connection = sqlite3.connect(db_path)
    # read database and process data
    connection.close()
except sqlite3.IOError:
    print "Unable to connect to database"  # skip 

if Run:  # Run is a Boolean input
    update_component()  # try again

This is only sample code and should be refined further!
For instance, I would check separately before the try except statement whether the database exists and is a valid database. If it doesn’t the component could potentially try forever without ever realising that the database doesn’t exist.
You probably want to raise an exception or component error for that case. And there maybe others.

Here’s a function from my sqlite3 wrapper module that attempts to check those things:

import os 


def is_sqlite3_database(db_path, strict=True):
    """Verifies whether db_path exists, is a file, and indeed a SQLite3 database.

    Args:
      db_path (str): An absolute path to a SQLite3 database file to verify
      strict (bool): Optionally False to skip validating the SQLite3 header, 
        by default True

    Returns:
      True if db_path is a SQLite3 database file path, otherwise False.
    """
    if not os.path.isfile(db_path):
        return False
    if strict:
        db = open(db_path, mode='r', encoding="ISO-8859-1")
        header = db.read(100)
        if not header.startswith("SQLite format 3"):
            return False
    return True

Have fun!

3 Likes

I’m afraid I’ve not implemented that class. I believe @TomTom has some experience with file event watchers and might be able to help. And just to reiterate, before jumping down the rabbithole, one can dynamically read any text-based database file (e.g. CSV, JSON) using native Grasshopper components. Here’s a quick example manipulating coordinates in a CSV in a notepad and dynamically making some points from these in Grasshopper:


230627_ReadFileDynamically_00.gh (9.2 KB)

3 Likes

The FileSystemWatcher is a quite common utility function. Are we talking about C# or IronPython?

When you initialize it, just make sure to do it once for a definition and also make sure to dispose (= destruct) the instance, once the definition closes. Working properly with event, callbacks or event-alike patterns in Grasshopper is quite a challenge. I might come up with an example during this week. The class itself is quite straightforward. You use this class to listen to file system changes within a folder, and subscribe to certain events. Once the events are fired, e.g. a file has changed in that particular directory, then you perform an action in Grasshopper and trigger a re-computation of the whole solution. This way you don’t need to work with timers at all.

The alternative, as already mentioned, is using a local db. All you need to do is setting up a data-model and connect from different applications. As mentioned, databases are designed for concurrent access, but they may not trigger events. It’s more about storing data, not observing data. So then you may need to work with the timer approach.

There are more ways of Interprocess Communication. A local TCP/UDP connection, Pipes, Shared Memory etc… All of them are a little bit more advanced, but usually more efficent. Again, the big problem is properly injecting them in Grasshopper.

2 Likes

Indeed, which has been especially true in GHPython. For reference, here’s a strategy I’ve been using lately (that has worked for me/my team at least):

I think this is a good use case for the (relatively) new Hops component. If you can run a flask server you basically will have intercommunication (get requests) between your python in visual studio and GH directly without the need to export to any file. See the tutorial below.

https://developer.rhino3d.com/guides/compute/hops-component/#calling-a-cpython-server

3 Likes

This one is super helpful! I’ll test that out once I find some time

Thanks! I did the same thing, but the tricky part is once I start to generate that data and write into the CSV file, the reading in grasshopper and writing in python conflict each other and break the program in python.

FileSystemWatcher Class does sound very promising. I spent a while on reading the documentation but it seems like the syntax is different from C# or python in grasshopper, or I am just not familiar with reading the documentation. If you could make an instance, that would be really helpful!

I have implemented the local db approach. Although there are some conflicts sometimes, at least It works, as @diff-arch suggested, we could also write some functions to check if any errors. Just not elegant as the approach of FileWatcher.

Looking forward to your instance!

This one is so cool. I had a tiny little experience in flask on the website. Never imagine it could be in the grasshopper. sounds like another whole territory with a lot of new functions.