Leap motion + rhino python


#1

Hi all,

I have been toying around with the Leap Motion controller and it’s API in Python.

My goal is to write some scripts in Rhino Python which leverage the data acquired from the Leap controller. I thought I’d share my findings so that others may benefit but also to ask questions relating to the stability of running Leap within Rhino Python.

The setup process to access the Leap libraries from within Rhino Python is as follows:
Created a rhino python script called leapTst.py in a folder
To the folder I added the relevant files from the Leap SDK Dev Kit. The files in the folder are the following, including the leapTst.py test script:
Leap.dll
Leap.py
Leap.pyc
LeapCSharp.dll
LeapCSharp.NET4.0.dll
leapTst.py

At the beginning of the leapTst.py script I use clr.AddReferenceToFileAndPath and point it to the LeapCSharp.NET4.0.dll assembly inside the folder.

Also in the script, the path to the Leap SDK x64 libraries folder is appended to sys.path to reference the correct libraries.

I then adapted the Sample.py file from the Leap SDK samples folder in Rhino Python to create a custom SampleListener which retrieves frame data from the Leap controller and prints it to the Rhino console.

I noticed that while using the listener callback function and the OnFrame() method to retrieve frame data, Rhino would crash as soon as the listener began ‘listening’ to the incoming stream of data. In order to debug the script I customized the SampleListener class so that it would just return the Frame ID and Timestamp of the current frame from the Leap controller. This is very stable and Rhino does not crash.

As soon as I make the OnFrame method retrun any other frame data, such as the Fingers.TipPosition or HandSphereRadius, then the script crashes Rhino.

Reading the Leap API, there is a note on using the listener callback functions to retrieve frame data that mentions the process is multithreaded and therefore one must ensure that it is run in a threadsafe manner. Could this be the reason why Rhino crashes?

What other methods could be used in Rhino Python (considering that Rhino itself has no inherent application frame rate) to poll the controller for frame data? Timer? Event based polling?

Any help on this would be greatly appreciated. Following is the test script code:

import sys
import clr
import scriptcontext
import rhinoscriptsyntax as rs
import Rhino
import time

clr.AddReferenceToFileAndPath("C:\Users\GMAC\Desktop\leap RHINO test\LeapCSharp.NET4.0.dll")

pathsToLibs = ["C:\\Users\\GMAC\\Documents\\PROJECTS\\CODING\\LEAP MOTION\\LeapDeveloperKit\\LeapDeveloperKit\\LeapSDK\\lib\\x64"]

for path in pathsToLibs:
    sys.path.append(path)

print sys.path

import Leap

from Leap import CircleGesture, KeyTapGesture, ScreenTapGesture, SwipeGesture

class SampleListener(Leap.Listener):
    def OnInit(self, controller):
        print "Initialized"

    def OnConnect(self, controller):
        print "Connected"

    def OnDisconnect(self, controller):
        # Note: not dispatched when running in a debugger.
        print "Disconnected"

    def OnExit(self, controller):
        print "Exited"

    def OnFrame(self, controller):
        # Get the most recent frame and report some basic information
        print "This is a frame"
        frame = controller.Frame()
        print frame
        
        hands = frame.Hands
        if hands.Count > 0:
            print "Found {} hands".format(hands.Count)
            if hands.Count == 1:
                hand = hands[0]
                Leap.Hand.SphereRadius
                handSphereRadius = hand.SphereRadius
                handSphereCentre = hand.SphereCentre
                print handSphereRadius
                print handSphereCentre

              

def main():
    # Create a sample listener and controller
    listener = SampleListener()
    controller = Leap.Controller()

    # Have the sample listener receive events from the controller
    controller.AddListener(listener)

    # Keep this process running until Enter is pressed
    print "Press ESCape to quit..."
    running = True
    while running:
        escape = scriptcontext.escape_test(False)
        if escape:
            running = False
    # Remove the sample listener when done
    print "Shutting down"
    controller.RemoveListener(listener)


if __name__ == "__main__":
    
    main()

#2

Something I forgot to mention in the above:

When I open the script in the Rhino Python editor and run it, i receive the error message:
No module named LeapPython

However, if I start a new script and import Leap, no error message. When I then go back to the original script and run it, it works without error.


(Giulio Piacentino) #3

I suspect that an exception is thrown and it sent back to the base listener, which then decides to close the application. The exception might be due to multithreading (but it might also be something else). The method that is added as “listener” will be called from an alien thread and we do not really know what it will do if an error happens there. I would start adding a try-except block around called methods. Then, make sure to be able to view the the hypothetical exception. I think RhinoApp.Write() should work, as it should be callable safely from different threads (I think I remember correctly).

@dale and @Alain might know more, as they were involved in creating samples for event watchers
http://wiki.mcneel.com/developer/dotneteventwatcher

Thanks,

Giulio

Giulio Piacentino
for Robert McNeel & Associates
giulio@mcneel.com


(Giulio Piacentino) #4

Do you know where that module is imported?


(Alain Cormier) #5

It looks like the python files are what you’d reference if you were using CPython. Since Rhino uses IronPython all you need to reference is the LeapCSharp.Net4.0.dll and make sure it can find the native libs it depends on (Leap.dll and LeapCSharp.dll I think) are available.


#6

Thanks for the reply Giulio.

I will look into further debugging the exception thrown by the listener.

An alternative method I’ve employed now is using the sleep() method from the time module. Instead of using the listener callback function from the controller, I am simply polling frame data from the controller followed by calling sleep(0.045) which should equate to 22 ‘polls’ or frames per second. Using this method the data stream is stable and Rhino does not crash, but intuitively this feels like an ‘incorrect’ approach.


(Giulio Piacentino) #7

Yes, please let us know how this goes with catching a possible exception. Then, we can look into the right approach for you to hook into Rhino’s UI thread…

Giulio

Giulio Piacentino
for Robert McNeel & Associates
giulio@mcneel.com


(Alain Cormier) #8

Sorry if I wasn’t clear but I think (I hope) that my previous post explains why you get the “no module named LeapPython” error.

“import Leap” tries to load the Leap.py module but that script should only be used if you are using a CPython interpreter. Since Rhino uses IronPython what “import Leap” should be doing is loading the Leap namespace from the LeapCSharp.Net4.0.dll assembly. So Leap.py should not be on your path.

Trying to modify a UI component from a different thread causes an exception but in this case you are only calling the print function which I think is ok. The following test confirms that:

import System.Threading

def ct():
    return System.Threading.Thread.CurrentThread.ManagedThreadId
    
def f():
    print "other tread: {0}".format(ct())

print "main thread: {0}".format(ct())
t = System.Threading.Thread(System.Threading.ThreadStart(f))
t.Start()

So something else is going on.


#9

Thank you Alain,

That does make more sense. I have removed Leap.py from the script’s folder and by only adding the clr reference to the LeapCSharp.NET4.0.dll followed by import Leap now correctly loads the Leap namespace from the assembly.

Still haven’t managed to catch the exception. Will report back when successful.

Thanks again for your help