Using Gumball from MouseCallback?

I’m trying to show a gumball to manipulate a custom object which is not a Rhino object and is rendered via a DisplayConduit. I found the gumball sample, where a subclass of GetTransform is used to control the gumball from a command.

However I want to show the gumball on screen, and allow the user to interact with it like with the real gumball, not in a one-shot command. As far as I can tell, to do this I have to use a MouseCallback to detect MouseDown, and then start the gumball drag. However by the time I’ve gotten the MouseDown in the MouseCallback, it’s too late to start the Get() on the GetTransform object. Is there any way to tell the GetTransform object that it should not wait for a MouseDown and start dragging immediately?

Alternatively, I could not use GetTransform at all and just do everything from MouseCallback. But then I need a way to convert from ViewportPoint to 3d point which GetTransform does automatically.