RhinoVR - a sample plug-in for rendering Rhino viewports in virtual reality

Serengeti (Rhino WIP)
#1

Hello!

For the last couple of months I’ve been working on a virtual reality sample plug-in for Rhino which showcases how to render Rhino viewports in VR. This plug-in is intended to be a helpful example for developers of Rhino plug-ins with VR functionality.

It is now available from here: https://github.com/mcneel/RhinoVR

The first version of RhinoVR can be downloaded as an RHI from here: https://github.com/mcneel/RhinoVR/releases

System requirements:

  • HTC Vive or Oculus Rift
  • Steam with the SteamVR app installed.
  • Rhino 6 (Service Release 6, currently only available as a service release candidate) or Rhino WIP.

Installation:
Make sure you have at least Rhino 6 version 6.6.18156.11421 or Rhino WIP version 7.0.18156.04035. Rhino 6 SR 6 is currently available as a service release candidate. To enable the downloading of service release candidates, go to Tools -> Options… -> Updates and Statistics -> Update frequency: Service Release Candidate. Then click “Check Now…”.

Usage:
Start Rhino and load the file you want to view in VR. Make sure a perspective viewport is selected and that it is set to the display mode you want to use. Type RhinoVR into the command line. This should automatically start Steam and SteamVR. If you’re using the Rift, it will also start the Oculus client. Rhino tells you when it is ready and asks you to put the VR headset on.

  • It is possible to navigate around in the Rhino scene using the VR controllers (Vive and Rift). Vive uses the touchpads for navigation and the Rift uses the analog sticks. The left controller controls translation (forward, backward, left, right) and the right controller controls horizontal rotation (turning left/right) and up/down movement.
  • Objects can be selected by pressing the touchpad button (Vive) or pressing the analog stick (Rift).
  • The Rhino Move-command can be initiated by pressing the Application Menu button (Vive) or the B-button (Rift). Remember, the move command works in steps: Select objects. Enter. Pick reference point. Pick new point.
  • A Rhino Enter key press can be mimicked using the VR controller by pressing the trigger button (Vive) or the A-button (Rift).
  • A Rhino Esc key press can be mimicked by squeezing the grip buttons (Vive) or by pressing the primary trigger button (Rift).

Type RhinoVR into the command line again to close down RhinoVR.

Finally, you can download a rather amateurish screen recording of me testing it out: https://drive.google.com/file/d/1v4c1WQN9c2iQ3MXWLQ9ZZsi-RKlm--yv/view?usp=sharing

image
image1903×805 315 KB

15 Likes
VR and RHINO 6?
Oculus Quest and VR headsets in general
#2

This is cool, and thank you for working on this. My first reactions:

It has a lot more jitter than say GoogleVR or other Steam titles, especially with quick movement. Edge quality also seems a bit degraded from say Unity. Is that something that can be improved? It is interesting how much more objectionable Z fighting is in VR than it is in a viewport, I guess it is because each eye is perhaps seeing different things?

Is this plug-in something that you will continue to tinker with, or is it more of an example for other developers to use as a starting point to build their own things with your tool kit? I ask because I feel at least on the Vive the controls could be a bit more intuitive, but if its an example plug-in, I’ll just enjoy it as it is.

Thanks again,
Sam

#3

First paragraph:

image
image.png709×76 4.49 KB

#4

That said there are definitely major optimizations that we can make in the internal Rhino display pipeline that would help improve performance for VR type situations. I’m not sure when we be able to tackle these optimizations.

#5

Hi @SamPage,

As mentioned above, this is not intended to be a fully fledged product. It only serves as an example of how to create VR plug-ins for Rhino.

Regarding Vive movement controls: You are right, moving around with the Vive is not ideal. The analog sticks of the Rift make the movement feel much better, and the feeling is hard to replicate with the Vive’s touchpads. That said, maybe we can improve some things.

Regarding performance: VR rendering is very demanding. Currently we are rendering each eye at 1080x1200, for a total of 2160x1200 pixels. For a smooth experience this needs to happen at 90 FPS. That said, we are going to keep optimizing the Rhino display so hopefully viewport rendering and VR rendering will both get better.

Even though we are rendering at 1080x1200 per eye, this is still not the optimal resolution. The OpenVR library suggests 1330x1584 per eye for the Rift, but we decided to cap the resolution at 1080x1200 for now. Of course, this can be changed in the source code.

-David

2 Likes
#6

Hello David,

I have installed your plugin on Rhino Version 6 SR7 (6.7.18210.11281, 29/07/2018)

I wonder what I am doing wrong, because this is the message I get

image

Thanks

Luca

#8

Hi @luca_biselli,

Do you have a Vive or Rift? You need to make sure your VR device works using Steam and the SteamVR app. If that works, then the OpenVR library that Rhino uses should find the installation path.

-David

#9

Thanks David,
Sorry I am just at the beginning of the VR revolution.
I have an old Oculus Rift DK2, which doesn’t seem to work on the most current Oculus setup software. I thought that your plugin would have the Rift as a mirror. So my understanding is that if the VR Headset doesn’t work, then RhinoVR doesn’t work either.
Luca

#10

Yes, that’s correct. There is still a small chance that the DK2 will work with SteamVR. Maybe worth trying.

-David

#11

Hi David,
Thanks for sharing this. I am eager to try it!
However, I’m getting the following error when attempting to install the plugin using the RHI:
“this package is not compatible with the rhino installer engine”

I’m using Rhino Version 6 SR8 (6.8.18240.20051, 8/28/2018)
Should I use an alternative installation?

#12

If you downloaded https://github.com/mcneel/RhinoVR/releases/download/v0.2/RhinoVR.rhi then you should update your Rhino to 6SR9.

#13

Thanks Nathan,
Did it and it worked!
Sorry about the trouble.

#14

Hi, I managed to have RhinoVR and Steam working together, both on Oculus DK2 and CV1. However, I don’t have the touch controllers. Is there any way to map the controls on to the XBOX controller or any other input device? Thanks L.

#15

Hi @luca_biselli,

Have you tested it with an Xbox controller? It might actually just work out of the box thanks to OpenVR, but I can’t verify that because I don’t have an Xbox controller.

I created two Github issues:
https://github.com/mcneel/RhinoVR/issues/7 (Xbox controller support)
https://github.com/mcneel/RhinoVR/issues/8 (Keyboard and mouse support)

#16

Hi David, we have tried, but no luck. Considering the cost of the Vive/Rift controllers, vs the XBox controller, I think that a lot people could use it, saving some money. When not needing VR for editing within Rhino, but for visualisation purpose only, it should be more than enough.

#17

Hi everybody,
I’m experimenting with an alternative navigation system in RhinoVR, by transforming the virtual space according to relative transformations of the controller.
For that purpose, I’m using the transformation matrix from one controller at different times so I can apply the relative transformation to virtual space, multiplying it by m_cam_to_world. However, it seems that what I get from m_device_data[m_device_index_left_hand].m_xform is the controller’s transformation relative to the controller referential itself, and I need its absolute position in the real space. Is there a way of getting that?
Thanks in advance, Eduardo

#18

Hi @castroecosta,

I’m not sure I understand what you’re after. If you want to transform the left hand controller into a world-space location, then just multiply with the m_cam_to_world xform.

All the necessary information should be available in the source code. See the UpdateState() function.

1 Like
#19

Hi @DavidEranen,
Indeed, I’m editing the UpdateState() function. I am capturing “differential transformations” and applying it to cam_to_world:

if (dpad_up_pressed || dpad_down_pressed)
{
    // store controller xform before navigation
    first_xform = m_device_data[m_device_index_left_hand].m_xform;
}
else if (dpad_up || dpad_down)
{
    // store current controller Xform
    current_xform = m_device_data[m_device_index_left_hand].m_xform;

    // relative transformation from first_xform to current_xform
    ON_Xform diff_xform = first_xform.Inverse() * current_xform;

    // apply independent transformations to cam_to_world
    m_cam_to_world_xform = diff_xform * m_cam_to_world_xform;
}

However, the transformation that is being applied to the Rhino coordinate system (cam_to_world?) is relative to the controller’s coordinate system. I uploaded a video I hope illustrates the problem:

#20

Hi @castroecosta,

You need to express the controller xforms in world-space. first_xform should be transformed by m_cam_to_world_xform. So should current_xform. Now take the difference between them. Then apply the difference to m_cam_to_world_xform.

Does that work?

These things are tricky to get right.

-David

1 Like
#21

Also, what happens if you switch from

m_cam_to_world_xform = diff_xform * m_cam_to_world_xform;

to

m_cam_to_world_xform = m_cam_to_world_xform * diff_xform;

1 Like