Enable Batch render in the VRay options, so your script does not get focus until the render has been finished. You can script these rhino commands, to start the render, save it and close the renderwindow:
_Render
_SaveRenderWindowAs
_CloseRenderWindow
After closing the render window, you can control layers etc. Apart from this, a few VRay options are accessable using either VRay plugin object or via the VRayForRhinoNETInterface. Here is a long thread to show how to use them via python, it sould work the same for a C# plugin.
c.