Vray errors and crashing issues, GPU VRAM Problems

Hi all, I recently built a desktop system that can be considered high-end.

Quick specs: AMD Ryzen 9 5950x 16 core CPU, Asus ROG Strix RTX 3080 10GB OC Edition GPU, 4x16Gb 3600mhz RAM, Asus X570-E Motherboard, Water cooling, etc. and everything works fine in general, drivers are all up to date.

I do a lot of architectural projects, some are with other BIM/CAD software some are natively in rhino only, and also do many renders, sometimes using different softwares such as lumion. I really like vray but with my older system it was a bit painful to do my works with vray and in the end, I prepared a custom PC and with a big passion sit in front of it and started testing with my older projects. But it became such a disappointment for now as I am having troubles with vray for rhino. Here are the problems. I installed the latest version of vray (5.20) and rhino 7. First I tried to make a GPU render with RTX mode, so only GPU power. I used a mid-level project, basically a bus interior almost everything drawn by me in rhino, bus exterior is a mesh data which is not extreme. So it is absolutely an ordinary project(later I tried other projects) but I couldn’t render with GPU cause I got a VRAM exceeding issue. I changed settings like GPU texture resolution, mipmapping, etc. but still even without using displacement it calculates more than 20Gb of memory to render it so the first option failed at the beginning. Secondly, I tried CUDA render with GPU and CPU together as I learned that in this case CPU can carry a good part of memory and give some space to GPU. But every time when render starts it starts rendering and after just a minute it stops and says “Unhandled exception occurred, will exit render loop.”, showing any error code or definition so I don’t understand what is the reason. At the and wanted to check if I render properly only with CPU as I have one of the high-level CPUs on the market nowadays. And guess what, after again a few minutes this time rhino crashes. I thought there can be an incompatibility between vray and rhino versions so tried with rhino 6 and both new and old vray versions and other projects but it crashes every time exactly after couple of minutes without any error message. Sorry for the long story but I am stuck in a way and losing my hope about rhino and vray and even started to consider should I learn other software like 3ds max or blender but as a long time rhino user I am actually pretty happy with rhino’s capabilities and if I can solve the issue, will stay with rhino and will focus on my work rather than starting a new odyssey after years.

I would be really appreciated if someone can help. Also can share more details, system info…


Shouldn’t you be asking Chaosgroup about this?

Try the Cycles renderer. I use iRay. Rendering does not begin and end with Vray.

Of course, I can ask this there too but, here we have categories and I think it is not so much irrelevant under the rendering section. The problem can be about vray itself or some settings in rhino, if someone knows the rhino side, I and maybe in the future somebody can find the answer, is it bad?

You could pack your scene and send it to support(@)chaosgroup(.)com. If you need a quick test you could send it to me per PM and I start a test rendering here.

I use Rhino|V-Ray all days and it renders fine. Maybe there is something wrong with your scene. Did you try to render a little part of your scene only?

I would render parts of the scene and try to find the part which cause the trouble. Also you could use a global override material without texture. So you could see how it renders without textures. …

From my experience V-Ray works really nice and stable. I see no competitor at Rhino at this level of features. Learning an other software isn’t needed, there is something wrong with your setup. In the past I got a lot of trouble too and finally it was a hardware issue (RAM).

(General 10GB VRAM are quite limited if you use the card for system and display. Best you buy a cheap card for display use and use the 3080 for rendering only.)

Hello Micha, thanks for the suggestions. Yes, I tried many different scenes. At last, I created a super simple test scene just an empty room and in the middle a sphere with basic mirror material. I applied a basic wall material from Vrays native materials(no displacement etc.). There is no extra light, just rhino sun. Each method (RTX, CUDA, and CPU) starts normally but within a minute they all crash. I installed SketchUp in order to understand if the problem is between Rhino and Vray but the same happened and SketchUp crashed too. Btw I have the latest version of lumion, it works like charm. I tried rendering with native renderer of rhino it also works just normally. So it is clearly a vray and system incompatibility issue regardless of 3d software. I changed bios settings to default, tried almost everything I can guess but no changes :S I tried, if can see an error message just before it closes, so I opened the dialog window and tried to make a screenshot but I saw that it crashes without showing an error message. Also observed what is happening during the render process and saw that RAM usage, temperatures are all fine(it uses max. a quarter of 64gb RAM). I will contact to chaos support team, lets see if they will help.

You could try rendering with Rhino Render or switching a viewport to Raytraced - ensure you have your GPU set as the CUDA or OptiX render device set under Tools > Options > Cycles before you do these tests.

If rendering works fine even after a the time that Vray generally crashes out you could assume that the problem somehow is with that render engine.

Anyway, it would help if you also posted the unabridged results of the Rhino command _SystemInfo.

edit: btw, also ensure your PSU is up for the task of providing all the power needed under full load.