How do I get a Rhino image buffer data in C++?

Hi,
I am building an output buffer via CUDA and I am trying to get a Rhino image buffer data to “see” this data. As I understand the most straight-away but perhaps slow method is to copy pixel by pixel from the one buffer (CUDA device) to the Rhino buffer.

			for (int i = 0; i < num_pixels; ++i) {
				const uchar4 pixel = cuda_buffer_data[i];
				const int x = i % width;
				const int y = (num_pixels - i - 1) / width; // Reverse the vertical coordinate due to top-down pixel ordering

				float fColor[4];
				fColor[0] = pixel.x;
				fColor[1] = pixel.y;
				fColor[2] = pixel.z;
				fColor[3] = pixel.w;
				// Assuming pChannel is a color channel
				pChannel->SetValue(x, y, ComponentOrder::RGBA, fColor); // Set the pixel value in the Rhino SDK image buffer
			}

Are there any other options?

@jeff - is this something you can help with?

Rhino 7 SR29 2023-4-17 (Rhino 7, 7.29.23107.03001)
win10 64bit
VS2019
Rhino’s SystemInfo:

Rhino 7 SR29 2023-4-17 (Rhino 7, 7.29.23107.03001, Git hash:master @ 5f05ef3a2eea3d910c4c2fb79ccc75b413d33ca7)
License type: Commercial, build 2023-04-17
License details: QWERTY

Windows 10 (10.0.19045 SR0.0) or greater (Physical RAM: 16Gb)
Rhino’s SystemInfo:
Computer platform: DESKTOP 

Standard graphics configuration.
  Primary display and OpenGL: NVIDIA GeForce RTX 3050 (NVidia) Memory: 8GB, Driver date: 4-25-2023 (M-D-Y). OpenGL Ver: 4.6.0 NVIDIA 531.79
    > Accelerated graphics device with 4 adapter port(s)
        - Windows Main Display attached to adapter port #0
        - Secondary monitor attached to adapter port #1

OpenGL Settings
  Safe mode: Off
  Use accelerated hardware modes: On
  Redraw scene when viewports are exposed: On
  Graphics level being used: OpenGL 4.6 (primary GPU's maximum)
  
  Anti-alias mode: 4x
  Mip Map Filtering: Linear
  Anisotropic Filtering Mode: High
  
  Vendor Name: NVIDIA Corporation
  Render version: 4.6
  Shading Language: 4.60 NVIDIA
  Driver Date: 4-25-2023
  Driver Version: 31.0.15.3179
  Maximum Texture size: 32768 x 32768
  Z-Buffer depth: 24 bits
  Maximum Viewport size: 32768 x 32768
  Total Video Memory: 8 GB

Rhino plugins that do not ship with Rhino
  C:\Users\QWERTY\Desktop\RHINO\code\----------------------------------------------------------------
  C:\Users\QWERTY\Desktop\RHINO\----------------------------------------------------------------
  C:\Users\QWERTY\AppData\Roaming\McNeel\Rhinoceros\packages\7.0\ColorPicker\6.0.0\ColorPicker.rhp	"ColorPicker"	

Rhino plugins that ship with Rhino
  C:\Program Files\Rhino 7\Plug-ins\Commands.rhp	"Commands"	7.29.23107.3001
  C:\Program Files\Rhino 7\Plug-ins\rdk.rhp	"Renderer Development Kit"	
  C:\Program Files\Rhino 7\Plug-ins\rdk_etoui.rhp	"RDK_EtoUI"	7.29.23107.3001
  C:\Program Files\Rhino 7\Plug-ins\rdk_ui.rhp	"Renderer Development Kit UI"	
  C:\Program Files\Rhino 7\Plug-ins\NamedSnapshots.rhp	"Snapshots"	
  C:\Program Files\Rhino 7\Plug-ins\Toolbars\Toolbars.rhp	"Toolbars"	7.29.23107.3001
  C:\Program Files\Rhino 7\Plug-ins\3dxrhino.rhp	"3Dconnexion 3D Mouse"	
  C:\Program Files\Rhino 7\Plug-ins\Displacement.rhp	"Displacement"

What is that data format for cuda_buffer_data? If it’s not 8 bits per channel, then there is no way to do this any differently than what you show here…because the conversion is going to obviously have to take place in order to pack things into a 32bit, 4 channel format.

If cuda_buffer_data is already in a packed 32bit, 4 channel format, then you could just move it all into a CRhinoDib, making sure size and alignments are all correct (based on Width and Height), and then use the display pipeline’s method CopyToBackBuffer(dib) …again assuming cuda_buffer_data size and format all matches the current back buffer’s size and format.

Just know that on Windows this will require that you store things into cuda_buffer_data in BGRA byte order.

If this isn’t what you’re requesting, then I’ll need more information and specifics.

-Jeff

Thanks for your response, after reading your suggestions I have managed to clear out some basic concepts. I am working on a Realtime Renderer based on Rhino SampleRenderer example and my image buffer is stored in the GPU via CUDA. After all the CUDA work is completed I have a CUDA buffer:

sutil::CUDAOutputBuffer<uchar4>output_buffer(sutil::CUDAOutputBufferType::CUDA_DEVICE, sz.cx, sz.cy);

Now during the rerender while loop a IRhRdkRenderWindow::IChannel* pChannel is created:

	IRhRdkRenderWindow::IChannel* pChannel = pRenderProcess->m_pRenderWnd->OpenChannel(IRhRdkRenderWindow::chanRGBA);

My problem is that I cannot find the way to insert the CUDA buffer directly into the pChannel data so I can have the image rendered in the Rhino window.

		IRhRdkRenderWindow::IChannel* pChannel = pRenderProcess->m_pRenderWnd->OpenChannel(IRhRdkRenderWindow::chanRGBA);
		const int32_t width = sz.cx;
		const int32_t height = sz.cy;
		const int num_pixels = width *height;

//***********This does not work, it is just the main idea*********************//

		// Copy the entire CUDA buffer to Rhino image buffer
		const size_t buffer_size = num_pixels * sizeof(uchar4);
		uchar4* rhino_buffer_data = reinterpret_cast<uchar4*>(pChannel); // Get pointer to Rhino image buffer data
		CUDA_CHECK(cudaMemcpy(rhino_buffer_data, cuda_buffer_data, buffer_size, cudaMemcpyDeviceToHost));

How can I achieve this? Some sample code would be really helpful!
I checked out the CRhinoDib documentation but I am not sure how it can help in my case.

Alternatively, I could cudaMemcpy the CUDA buffer to a local Image buffer via the getHostPointer() function of sutil::CUDAOutputBuffer<uchar4>output_buffer

sutil::ImageBuffer buffer;
buffer.data = output_buffer.getHostPointer();
buffer.width = sz.cx;
buffer.height = sz.cy;
buffer.pixel_format = sutil::BufferImageFormat::UNSIGNED_BYTE4;

But again I cannot find the proper way to insert the sutil::ImageBuffer buffer data into the IRhRdkRenderWindow::IChannel* pChannel

Ok, I initially thought you were trying to get results into the viewport(s) frame buffer. It sounds/looks like you’re talking about the standalone render window, and that you’re using the RDK API to do it.

That’s a different animal, and I believe @andy is the person you need to talk to about do this type of thing.

Sorry for the confusion.

-J

Hi,
As I wrote earlier I am working on a Realtime Renderer based on Rhino SampleRenderer example. I have made some progress as I found the IRhRdkRenderWindow::IChannel function SetValueRect:

	pChannel->SetValueRect(
		0,
		0,
		width,
		height,
		width  * sizeof(float4),
		ComponentOrder::RGBA,
		float_rhino_buffer_data);

This works and I can see my CUDA buffer in Rhino’s viewport, but unfortunately it only takes float/percentage color channels (0-1), so in order to make it work I have to convert each pixel of my CUDA buffer from uchar4 to float4 with a for loop.
Any ideas of how to insert directly a uchar imagebuffer into IRhRdkRenderWindow::IChannel ?

PS: The class IRhRdkRenderWindow contains a SetDib (const CRhinoDib &) function, I don’t know if that is how I can see the uchar Cuda buffer into Rhino’s viewport, but unfortunately I cannot find a lot of examples to try it.
https://developer.rhino3d.com/api/cpp/class_i_rh_rdk_render_window.html

any updates on this issue?

Since you have a CUDA buffer it shouldn’t be hard to create a CUDA kernel that does the conversion. Then pass on the float buffer.

Hi, this is true, it takes some milliseconds to do this conversion in CUDA, but the size of the buffer is 4 times bigger. Since I don’t really need such a color precision in Rhino if there was a way to insert an uchar4 buffer into pChannel->SetValueRect it would really increase the plugin performance.

Doing the conversion in CUDA would be way faster than for Rhino to provide the conversion for you. I don’t think offloading it to Rhino would at all increase plugin performance.

CUDA takes 1.5ms to render a sample image,
2ms to cudaMemcpy it to a host image buffer and then
70ms to load this image buffer to pChannel->SetValueRect.

If there is no other alternative to load Rhino image buffer than float4 and SetValueRect() I will keep it like this.