OK, now I better understand why you want something besides nth.
So far I have only decimated non-critical clouds, the exterior scans of buildings where the primary focus is on the interior scans. After decimating the exterior clouds by 3-5X, the interior ones do not need to be decimated in order to get reasonable performance in Rhino.
As you know, I use Python and C++ for all my scripts. C# is somewhere between these, having more complex elements than C++ but not as complex as Python. I do all my development work in Python which is the fastest language for development I have ever used (and I date back to the time where there were no transistors, no practical computers and no languages). Then once I have all the bugs worked out, all the what if experiments done, all the graphical aids used to help the development explored, I just copy all the code to C++ and fix up the syntax differences and replace the Python specific elements with C++ equivalents. I almost never use containers or any complex elements in C++ if I want the best speed. I just use bare-bones basic elements. And the result is code that chews thru data at 2-5 GB/sec which is to be expected on a 4 GHz CPU. But if you use containers and other such fancy things, this typically drops below 1 GB/sec. Anyway, that’s my take on getting the best performance possible. C# may allow you to get close if you stick to the more basic elements and it does have the advantage of fancier elements that can be used for more logical coding in non-time critical sections of the code. But then I just do these in Python which has even fancier elements and call a C++ procedure from Python whenever I need performance.
Getting back to Voxels. Are you using these in order to retain maximum overall information in a given data volume or do some algorithms you use only work with iso-density cloud?