as for the error in your step #6, it just means that you cannot start a .dll for debugging, and must instead start some executable that loads it, which you do in the project properties > debugger
this question comes up often when people try to make a rhino plugin, and forget to set the path to the rhino executable here, instead of the plugin .rhp they are building
in your case though, your plugin is probably already set to start rhino for debugging, and you just need to choose it as the “Startup Project”, instead of your cuda .dll project – either way, the point is just that you need to have the debugger start rhino, instead of trying to directly execute a .dll