Unable to start a local compute server

I ran through this guide for setting up compute on a windows server with no issues, however the server is not launching. /healthcheck returns Healthy; /activechildren returns 0.

I found the logs below in the inetpub/.../rhino.compute/logs but I can’t make out anything obvious from this stack trace.

Any ideas to further debug this?

Logs

RC   [00:36:33 INF] Max concurrent requests = 0
RC   [00:37:06 INF] Unable to start a local compute server
RC   [00:37:06 ERR] HTTP GET /version responded 500 in 60418.4128 ms
System.Exception: Unable to start a local compute server
   at rhino.compute.ComputeChildren.LaunchCompute(Queue`1 processQueue, Boolean waitUntilServing) in /home/runner/work/compute.rhino3d/compute.rhino3d/src/rhino.compute/ComputeChildren.cs:line 229
   at rhino.compute.ComputeChildren.GetComputeServerBaseUrl() in /home/runner/work/compute.rhino3d/compute.rhino3d/src/rhino.compute/ComputeChildren.cs:line 98
   at rhino.compute.ReverseProxyModule.<>c.<InitializeChildren>b__5_0() in /home/runner/work/compute.rhino3d/compute.rhino3d/src/rhino.compute/ReverseProxy.cs:line 43
   at System.Threading.ExecutionContext.RunFromThreadPoolDispatchLoop(Thread threadPoolThread, ExecutionContext executionContext, ContextCallback callback, Object state)
--- End of stack trace from previous location ---
   at System.Threading.ExecutionContext.RunFromThreadPoolDispatchLoop(Thread threadPoolThread, ExecutionContext executionContext, ContextCallback callback, Object state)
   at System.Threading.Tasks.Task.ExecuteWithThreadLocal(Task& currentTaskSlot, Thread threadPoolThread)
--- End of stack trace from previous location ---
   at rhino.compute.ReverseProxyModule.AwaitInitTask() in /home/runner/work/compute.rhino3d/compute.rhino3d/src/rhino.compute/ReverseProxy.cs:line 119
   at rhino.compute.ReverseProxyModule.ReverseProxyGet(HttpRequest req, HttpResponse res) in /home/runner/work/compute.rhino3d/compute.rhino3d/src/rhino.compute/ReverseProxy.cs:line 162
   at rhino.compute.ApiKeyMiddleware.InvokeAsync(HttpContext context) in /home/runner/work/compute.rhino3d/compute.rhino3d/src/rhino.compute/ApiKeyMiddleware.cs:line 34
   at Serilog.AspNetCore.RequestLoggingMiddleware.Invoke(HttpContext httpContext)
RC   [00:37:06 ERR] Connection ID "18374686484771897353", Request ID "4000000a-0001-ff00-b63f-84710c7967bb": An unhandled exception was thrown by the application.
System.Exception: Unable to start a local compute server
   at rhino.compute.ComputeChildren.LaunchCompute(Queue`1 processQueue, Boolean waitUntilServing) in /home/runner/work/compute.rhino3d/compute.rhino3d/src/rhino.compute/ComputeChildren.cs:line 229
   at rhino.compute.ComputeChildren.GetComputeServerBaseUrl() in /home/runner/work/compute.rhino3d/compute.rhino3d/src/rhino.compute/ComputeChildren.cs:line 98
   at rhino.compute.ReverseProxyModule.<>c.<InitializeChildren>b__5_0() in /home/runner/work/compute.rhino3d/compute.rhino3d/src/rhino.compute/ReverseProxy.cs:line 43
   at System.Threading.ExecutionContext.RunFromThreadPoolDispatchLoop(Thread threadPoolThread, ExecutionContext executionContext, ContextCallback callback, Object state)
--- End of stack trace from previous location ---
   at System.Threading.ExecutionContext.RunFromThreadPoolDispatchLoop(Thread threadPoolThread, ExecutionContext executionContext, ContextCallback callback, Object state)
   at System.Threading.Tasks.Task.ExecuteWithThreadLocal(Task& currentTaskSlot, Thread threadPoolThread)
--- End of stack trace from previous location ---
   at rhino.compute.ReverseProxyModule.AwaitInitTask() in /home/runner/work/compute.rhino3d/compute.rhino3d/src/rhino.compute/ReverseProxy.cs:line 119
   at rhino.compute.ReverseProxyModule.ReverseProxyGet(HttpRequest req, HttpResponse res) in /home/runner/work/compute.rhino3d/compute.rhino3d/src/rhino.compute/ReverseProxy.cs:line 162
   at rhino.compute.ApiKeyMiddleware.InvokeAsync(HttpContext context) in /home/runner/work/compute.rhino3d/compute.rhino3d/src/rhino.compute/ApiKeyMiddleware.cs:line 34
   at Serilog.AspNetCore.RequestLoggingMiddleware.Invoke(HttpContext httpContext)
   at Microsoft.AspNetCore.Server.IIS.Core.IISHttpContextOfT`1.ProcessRequestAsync()
RC   [00:37:13 ERR] HTTP GET /version responded 500 in 0.1890 ms
System.Exception: Unable to start a local compute server
   at rhino.compute.ComputeChildren.LaunchCompute(Queue`1 processQueue, Boolean waitUntilServing) in /home/runner/work/compute.rhino3d/compute.rhino3d/src/rhino.compute/ComputeChildren.cs:line 229
   at rhino.compute.ComputeChildren.GetComputeServerBaseUrl() in /home/runner/work/compute.rhino3d/compute.rhino3d/src/rhino.compute/ComputeChildren.cs:line 98
   at rhino.compute.ReverseProxyModule.<>c.<InitializeChildren>b__5_0() in /home/runner/work/compute.rhino3d/compute.rhino3d/src/rhino.compute/ReverseProxy.cs:line 43
   at System.Threading.ExecutionContext.RunFromThreadPoolDispatchLoop(Thread threadPoolThread, ExecutionContext executionContext, ContextCallback callback, Object state)
--- End of stack trace from previous location ---
   at System.Threading.ExecutionContext.RunFromThreadPoolDispatchLoop(Thread threadPoolThread, ExecutionContext executionContext, ContextCallback callback, Object state)
   at System.Threading.Tasks.Task.ExecuteWithThreadLocal(Task& currentTaskSlot, Thread threadPoolThread)
--- End of stack trace from previous location ---
   at rhino.compute.ReverseProxyModule.AwaitInitTask() in /home/runner/work/compute.rhino3d/compute.rhino3d/src/rhino.compute/ReverseProxy.cs:line 119
   at rhino.compute.ReverseProxyModule.ReverseProxyGet(HttpRequest req, HttpResponse res) in /home/runner/work/compute.rhino3d/compute.rhino3d/src/rhino.compute/ReverseProxy.cs:line 162
   at rhino.compute.ApiKeyMiddleware.InvokeAsync(HttpContext context) in /home/runner/work/compute.rhino3d/compute.rhino3d/src/rhino.compute/ApiKeyMiddleware.cs:line 34
   at Serilog.AspNetCore.RequestLoggingMiddleware.Invoke(HttpContext httpContext)
RC   [00:37:13 ERR] Connection ID "18374686506246733831", Request ID "40000008-0006-ff00-b63f-84710c7967bb": An unhandled exception was thrown by the application.
System.Exception: Unable to start a local compute server
   at rhino.compute.ComputeChildren.LaunchCompute(Queue`1 processQueue, Boolean waitUntilServing) in /home/runner/work/compute.rhino3d/compute.rhino3d/src/rhino.compute/ComputeChildren.cs:line 229
   at rhino.compute.ComputeChildren.GetComputeServerBaseUrl() in /home/runner/work/compute.rhino3d/compute.rhino3d/src/rhino.compute/ComputeChildren.cs:line 98
   at rhino.compute.ReverseProxyModule.<>c.<InitializeChildren>b__5_0() in /home/runner/work/compute.rhino3d/compute.rhino3d/src/rhino.compute/ReverseProxy.cs:line 43
   at System.Threading.ExecutionContext.RunFromThreadPoolDispatchLoop(Thread threadPoolThread, ExecutionContext executionContext, ContextCallback callback, Object state)
--- End of stack trace from previous location ---
   at System.Threading.ExecutionContext.RunFromThreadPoolDispatchLoop(Thread threadPoolThread, ExecutionContext executionContext, ContextCallback callback, Object state)
   at System.Threading.Tasks.Task.ExecuteWithThreadLocal(Task& currentTaskSlot, Thread threadPoolThread)
--- End of stack trace from previous location ---
   at rhino.compute.ReverseProxyModule.AwaitInitTask() in /home/runner/work/compute.rhino3d/compute.rhino3d/src/rhino.compute/ReverseProxy.cs:line 119
   at rhino.compute.ReverseProxyModule.ReverseProxyGet(HttpRequest req, HttpResponse res) in /home/runner/work/compute.rhino3d/compute.rhino3d/src/rhino.compute/ReverseProxy.cs:line 162
   at rhino.compute.ApiKeyMiddleware.InvokeAsync(HttpContext context) in /home/runner/work/compute.rhino3d/compute.rhino3d/src/rhino.compute/ApiKeyMiddleware.cs:line 34
   at Serilog.AspNetCore.RequestLoggingMiddleware.Invoke(HttpContext httpContext)
   at Microsoft.AspNetCore.Server.IIS.Core.IISHttpContextOfT`1.ProcessRequestAsync()
RC   [00:37:16 ERR] HTTP GET /sdk responded 500 in 0.1718 ms
System.Exception: Unable to start a local compute server
   at rhino.compute.ComputeChildren.LaunchCompute(Queue`1 processQueue, Boolean waitUntilServing) in /home/runner/work/compute.rhino3d/compute.rhino3d/src/rhino.compute/ComputeChildren.cs:line 229
   at rhino.compute.ComputeChildren.GetComputeServerBaseUrl() in /home/runner/work/compute.rhino3d/compute.rhino3d/src/rhino.compute/ComputeChildren.cs:line 98
   at rhino.compute.ReverseProxyModule.<>c.<InitializeChildren>b__5_0() in /home/runner/work/compute.rhino3d/compute.rhino3d/src/rhino.compute/ReverseProxy.cs:line 43
   at System.Threading.ExecutionContext.RunFromThreadPoolDispatchLoop(Thread threadPoolThread, ExecutionContext executionContext, ContextCallback callback, Object state)
--- End of stack trace from previous location ---
   at System.Threading.ExecutionContext.RunFromThreadPoolDispatchLoop(Thread threadPoolThread, ExecutionContext executionContext, ContextCallback callback, Object state)
   at System.Threading.Tasks.Task.ExecuteWithThreadLocal(Task& currentTaskSlot, Thread threadPoolThread)
--- End of stack trace from previous location ---
   at rhino.compute.ReverseProxyModule.AwaitInitTask() in /home/runner/work/compute.rhino3d/compute.rhino3d/src/rhino.compute/ReverseProxy.cs:line 119
   at rhino.compute.ReverseProxyModule.ReverseProxyGet(HttpRequest req, HttpResponse res) in /home/runner/work/compute.rhino3d/compute.rhino3d/src/rhino.compute/ReverseProxy.cs:line 162
   at rhino.compute.ApiKeyMiddleware.InvokeAsync(HttpContext context) in /home/runner/work/compute.rhino3d/compute.rhino3d/src/rhino.compute/ApiKeyMiddleware.cs:line 34
   at Serilog.AspNetCore.RequestLoggingMiddleware.Invoke(HttpContext httpContext)
RC   [00:37:16 ERR] Connection ID "18230571305285976065", Request ID "40000002-0003-fd00-b63f-84710c7967bb": An unhandled exception was thrown by the application.
System.Exception: Unable to start a local compute server
   at rhino.compute.ComputeChildren.LaunchCompute(Queue`1 processQueue, Boolean waitUntilServing) in /home/runner/work/compute.rhino3d/compute.rhino3d/src/rhino.compute/ComputeChildren.cs:line 229
   at rhino.compute.ComputeChildren.GetComputeServerBaseUrl() in /home/runner/work/compute.rhino3d/compute.rhino3d/src/rhino.compute/ComputeChildren.cs:line 98
   at rhino.compute.ReverseProxyModule.<>c.<InitializeChildren>b__5_0() in /home/runner/work/compute.rhino3d/compute.rhino3d/src/rhino.compute/ReverseProxy.cs:line 43
   at System.Threading.ExecutionContext.RunFromThreadPoolDispatchLoop(Thread threadPoolThread, ExecutionContext executionContext, ContextCallback callback, Object state)
--- End of stack trace from previous location ---
   at System.Threading.ExecutionContext.RunFromThreadPoolDispatchLoop(Thread threadPoolThread, ExecutionContext executionContext, ContextCallback callback, Object state)
   at System.Threading.Tasks.Task.ExecuteWithThreadLocal(Task& currentTaskSlot, Thread threadPoolThread)
--- End of stack trace from previous location ---
   at rhino.compute.ReverseProxyModule.AwaitInitTask() in /home/runner/work/compute.rhino3d/compute.rhino3d/src/rhino.compute/ReverseProxy.cs:line 119
   at rhino.compute.ReverseProxyModule.ReverseProxyGet(HttpRequest req, HttpResponse res) in /home/runner/work/compute.rhino3d/compute.rhino3d/src/rhino.compute/ReverseProxy.cs:line 162
   at rhino.compute.ApiKeyMiddleware.InvokeAsync(HttpContext context) in /home/runner/work/compute.rhino3d/compute.rhino3d/src/rhino.compute/ApiKeyMiddleware.cs:line 34
   at Serilog.AspNetCore.RequestLoggingMiddleware.Invoke(HttpContext httpContext)
   at Microsoft.AspNetCore.Server.IIS.Core.IISHttpContextOfT`1.ProcessRequestAsync()
RC   [00:37:21 INF] HTTP GET /activechildren responded 200 in 1.9911 ms
RC   [00:37:23 INF] HTTP GET /activechildren responded 200 in 2.2065 ms
RC   [00:37:25 INF] Starting compute.geometry instance on port 6001
RC   [00:37:33 INF] Max concurrent requests = 0

I have some ideas, but wanted to see if folks might know before I dig deeper.

  • config issue?
  • port binding conflict?
  • license issues?
  • permission/firewall?

More details:

# is this expected?
echo %RHINO_COMPUTE_URLS%
http://+:80
# web.config
<?xml version="1.0" encoding="utf-8"?>
<configuration>
  <system.webServer>
    <handlers>
      <add name="aspNetCore" path="*" verb="*" modules="AspNetCoreModuleV2" resourceType="Unspecified"/>
    </handlers>
    <aspNetCore processPath=".\rhino.compute.exe" arguments="--port 80" stdoutLogEnabled="true" stdoutLogFile="..\..\..\..\..\logs\LogFiles\W3SVC1\" forwardWindowsAuthToken="true" hostingModel="InProcess"/>
  </system.webServer>
</configuration>

I did have some errors on one of the steps but eventually it did run through the end.
I don’t recall, but it’s possible forgot to run as admin 1st time.

bootstrap_step-1_log.txt
bootstrap_step-2_log.txt

Can you upload the bootstrap logs (located on the C:/ drive) so I can see if anything odd happened during the installation process? I think since Rhino 8 is loading (per your screenshot), we can rule out that it’s a license issue.

Oops looks like I edited my reply above instead of a new one :man_facepalming: :laughing:
:backhand_index_pointing_up:

Ok. Yeah, I see a few things that stand out to me. In the first step, there’s a few messages that say:

===>  Configuring URL reservation (80 and 443)

Invalid Top Level command name: (null)

Invalid Top Level command name: (null)

Invalid command name: (null)

Invalid command name: (null)
Ok.

Invalid Top Level command name: (null)

Invalid Top Level command name: (null)

Invalid command name: (null)

Invalid command name: (null)
Ok.

So, that’s troubling. And then in step 2 there are a ton of errors (I’m assuming you ran this several times) but it does look like it completed eventually at the end.

What happens if you open a powershell window and navigate to C:\inetpub\wwwroot\aspnet_client\system_web\4_0_30319\rhino.compute. Then enter the following:

.\rhino.compute.exe --spawn-on-startup

Essentially, I want to manually start the rhino.compute.exe process via the command line and see if there are any messages that show up.
PS, if you haven’t already done so, I would recommend adding an environment variable for getting more verbose logging information printed to the console output. You can read more on how to do this here.

Ha, I think that’s the smoking gun:

Weird. Because you can see in the step 2 logs that it says this:

===>  Download ASP.NET Core 8.0 Hosting Bundle
dotnet-hosting-8.0.14-win.exe downloaded

Still, maybe try to download and manually install it again and see if it works? You should be able to get the correct version here.

Already installed - will try reinstalling just in case

What happens if you type

dotnet --list-runtimes

into a powershell window. Do you see a version of .NET specifically for ASP (not desktop)? You need to have the ASP version of the NET runtime in order for compute to work. The next official release of Rhino (8.21 I think) should automatically install this for you, but that won’t be released for about another month. The hosting bundle in the bootstrap script should have been fine, but I’m curious what shows up when you run the command above.

Ok. But it still says that it won’t work if you try to run rhino.compute via the command line?

Yep same.

I could try installing the Runtime that the error message suggested?

That seems to have worked :crossed_fingers:

Wohoo. Thank you so much @AndyPayne

In retrospect, some ideas related to the my debugging process:

  1. Could bootstrap scripts assert elevated permissions? otherwise, they may run and confuse users with errors when it was always destined to fail. Also helps distracted users like me that tried running them in the wrong shell.

  2. Could the output of the spawned rhino.compute.exe process (stdout/err) be collected by the compute server and surfaced to the user in the stack trace (here?). Or maybe into another dedicated log file that can be documented?
    That would have taken be straight to the root cause directly from the logs.

Those are all good suggestions. I’ll see what I can do.