Yes, it should work to call the KangarooSolver.dll from a Rhino plugin, without opening Grasshopper, and when doing this the Grasshopper reference isn’t needed.
The issue I see with the code you show is that the goals do not get their indexing assigned. It’s also not clear where you are initialising your instance of PhysicalSystem.
Here’s a simple example:
var PS = new KangarooSolver.PhysicalSystem();
double tol = 0.0001; // points closer than this will get combined into a single particle
var GoalList = new List<IGoal>();
//Assign indexes to the particles in each Goal:
foreach(IGoal G in x)
double threshold = 1e-10;
PS.Step(GoalList, false, threshold);
A = PS.GetOutput(GoalList);
AssignPIndex is the function which looks at the point positions and adds them to the particle list of the PhysicalSystem, and sets the index in the goal. The step method does not work until this indexing is assigned.