Error: 2 grasshopper.dll files

When trying to create a component with visual studio, I had problems with conflicting versions. Therefore I deleted the NuGet packages (RhinoCommon, Grasshopper) and added as assembly references the ones that the NuGet package was using. So now those dlls are both in the Programs/Rhino7 folder and in the Grasshopper/Libraries one. It still works, it’s just that when I open Grasshopper it throws the error message that there are two Grasshopper.dll files paths. Could I delete the references assemblies in the Grasshopper/Libraries while keep the .gha file working?