I am just looking for a general discussion here and not for direct script advice.
For example, if I take Paris for instance, I have 500,000 curves of a city which is the street network. If I want to do a city analyse or join curves, grasshopper computationally seriously struggles and I have a pretty good machine. In GIS this command takes seconds to do.
Is this just a limitation of Grasshopper, I thought about coding my own customised components but pretty sure the Grasshopper Join Curves is pretty optimised already. Or is there work arounds for it handling large data-sets.
The join in Rhino (which Grasshopper uses) is indeed pretty optimised and I doubt you can improve much upon it. It is possible GIS uses an entirely different approach to curves which makes joining them much more efficient. Are all the curves polylines in such a model? Because that limitation might speed things up considerably. A polyline is just an array of coordinates, so joining two polylines into one is just:
Allocate a block of memory big enough for n+m-1 coordinates.
Block copy the memory from the first polyline into the [0, n-1] range.
Block copy the memory (except for the first coordinate) from the second polyline into the [n, n+m-1] range.
These three are all O(1) operations. Possible complication is that one or two of the arrays need to be reversed prior to copying, and whether that’s O(1) or O(n) depends on the memory layout of the polyline type.
Joining together curves of various types into polycurves is just more complicated.
Polyline is a kind of curve, but Polyline has a faster joining algorithm whereas it doesn’t apply to general curves. Rhino/GH’s “Join Curve” works for all curves, so Rhino would use a much slower approach.