To try and clarify why I used FitLine here-

If you have the 3 red vectors and take their average, you get the result in green, because one of the inputs points is almost the opposite direction to the other two, so they partially cancel out:

If instead for each of the input vectors, you place a point at its end, and also at the end of the reversed vector, you can fit a line like this:

The advantage of this over choosing one of the vectors or a global direction and flipping the input vectors based on their sign relative to this is that there thereâ€™s always a chance you get unlucky and the flipping reference vector ends up being nearly perpendicular to the other inputs so the original problem still occurs.