I guess there must be a way to remove I was asking myself to remove outliers from a pointcloud.
As shown in the picture I want to remove the points on the outside.

I have an idea to calculate each points distance to ti’s closest point, but I think this might be a way too tremendous calculation, right?

I know that there are tools like Cockroach, but in this case I would like to be able to control this myself and I would like to understand the idea and approaches behind.

A few solutions come to mind depending on how reliable / fast you want your algorithm to be

1)Easiest to implement, very reliable, but computationally expensive (slower):
You could calculate the mean and standard deviation of the distances between each point and its k-nearest neighbors, and then remove any points that are more than some multiple of the standard deviation away from the mean.
2) Efficient but less reliable :
A sort of density-based algorithm (such as DBSCAN) to identify clusters of points within the cloud, and then remove any points that are not part of a cluster.

An implementation of the Convex Hull Algorithm could also work I guess, but I’d both less reliable and not that fast