A few solutions come to mind depending on how reliable / fast you want your algorithm to be
1)Easiest to implement, very reliable, but computationally expensive (slower):
You could calculate the mean and standard deviation of the distances between each point and its k-nearest neighbors, and then remove any points that are more than some multiple of the standard deviation away from the mean.
2) Efficient but less reliable :
A sort of density-based algorithm (such as DBSCAN) to identify clusters of points within the cloud, and then remove any points that are not part of a cluster.
An implementation of the Convex Hull Algorithm could also work I guess, but I’d both less reliable and not that fast