Strategy for removing points inside PointCloud

So the main issue here seems to be determining the boundary of the point cloud?

Otherwise, something along these lines might work: