What you’re looking for are generalised barycentric coordinates, of which probably the most popular form is mean value coordinates.
There’s also a generalisation of this to map points in 3d within closed meshes, which is what I implemented for the Mesh Cage Morph (Skeleton fattener + mesh cage morph)
but along the way, I also tried the 2d polygon version:
meanvaluecoordinates2d.gh (13.7 KB)