Unable to extract mesh colours from photogrammetry scan

[ferg_m]

I’m playing around with some photogrammetry scans to try and extract some colour data from them to use in a separate workflow

I’m unable to retrieve the vertex or face colour data from the mesh in grasshopper after deconstructing the mesh.

I’ve been using Meshroom to generate the images into a mesh, and have been iterating through various export settings to try and get a better result but I haven’t had much luck there either.

Frustratingly I cannot upload the rhino file due the the file size limit - but I have internalized a small snippet of the mesh in the grasshopper file

Any assistance and guidance would be much appreciated

colour extraction tests.gh (1.3 MB)

[martinsiegrist]

Did you try _ComputeVertexColors in Rhino?

[laurent_delrieu]

If you want to use just grasshopper, you can extract UV with C#/Python or plugin then use image sampler.

https://developer.rhino3d.com/api/rhinocommon/rhino.geometry.mesh#properties

[ferg_m]

Hi @martinsiegrist
Thankyou for the fat reply
I did try this - I found a previous post where this was a recommended approach. But unfortunately this did not work in my case

[ferg_m]

Hi @laurent_delrieu
Thank you for the fast reply

With this workflow I am trying to extract the colours of the mesh from the scan rather than applying a new colour to the input mesh
One of the outputs from the processing the images in Meshroom is some jpeg images of the mesh unrolled

But when i apply this to the mesh colouring the outcome is chaos… as we would expect

My goal is to get the color values from the textured mesh as shown below:

@martinsiegrist @laurent_delrieu
the end goal here is to extract curves from specific coloured mesh like what’s been achieved in this post from a while back:
How to select vertices with a specific colour, and fit curve through a subset of these vertices? - Grasshopper - McNeel Forum

But I cant seem to get past this early step

If you’ve had some success with this photogrammetry workflow on a previous project I’m open to hearing it

Thanks again

[martinsiegrist]

You might just have to share your *.obj with the texture *.png so someone can take a look.

[laurent_delrieu]

I think Graft is needed

I remake a mesh to be sure it is correct.

Withj Graft

[laurent_delrieu]

But UV is a bit strange, it is more than [0,1][0,1] !!!

Perhaps it is for each mesh incremented in X !!!

Or one image per file.

Quite hard to work with half information.

[ferg_m]

Yes I agree

I’ve tried to zip the file and textures a few times but am still above the max file limit that I can attach to a message

@laurent_delrieu @martinsiegrist
Would either of you be open to me sharing the .obj file via a WeTransfer link?

[laurent_delrieu]

No problem, I have time this night. Message if not working on Discourse

[ferg_m]

@laurent_delrieu @martinsiegrist
Thanks again - really appreciate the time and help

The link to the WeTransfer files below:

texturedMesh

The link contains the textured obj file, the rhino and grasshopper file, and the unrolled mesh jpegs

I’ve set the link expiry for 3 days if an onlooker also wants to have a go

[laurent_delrieu]

As advised by @martinsiegrist why not using computevertexcolor. It works ! And it will be more easy in Grasshopper.

*****************************************************

In Grasshopper this is working as expected here, UV are all in range [0, 1]

I really don’t understand how you had some files with UV not in [0, 1] range !!!

Here a Grasshopper file

colour extraction tests LD_2.gh (6.6 MB)

*************************************************

You can also extract colors using UV coordinates and some index, here an attend but I was thinking UV were with range greater than [0 1] on X !!!

```cs

public class Script_Instance : GH_ScriptInstance

{

#region Notes

/\* 

  Members:

    RhinoDoc RhinoDocument

    GH_Document GrasshopperDocument

    IGH_Component Component

    int Iteration



  Methods (Virtual & overridable):

    Print(string text)

    Print(string format, params object\[\] args)

    Reflect(object obj)

    Reflect(object obj, string method_name)

\*/

#endregion



private void RunScript(

    List<Point3d> UVs,

    List<string> filesNames,

    ref object colors)

{

     List<Bitmap> bitmaps = new List<Bitmap>();

     foreach (string fileName in filesNames)

     {

        bitmaps.Add(new Bitmap(fileName));

     }

    // Write your logic here

    colors = ExtractColorsFromBitmaps(UVs,bitmaps);

}

 static List<Color> ExtractColorsFromBitmaps(List<Point3d> points, List<Bitmap> bitmaps)

    {

        List<Color> colors = new List<Color>();



        // Process each point and bitmap

        for (int i = 0; i < points.Count; i++)

        {

            int bitmapIndex = (int)points\[i\].X;



               if (bitmapIndex >=0 && bitmapIndex< bitmaps.Count)

            {

                // Convert normalized coordinates to pixel coordinates

                int pixelX = (int)(points\[i\].X \* (bitmaps\[bitmapIndex\].Width - 1));

                int pixelY = (int)(points\[i\].Y \* (bitmaps\[bitmapIndex\].Height - 1));



                // Get the color at the pixel

                Color color = bitmaps\[bitmapIndex\].GetPixel(pixelX, pixelY);

                colors.Add(color);

            }

        }

        return colors;

    }

}

[laurent_delrieu]

For this it could also be better to directly compute everything during the obj reading. But mtl must be present in order to read the correct file.

[ferg_m]

Hi @laurent_delrieu
Thankyou! super clear in what i was missing!
I guess I must have missed a step in the computevertexcolor process and not ported that into grasshopper after, as now its working for me as well - Also thankyou @martinsiegrist

@laurent_delrieu
I’ll have a deeper look into Meshrooms export settings so that mtl can be present in the file creation

Thanks again - you both have been a big help!

[crz_06]

side question, why do you use mesh and not pointcloud? I my experience faster then meshes and easier to clean

[ferg_m]

Honestly I haven’t tried to export it as one
Meshroom has a default photogrammetry workflow that I was using that doesn’t give the option of point clouds as an export format.

But after a quick google search I can see that there is a node in meshroom that can do that
Thanks for the tip