ML/AI Image Recognition Plugin For Grasshopper?


Does anyone know of any tools for Rhino/GH that can take an image input (such as a floor plan .jpg, .png, or .pdf) and create a prediction of what elements it sees in the image?

An example is uploading an architectural floorplan (ideally hand sketched) into GH, running the predictive logic, and returning results such as a colored bounding box around similar objects with a description or value of what it assumes it is, aka “Door” “Window” “Furniture” “Unknown”.

Similar to what the Rhino Viewer app is doing on IOS by leveraging the LiDAR data of a scan and Apple RoomScan API but from a 2D image input rather than the 3D LiDAR. (I realize most of the heavy lifting here is probably on Apple’s end of the API)

Use case example is a lot of designers still hand sketch plans and having even a general starting point of a 2D/3D model from that data would be very helpful as you all know it always takes a bit of upfront work going from 2D (particularly non-vector images) to even a first pass 3D representation.

I have tested out some forum posted scripts that vectorize line work and then extruded those curves and such but really what I am after is the classification of “this is probably a wall” “this is probably a door” and the generic bounding box of said items. I can do more with this data then some curves that have no classification.

Thanks for any leads or ideas!

Am I seeing this correctly? No replies? This seems like such a great use for AI that alot of architects and engineers could benefit from.

Nothing so far! In re-reading my post, what I am probably after is AI image classification.

I’m curious if any trained data sets cover architectural drawing as this would be important to classify the images/line work found in a drawing.

Perhaps a new model could be trained for the purpose.