I am copying @stevebaer and @Alain, as they have worked on this subject earlier.
The message pump is rather an issue, even when using Windows Forms from .Net in Rhino. That being said, were you thinking about this for the interface to a specific plug-in, or for a complete skin? My understanding is that using other libraries is discouraged because of limitations, but it might also be dependent on the type of project on which you are about to embark.
Also, unless the scope is rather limited, I fear that having at least some platform-specific UI code is almost unavoidable at present, given the state of the projects. Rhino for Win and Rhino for Mac are two distinct products, technically.
for Robert McNeel & Associates