Sounds annoying to me. I’m having difficult time reproducing this in the latest WIP (5A794w). I’ve opened four windows in the same OS X space and have been panning and zooming with my Magic Mouse around four different models for the past 5 minutes. Everything is working fine.
Can you please post your System Information (Rhinoceros > About Rhinoceros > More Info… button > Copy to clipboard) as well as your Magic Mouse settings in Rhino’s Preferences? Perhaps others would have better luck reproducing this than I.
Please post any other steps you find to make this happen repeatedly.