If I quickly click the middle mouse button no pop-up pops up, I have to click and hold for a very short time for it to pop up. Panning works as expected. But it seems like a glitch that the pop-up is activated on fast clicks.
(V5 has the same behaviour)