You’re not alone. I have problems with it, too. Especially when I make a mistake clicking or dragging from an inopportune spot, with some elements of the UI being super sensitive to mouse movements and others just not, it’s impossible to undo without putting the mouse in a totally different place to repair it. It just doesn’t make sense. Drives me insane. R7 was intuitive in the way you’re describing, like most software since like the mid 2000s.
I hope and wish there were a different way than R8’s choice. Gotta be somehow.