Thank you @Eckhardt, I was able to reproduce the undesired behavior using the steps you provided. I reopened:
RH-57945 Preferences: Added advanced setting to suppress tool palettes reappearing after command completes