We intentionally removed the capability to customize what mouse behavior would invoke each of the possible actions for a tool button. All the original actions are still there - they now have fixed mouse button actions assigned to them.
We found that people would click the box icons without understanding that they were changing how tool buttons worked, and then we would get support questions about why their tool buttons no longer worked. The previous customization was too subtle and too easy to inadvertently change.
Now if you want to display a tool palette that is attached to a tool button, either right-click and hold, option left click and hold, or control left click and hold on the tool button.