I dont think its about model vs layout space. Once i want my text to be 3mm high it will be 0,003m big in model space whether its in meters and 3mm big when i am in layouts.
Now its 3 “units” big which can be anything thus confusing (different units in model/layout space…).
In length units section you can define exacly the units you want your dimensions to be. I dont know why everything else is relative.
You need to have different annotation styles depending on units you are using which is excessive.
I cannot think of a case where you would prefer your dimension styles to be set dependent on model units rather than be strictly defined in real units.