Setting -> Units -> Tolerance ... wired tolerance settings


#1

Hi,
on the attached image you may notice in the Field “Absolute Tolerance” that it’s set to 0,00999999.

If i switch from Centimeter to millimeter, and set absolute tolerance to e.g. 0,01 and than do the same on the “Layout” tab, as soon as i switch back to the “Model” tab the tolerance is set to 0,0099999.

If i do this once again, the absolute tolerance is correctly set to 0,01.

I think this is a bug by converting the tolerance settings.


#2

This comes up again and again. Looks like this is limited to the mac version…

A computer cannot digitally store any multiple of 10 in an exact way. So your input of 0.01 will be stored as something very close but not exactly 0.01. For display purposes, this should have been rounded to a sensible number but obviously isn’t.

Short answer: don’t worry, its just a display issue.


#3

But if i set it twice … the display is correct.


#4

the closest thing to 0.01 in double precision binary is 0.009999999999999787. That’s what will be stored if you want 0.01 tolerance.

Now I don’t know about that dialog but in order to convert the stored number to a readable string it will need to be converted and possibly rounded. There are settings that control how the number will be converted for display purposes. I guess that in between your switching the panels, that rounding logic gets reset to no rounding and you see the full 0.009999… after your second input that will be set to the intended behavior and you will see 0.01 again.

Then again, that’s a tolerance. So it’s the maximum allowable deviation. If that was 0.000000000000000213 smaller than your intended (and at that pretty arbitrary) 0.01 tolerance, would that be a problem?