Page 1 of 1
Unexpected result in round
Posted: Wed Mar 20, 2019 10:34 am
by HPW
Hello,
I get this unexpected result in round:
Code: Select all
> (round 8.3 -1)
8.300000000000001
> (round 8.30 -1)
8.300000000000001
> (round 8.430 -1)
8.4
> (round 8.4 -1)
8.4
> (round 8.2 -1)
8.199999999999999
>
The doc contains not this behaviour.
Regards
Hans-Peter
Re: Unexpected result in round
Posted: Wed Mar 20, 2019 11:46 am
by ralph.ronnquist
If you don't mind, I would point you at this fairly in-depth discussion about "representable numbers"
https://en.wikipedia.org/wiki/Floating- ... d_rounding
which I think is the issue at hand here.
In short, the numbers you see printed are the nearest representable numbers to the numbers you asked for, given that the rounding is done wrt a decimal base whilst the actual values are held in binary.
Re: Unexpected result in round
Posted: Wed Mar 20, 2019 2:53 pm
by Lutz
If you are concerned about the visual representation use format:
Code: Select all
> > (round 8.300 -1)
8.300000000000001
> (format "%2.1f" 8.300)
"8.3"
>
Re: Unexpected result in round
Posted: Wed Mar 20, 2019 4:37 pm
by HPW
Hello,
Thanks for the advices.
I am generating nc-code for a nc-drilling mashine.
Format will do the Job.
Regards
Hans-Peter