Unexpected result in round

Q&A's, tips, howto's
Locked
HPW
Posts: 1390
Joined: Thu Sep 26, 2002 9:15 am
Location: Germany
Contact:

Unexpected result in round

Post by HPW »

Hello,

I get this unexpected result in round:

Code: Select all

> (round 8.3 -1)
8.300000000000001
> (round 8.30 -1)
8.300000000000001
> (round 8.430 -1)
8.4
> (round 8.4 -1)
8.4
> (round 8.2 -1)
8.199999999999999
>
The doc contains not this behaviour.

Regards
Hans-Peter
Hans-Peter

ralph.ronnquist
Posts: 228
Joined: Mon Jun 02, 2014 1:40 am
Location: Melbourne, Australia

Re: Unexpected result in round

Post by ralph.ronnquist »

If you don't mind, I would point you at this fairly in-depth discussion about "representable numbers"
https://en.wikipedia.org/wiki/Floating- ... d_rounding
which I think is the issue at hand here.

In short, the numbers you see printed are the nearest representable numbers to the numbers you asked for, given that the rounding is done wrt a decimal base whilst the actual values are held in binary.

Lutz
Posts: 5289
Joined: Thu Sep 26, 2002 4:45 pm
Location: Pasadena, California
Contact:

Re: Unexpected result in round

Post by Lutz »

If you are concerned about the visual representation use format:

Code: Select all

> > (round 8.300 -1)
8.300000000000001

> (format "%2.1f" 8.300)
"8.3"
> 

HPW
Posts: 1390
Joined: Thu Sep 26, 2002 9:15 am
Location: Germany
Contact:

Re: Unexpected result in round

Post by HPW »

Hello,

Thanks for the advices.
I am generating nc-code for a nc-drilling mashine.
Format will do the Job.

Regards
Hans-Peter
Hans-Peter

Locked