Unexpected result in round

Q&A's, tips, howto's

Unexpected result in round

Postby HPW » Wed Mar 20, 2019 10:34 am

Hello,

I get this unexpected result in round:
Code: Select all
> (round 8.3 -1)
8.300000000000001
> (round 8.30 -1)
8.300000000000001
> (round 8.430 -1)
8.4
> (round 8.4 -1)
8.4
> (round 8.2 -1)
8.199999999999999
>


The doc contains not this behaviour.

Regards
Hans-Peter
Hans-Peter
HPW
 
Posts: 1367
Joined: Thu Sep 26, 2002 9:15 am
Location: Germany

Re: Unexpected result in round

Postby ralph.ronnquist » Wed Mar 20, 2019 11:46 am

If you don't mind, I would point you at this fairly in-depth discussion about "representable numbers"
https://en.wikipedia.org/wiki/Floating-point_arithmetic#Representable_numbers,_conversion_and_rounding
which I think is the issue at hand here.

In short, the numbers you see printed are the nearest representable numbers to the numbers you asked for, given that the rounding is done wrt a decimal base whilst the actual values are held in binary.
ralph.ronnquist
 
Posts: 188
Joined: Mon Jun 02, 2014 1:40 am
Location: Melbourne, Australia

Re: Unexpected result in round

Postby Lutz » Wed Mar 20, 2019 2:53 pm

If you are concerned about the visual representation use format:

Code: Select all
> > (round 8.300 -1)
8.300000000000001

> (format "%2.1f" 8.300)
"8.3"
>
Lutz
 
Posts: 5269
Joined: Thu Sep 26, 2002 4:45 pm
Location: Pasadena, California

Re: Unexpected result in round

Postby HPW » Wed Mar 20, 2019 4:37 pm

Hello,

Thanks for the advices.
I am generating nc-code for a nc-drilling mashine.
Format will do the Job.

Regards
Hans-Peter
Hans-Peter
HPW
 
Posts: 1367
Joined: Thu Sep 26, 2002 9:15 am
Location: Germany


Return to newLISP in the real world

Who is online

Users browsing this forum: No registered users and 1 guest

cron