why the (sub) behavior like this?

Q&A's, tips, howto's

why the (sub) behavior like this?

Postby dexter » Sun Jul 19, 2015 3:11 pm

Code: Select all
(sub 1.9 0.2) will work as I thought

but (sub 20000.9 20000.2) will not return 0.7 insead of
0.700000000000728



why not just return 0.7 ??

my newlisp is 10.6.0
dexter
 
Posts: 74
Joined: Fri Nov 11, 2011 12:55 am

Re: why the (sub) behavior like this?

Postby ralph.ronnquist » Sun Jul 19, 2015 9:21 pm

Maybe it has something to do with https://en.wikipedia.org/wiki/Single-precision_floating-point_format
Try
Code: Select all
(format "%.30f" 20000.9)
ralph.ronnquist
 
Posts: 209
Joined: Mon Jun 02, 2014 1:40 am
Location: Melbourne, Australia

Re: why the (sub) behavior like this?

Postby dexter » Mon Jul 20, 2015 1:04 am

thanks

in the end ,I (mul) the number to be an integer,than I can deal the number with (-)
dexter
 
Posts: 74
Joined: Fri Nov 11, 2011 12:55 am


Return to newLISP in the real world

Who is online

Users browsing this forum: No registered users and 1 guest

cron