why the (sub) behavior like this?

Q&A's, tips, howto's
Locked
dexter
Posts: 74
Joined: Fri Nov 11, 2011 12:55 am

why the (sub) behavior like this?

Post by dexter »

Code: Select all

(sub 1.9 0.2) will work as I thought

but (sub 20000.9 20000.2) will not return 0.7 insead of 
0.700000000000728

why not just return 0.7 ??

my newlisp is 10.6.0

ralph.ronnquist
Posts: 228
Joined: Mon Jun 02, 2014 1:40 am
Location: Melbourne, Australia

Re: why the (sub) behavior like this?

Post by ralph.ronnquist »

Maybe it has something to do with https://en.wikipedia.org/wiki/Single-pr ... int_format
Try

Code: Select all

(format "%.30f" 20000.9)

dexter
Posts: 74
Joined: Fri Nov 11, 2011 12:55 am

Re: why the (sub) behavior like this?

Post by dexter »

thanks

in the end ,I (mul) the number to be an integer,than I can deal the number with (-)

Locked