Page 1 of 1

why the (sub) behavior like this?

PostPosted: Sun Jul 19, 2015 3:11 pm
by dexter
Code: Select all
(sub 1.9 0.2) will work as I thought

but (sub 20000.9 20000.2) will not return 0.7 insead of
0.700000000000728



why not just return 0.7 ??

my newlisp is 10.6.0

Re: why the (sub) behavior like this?

PostPosted: Sun Jul 19, 2015 9:21 pm
by ralph.ronnquist
Maybe it has something to do with https://en.wikipedia.org/wiki/Single-precision_floating-point_format
Try
Code: Select all
(format "%.30f" 20000.9)

Re: why the (sub) behavior like this?

PostPosted: Mon Jul 20, 2015 1:04 am
by dexter
thanks

in the end ,I (mul) the number to be an integer,than I can deal the number with (-)