Q&A's, tips, howto's
kosh
Posts: 72 Joined: Sun Sep 13, 2009 5:38 am
Location: Japan
Contact:
Post
by kosh » Fri Feb 18, 2011 1:48 am
Hello, Lutz.
The problem that does not correctly convert float to integer was found.
Code: Select all
newLISP v.10.3.0 on Linux IPv4/6 UTF-8, execute 'newlisp -h' for more info.
> (setq x (exp (gammaln 6)))
120
> (float? x)
true
> (int x)
119
> (bits x)
"1110111" ; (int "1110111" 0 2) is 119
> (bits 120.0)
"1111000"
> (dump x)
(147342976 387 147335024 -9 1079902207)
Lutz
Posts: 5289 Joined: Thu Sep 26, 2002 4:45 pm
Location: Pasadena, California
Contact:
Post
by Lutz » Fri Feb 18, 2011 2:20 am
Floats in newLISP are displayed using a default format string "%1.10g". You see a rounding effect. When formatting for more precision, you see that the value is just under 120.
Code: Select all
> (setq x 119.99999999999976)
120
> (format "%5.14f" x)
"119.99999999999976"
>
You can change the default print format for floats using the 'pretty-print' function:
Code: Select all
> (pretty-print)
(80 " " "%1.10g")
> (pretty-print 80 " " "%1.15f")
(80 " " "%1.15f")
> (setq x (exp (gammaln 6)))
119.999999999999758
>
kosh
Posts: 72 Joined: Sun Sep 13, 2009 5:38 am
Location: Japan
Contact:
Post
by kosh » Fri Feb 18, 2011 7:55 am
I've understood this problem.
Thanks, Lutz.