development release newLISP v.9.2.6

Lutz · Post by **Lutz** » Wed Nov 21, 2007 6:35 pm

Additions and changes in both, the core newLISP executable and newLISP-GS.

for files and changes notes see: http://newlisp.org/downloads/development/

Lutz

cormullion · Post by **cormullion** » Thu Nov 22, 2007 8:23 am

So far it works fine! (MacOS X Tiger 10.4.11 Intel). And more stuff in the manual too.

Lutz · Post by **Lutz** » Thu Nov 22, 2007 1:04 pm

And more stuff in the manual too.

any improvements to the writing in the new chapter "17. Object Oriented Programming in newLISP" are greatly appreciated.

Lutz

newdep · Post by **newdep** » Thu Nov 22, 2007 5:17 pm

Hi Lutz,

Slackware 11.0 (where 9.2.5 compiled fine)>

$make linux_readline

make -f makefile_linux_readline
make[1]: Entering directory `/newlisp-9.2.6'
gcc -Wall -pedantic -Wno-uninitialized -Wno-strict-aliasing -Wno-long-long -c -O2 -g -DREADLINE -DLINUX newlisp.c
gcc -Wall -pedantic -Wno-uninitialized -Wno-strict-aliasing -Wno-long-long -c -O2 -g -DREADLINE -DLINUX nl-symbol.c
gcc -Wall -pedantic -Wno-uninitialized -Wno-strict-aliasing -Wno-long-long -c -O2 -g -DREADLINE -DLINUX nl-math.c
gcc -Wall -pedantic -Wno-uninitialized -Wno-strict-aliasing -Wno-long-long -c -O2 -g -DREADLINE -DLINUX nl-list.c
gcc -Wall -pedantic -Wno-uninitialized -Wno-strict-aliasing -Wno-long-long -c -O2 -g -DREADLINE -DLINUX nl-liststr.c
gcc -Wall -pedantic -Wno-uninitialized -Wno-strict-aliasing -Wno-long-long -c -O2 -g -DREADLINE -DLINUX nl-string.c
gcc -Wall -pedantic -Wno-uninitialized -Wno-strict-aliasing -Wno-long-long -c -O2 -g -DREADLINE -DLINUX nl-filesys.c
gcc -Wall -pedantic -Wno-uninitialized -Wno-strict-aliasing -Wno-long-long -c -O2 -g -DREADLINE -DLINUX nl-sock.c
gcc -Wall -pedantic -Wno-uninitialized -Wno-strict-aliasing -Wno-long-long -c -O2 -g -DREADLINE -DLINUX nl-import.c
gcc -Wall -pedantic -Wno-uninitialized -Wno-strict-aliasing -Wno-long-long -c -O2 -g -DREADLINE -DLINUX nl-xml.c
gcc -Wall -pedantic -Wno-uninitialized -Wno-strict-aliasing -Wno-long-long -c -O2 -g -DREADLINE -DLINUX nl-web.c
gcc -Wall -pedantic -Wno-uninitialized -Wno-strict-aliasing -Wno-long-long -c -O2 -g -DREADLINE -DLINUX nl-matrix.c
gcc -Wall -pedantic -Wno-uninitialized -Wno-strict-aliasing -Wno-long-long -c -O2 -g -DREADLINE -DLINUX nl-debug.c
gcc -Wall -pedantic -Wno-uninitialized -Wno-strict-aliasing -Wno-long-long -c -O2 -g -DREADLINE -DLINUX pcre.c
gcc newlisp.o nl-symbol.o nl-math.o nl-list.o nl-liststr.o nl-string.o nl-filesys.o nl-sock.o nl-import.o nl-xml.o nl-web.o nl-matrix.o nl-debug.o pcre.o -g -lm -ldl -lreadline -lncurses -o newlisp
/usr/lib/gcc/i486-slackware-linux/3.4.6/../../../crt1.o(.text+0xc): In function `_start':
../sysdeps/i386/elf/start.S:109: undefined reference to `__libc_csu_fini'
/usr/lib/gcc/i486-slackware-linux/3.4.6/../../../crt1.o(.text+0x11):../sysdeps/i386/elf/start.S:110: undefined reference to `__libc_csu_init'
collect2: ld returned 1 exit status
make[1]: *** [default] Error 1
make[1]: Leaving directory `/newlisp-9.2.6'
make: *** [linux_readline] Error 2
$

Lutz · Post by **Lutz** » Thu Nov 22, 2007 5:43 pm

I bet 9.2.5 doesn't compile anymore on your machine too. There is only one algorithmic change for Unix/Linux in nl-math.c/p_rand(), and which doesn't involve any different library calls. All other changes only involve Win32 and Java in newLISP-GS.

Lutz

newBert · Post by **newBert** » Thu Nov 22, 2007 6:34 pm

I've got some problems with special characters (in my language) in the version 9.2.6 !

The characters with an accent (e.g. "é" or "è") or other special character ("ç") count for two characters.

Example :
NewLISP v.2.9.5 :
(set 'str "Stéphanie")
(println (str 0) (str 3))
=> Sp

With NewLISP v.9.2.6 :
I must do
(println (str 0) (str 4))
to obtain
=> Sp
because of the "é" , at index #2 (and #3 !)

Sorry, if I'm not clear.

I come back to v.9.2.5 for the moment.

P.S.: I'm on Windows XP ...

newdep · Post by **newdep** » Thu Nov 22, 2007 6:56 pm

You already knew i was stupid ;-)

..The problem isnt newlisp..

Im working on my 2nd linux machine which has an nfs mount.
The P4 wont compile, the AMD64 does.. machines are copy's
so thats odd... Anyway i need to throw something out of the attic window..

Lutz · Post by **Lutz** » Thu Nov 22, 2007 7:19 pm

Unfortunately I posted the Win32 version of newLISP 9.2.6 with the newLISP-GS UTF-8 Java GUI, but the newlisp.exe excutable is compiled as non UTF-8 version. I will repost newlisp-9206-win-gs-107.exe tomorrow morning, where the executable is UTF-8 too.

In version 9.2.6 (newLISP-GS) characters are UTF-8 encoded. In UTF-8 all non-ASCII characters are encoded in 2 to 4 bytes, i.e. the accented é is encoded as "\195\169", a double byte character.

The newLISP executable if compiled as UTF-8 knows about it and when doing indexing as in (str 0) (str 3) etc. can pick out variable-byte-wide characters correctly.

In the future (as already happening on Mac OS X) all binary packages will be shipped full UTF-8. The world is growing together and it makes a lot of sense to use a character set like UTF-8 able to display any character in the world. On the web we have most pages already encoded in UTF-8 and web browsers know how to handle this.

You will then be able to use no-ascii multibyte characters for names, strings and programming text in newLISP, and these programs and their output will look identical on all newLISP-GS platforms, no matter if the computer is running in France, Russia, Turkey or China.

When running the UTF-8 version of newLISP be aware that most (but not all) string functions work on character and not byte boundaries. This chapter: http://newlisp.org/downloads/newlisp_ma ... icode_utf8 contains a list of those functions.

Lutz

Lutz · Post by **Lutz** » Thu Nov 22, 2007 7:38 pm

Just reposted newlisp-9206-win-gs-107.exe as consistent UTF-8 for the GUI part and the executable newlisp.exe.

see: http://newlisp.org/downloads/development/

Lutz

HPW · Post by **HPW** » Thu Nov 22, 2007 10:50 pm

The new UTF compiled DLL broke my turtls-demo in the neobook demo pub.
No idea what's the problem is.
Have to futher investigate.

HPW · Post by **HPW** » Thu Nov 22, 2007 11:00 pm

9.2.6 also broke my prodcut configurator (from the contest) which runs fine with 9.2.5!

Still searching for the difference.

HPW · Post by **HPW** » Thu Nov 22, 2007 11:17 pm

When I load my turtle.lsp with this line:

(set 'direction -425.6857918)

I get a '12 symbol expected in function set : .6857918'

Seems it is now starting with a different default locale then before.
When I use (set-locale "C") the turtle demo is running again.

Lutz · Post by **Lutz** » Fri Nov 23, 2007 2:08 am

When I use (set-locale "C") the turtle demo is running again.

The UTF-8 version of the newlisp executable switches to the local locale on startup. The non-UTF-8 version enables the 'C' locale on startup.

(set 'direction -425.6857918) ... I get a '12 symbol expected in function set : .6857918'

the switch to the local (German) locale on the UTF-8 version enables the comma as a decimal separator, so the: -425.6857918 now gets parsed as -425 and .6857918 making the 'set' statement fail. The "C" locale assumes the dot "." as the decimal separator.

Lutz

HPW · Post by **HPW** » Fri Nov 23, 2007 7:11 am

Still having troubles with my product configurator.

What is the benefit of having UTF enabled by default on WIN EXE and DLL?

17 KB plus (DLL).
Is there a performance difference?

HPW · Post by **HPW** » Fri Nov 23, 2007 7:59 am

My others troubles seems to come from (char ...).

I used it to made a autolisp-compatibel simple ASCII-shift encoding.
And now I get different results:

newLISP v.9.2.5 on Win32.
> (char 153)
"\153"

newLISP v.9.2.6 on Win32 UTF-8.
> (char 153)
"?"

What are my options now?

A non-UTF native version of (char ..) in the UTF enabled newlisp?
A own (asciichar ..) function written in newlisp?
Stick with the NON-UTF compile on WIN-DLL?

BTW: Do I have to update the compile-toolchain to MinGW 5.1.3 with gcc v.3.4.5 (as announced in change-history)

newBert · Post by **newBert** » Fri Nov 23, 2007 8:07 am

Sorry but newlisp-edit.lsp doesn't work any more with the new release. I can only launch newlisp.exe 9.2.6 UTF-8 and newlisp-edit doesn't react !!!

When I double-clic on a NewLISP script I get only a black DOS-console ... or nothing.

HPW · Post by **HPW** » Fri Nov 23, 2007 8:15 am

When I double-clic on a NewLISP-GS start-icon I get also only the splash popup.

Cyril · Post by **Cyril** » Fri Nov 23, 2007 1:02 pm

New version is completely broken for Windows 98! (yes, I know, there is a few Windows 98 users around today, but I am one of them). It does no file I/O. I mean no I/O at all! The interpreter even cannot run scripts! And I have removed 9.2.5 release before I have noted this... No, no, don't worry, I, as a responsible citizen, have a backup, 9.2.5 is reinstalled already!

Lutz, is it possible to compile non-unicode version of newlisp executable and put in the development directory (marking clearly which is which)?

Lutz · Post by **Lutz** » Fri Nov 23, 2007 1:52 pm

All Win32 users:

I posted http://newlisp.org/downloads/developmen ... gs-107.exe again.

Thanks for all of your patience, to get this UTF-8 stuff on Win32 right, may take a while. Please make sure to report with each problem the version you are running and the country or locale of your Windows installation. I believe that France and Germany use both comma and Great Britain uses the point for decimals, but I am not sure about any other country.

For this version the newlisp.exe runtime is still UTF-8 enabled but newlisp-edit.lsp and all demo programs do a (set-locale "C") in the beginning because newlisp-edit.lsp and most demo programs use the decimal point when specifying color floats.

Worst case I can conditionally go back to non-UTF8 in both newLISP-GS (the Java part) and ship the non-UTF-8 newlisp.exe executable with it (on Win32).

I would hate to so this though, and we should at least try to get it going with all-UTF-8 :-). Java is Unicode through and through and it would be sad not to take advantage of it.

Also most newLISP users live in countries with non-ASCII character sets and would appreciate the flexibility UTF-8 brings to the table.

HPW:

I am not familiar with Autocad and do not fully understand the problem you are having, but here is a function which does more or less what you are trying to do:

Code: Select all

(define (encode str)
	(letn (len (length str) lst (unpack (dup "b" len) str))
		(format (dup {\%03d} (length lst)) lst)))

> (println (encode "ABC"))
\041\042\043
"\\041\\042\\043"
>

or this:

Code: Select all

(define (encode str)
	(letn (len (length str) lst (unpack (dup "b" len) str))
		(format (dup {\x%2X} (length lst)) lst)))

> (println (encode "ABC"))
\x41\x42\x43
"\\x41\\x42\\x43"
>

unpack works always on byte boundaries. Is there perhaps a way to switch Autocad to UTF-8? IT seems to use one of the 256byte ISO character set, where special characters still only use one byte?

Lutz

Cyril · Post by **Cyril** » Fri Nov 23, 2007 2:42 pm

Lutz wrote:Please make sure to report with each problem the version you are running and the country or locale of your Windows installation.

Alas still doesn't working with Windows 98. It just cannot open any file (including script files intended to run). Russian Windows 98, Russian locale.

I believe there are not many Windows 98 users around here (maybe I am the only one), and I don't need GS. In fact GS just doesn't run on my system (I have not reported this, because I believe it is Windows 98 specific problem, and nobody aside me uses it, and I don't need GS at all -- I am a console fan).

Maybe it is possible to put non-utf8 newlisp.exe on the site somewhere? Neither non-utf newlisp.dll nor non-utf newlisp-edit.lsp are needed by me.

HPW · Post by **HPW** » Fri Nov 23, 2007 3:25 pm

For my decode problem I solved it with this:

Code: Select all

(if utf8
 (setq pchar (pack "c"(-((unpack "b"(substr cstr dcount))0)shiftval)))
 (setq pchar (char(-(char(substr cstr dcount))shiftval)))
)

But still struggling with the other influenced commands:

newLISP v.9.2.6 on Win32, execute 'newlisp -h' for more info.
> (trim(trim "\tinterlübke\t" "\t")" ")
"interl\129bke"
>

newLISP v.9.2.6 on Win32 UTF-8, execute 'newlisp -h' for more info.
> (trim(trim "\tinterlübke\t" "\t")" ")
"interl-übke"

The Umlauts readed from ASCII-Text files and processed in UTF gets different when I write them back to disk.

I understand your wish to move to UTF and as long you provide the makefile for NON-UTF, I could switch myself to the needed flavour.
I have set up MinGW to the version mentioned in the history and can now compile myself again.

But what happens when I do not know which flavour is installed on the end-user mashine?

HPW · Post by **HPW** » Fri Nov 23, 2007 3:38 pm

Not sure if it is possible, but how about a dual-mode version which runs per default in UTF8-mode.
Then it can be set to ASCII-mode per command:

(set-encode "ASCII")
(set-encode "UTF8")
(set-encode)
>UTF8

Would add some size to the core, but everyone could use what he needs.

Lutz · Post by **Lutz** » Fri Nov 23, 2007 4:06 pm

The Umlauts readed from ASCII-Text files and processed in UTF gets different when I write them back to disk.

Yes, they get written back in UTF-8 by newLISP-GS. Your other applications probably generate files where umlaut characters are ISO/IEC-8859 one-byte-length characters.

This is, what we will try for version 9.2.7:

(1) there will be only one version of the newLISP executable on all platforms and the utf-8 mode is switchable by a function 'set-utf8' 'true/nil'

(2) Win32 versions are by default (set-utf8 nil) while Mac OS X/Unix/Linux versions are by default (set-utf8 true) on startup. Or should Windows also be UTF8 by default ???

(3) The newLISP-GS guiserver has a gs:set-utf8 to switch the mode on an of too and will set itself to 'nil' when starting up on Windows and to 'true' when starting up on any other platform.

It seems that on Windows, although it can support UTF-8, most applications are written to use the ISO/IEC-8859 one-byte-length character set, where the accented, umlaut and other special characters are encoded in one-byte values > 128. I believe even Russian language or Hebrew language Windows uses ISO 88xx one-byte-length character sets. Perhaps some Russian and Israeli users can confirm this.

Comments from users familiar with UTF-8 issues, and specially those using multi-byte character sets (Asia, Middle East, Africa) are greatly appreciated.

Lutz

newdep · Post by **newdep** » Fri Nov 23, 2007 4:25 pm

I dont understand why UTF-8 should be on by default?

From the history of computing UTF-8 is an extention to the OS
which is enabled by the user but default its off.

I would at least make unix versions default off.

cormullion · Post by **cormullion** » Fri Nov 23, 2007 4:52 pm

Please leave it on for Mac users ... :-) I don't want to go backwards...!