newlisp on AMD64

[newdep]

Hi Lutz,

Well i had a system crash and needed to buy a new system, unfortunatly it
became an amd64 ;-)

But now ofcourse newlisp needs to be compiled... ;-)
So im running a slackware amd64 clone (www.slamd64.com) which is always
better then using Win-XP ;-)



here is what i get ->

make linux (this works!!! but with lots of -> warning: cast from pointer to integer of different size)



make linux_lib (does not work) ->

gcc -Wall -pedantic -Wno-uninitialized -c -O2 -DLINUX -DLIBRARY unix-lib.c
unix-lib.c: In function `initializeMain':
unix-lib.c:41: warning: cast from pointer to integer of different size
gcc newlisp.o nl-symbol.o nl-math.o nl-list.o nl-liststr.o nl-string.o nl-filesys.o nl-sock.o nl-import.o nl-xml.o nl-web.o nl-matrix.o nl-debug.o pcre.o unix-lib.o -lm -ldl -shared -o newlisp.so
/usr/lib/gcc/x86_64-slackware-linux/3.4.4/../../../../x86_64-slackware-linux/bin/ld: newlisp.o: relocation R_X86_64_32 against `a local symbol' can not be used when making a shared object; recompile with -fPIC
newlisp.o: could not read symbols: Bad value
collect2: ld returned 1 exit status
make[1]: *** [default] Error 1


make linux64 (does not work) ->

make -f makefile_linux64
make[1]: Entering directory `/newlisp-8.7.4'
gcc -m32 -Wall -pedantic -Wno-uninitialized -c -O2 -g -DLINUX newlisp.c
gcc -m32 -Wall -pedantic -Wno-uninitialized -c -O2 -g -DLINUX nl-symbol.c
gcc -m32 -Wall -pedantic -Wno-uninitialized -c -O2 -g -DLINUX nl-math.c
gcc -m32 -Wall -pedantic -Wno-uninitialized -c -O2 -g -DLINUX nl-list.c
gcc -m32 -Wall -pedantic -Wno-uninitialized -c -O2 -g -DLINUX nl-liststr.c
gcc -m32 -Wall -pedantic -Wno-uninitialized -c -O2 -g -DLINUX nl-string.c
gcc -m32 -Wall -pedantic -Wno-uninitialized -c -O2 -g -DLINUX nl-filesys.c
gcc -m32 -Wall -pedantic -Wno-uninitialized -c -O2 -g -DLINUX nl-sock.c
gcc -m32 -Wall -pedantic -Wno-uninitialized -c -O2 -g -DLINUX nl-import.c
gcc -m32 -Wall -pedantic -Wno-uninitialized -c -O2 -g -DLINUX nl-xml.c
gcc -m32 -Wall -pedantic -Wno-uninitialized -c -O2 -g -DLINUX nl-web.c
gcc -m32 -Wall -pedantic -Wno-uninitialized -c -O2 -g -DLINUX nl-matrix.c
gcc -m32 -Wall -pedantic -Wno-uninitialized -c -O2 -g -DLINUX nl-debug.c
gcc -m32 -Wall -pedantic -Wno-uninitialized -c -O2 -g -DLINUX pcre.c
gcc newlisp.o nl-symbol.o nl-math.o nl-list.o nl-liststr.o nl-string.o nl-filesys.o nl-sock.o nl-import.o nl-xml.o nl-web.o nl-matrix.o nl-debug.o pcre.o -m32 -g -lm -ldl -o newlisp
nl-import.o(.text+0x3b): In function `p_importLib':
/newlisp-8.7.4/nl-import.c:109: warning: Using 'dlopen' in statically linked applications requires at runtime the shared libraries from the glibc version used for linking
/usr/lib/gcc/x86_64-slackware-linux/3.4.4/../../../../lib/libdl.a(dlopen.o)(.text+0x1b): In function `dlopen':
: undefined reference to `__dlopen'
/usr/lib/gcc/x86_64-slackware-linux/3.4.4/../../../../lib/libdl.a(dlsym.o)(.text+0x1b): In function `dlsym':
: undefined reference to `__dlsym'
/usr/lib/gcc/x86_64-slackware-linux/3.4.4/../../../../lib/libdl.a(dlerror.o)(.text+0x5): In function `dlerror':
: undefined reference to `__dlerror'
collect2: ld returned 1 exit status
make[1]: *** [default] Error 1


Any idea on the linux64 ??? (i think a library setting somewhere )

Regards, Norman.

[Lutz]

This is aht Linux64 they run on Sourceforge where it compiles/links fine:

Linux amd64-linux1.sourceforge.net 2.6.9-1.667smp

Yes, I think you need that libdl installed. On the Sourceforge installation it is:

/usr/lib/libdl.a and /usr/lib/libdl.so

Lutz

[newdep]

I feel like im 10 year back in time with this system running linux amd64..
All those library build are mixed up, different kernel build missing libs... (..Ieeekksss..)

Ill stick with XP for now because of the performance ;-) (yeah sound odd
and Ill install the 32bit distribution of slackware again.. or perhpas
debian..

So the linux amd64 ill test perhpas next year..when all those linux
versions outthere are stable for the amd64...(currently it seems that
Redhat is the only one claiming to be working correctly (which i dont like),
debian is far from finished and slackware is a mess, OpenBSD does not
support all the hardware yet.. So that brings me back to 1995....)

PS: the libdl.a was from kernel 2.4.6 and my libdl.so was a x86_64 one,
but no matching libdl.so around for 32bit... so that left me only with recompiling the Gcc stuff.. and I not realy willing to do that ;-)
So for the current release of slamd64 amd64 10.2 (out of the box) newlisp
is unable to compile on 64 bits. (thats a slamd64 problem!)


regards, Norman.

[newdep]

well after some thoughts ;-) Im not a windows guy so Ill be beating
this package and make that newlisp 64b running ;-) Ill be back on this topic ;-)

[newdep]

Hi Lutz,

I dont realy understand the makefile_linux64 compared to the makefile_linux. The only differnece is the -m32 in the linux64 which forces
to compile on 32bit.. So actualy the linux64 is a 32B version. In that case Why would i use makefile_linux64 ?

For a real 64bit version is need -m64 ... Its not realy clear to me yet...

Regards, Norman.

[Lutz]

The amd64 is a 64 bit processor with integers and pointers of 64bits, but currently newLISP is a 32 bit language and therefore has to be compiled using a basic integer/pointer size of 32 bits.

So newLISP compiled on an amd64 is still a 32 bit language. Depending on the quality of the compiler though it may still be an advantage running on a 64Bit CPU/OS because memory and string moves and copies can take advantage of the bigger native word size of the CPU if the compiler is well written.

Lutz

[pjot]

Hi Norman,

As you may see from the Tru64Unix makefile, I suffered from the same problems with the 64-bit Alpha:

CFLAGS = -ieee -xtaso -pedantic -c -O2 -DSOLARIS -DTRU64 -D_POSIX_PII_SOCKET

The '-xtaso' flag forces 32 bit pointers. So on Tru64 newLisp will run in 32 bit mode also. (At the time when I was porting newLisp, I had everything running in 64bit mode, except for arrays. That was the only thing; all the other tests in 'qa_dot' survived in 64 bit mode.)

Peter

[newdep]

yeah thats what I thougth...

Well good news ..make linux64 Runs !! (thanks pjot!) was missing a symlink.


BUT now the make llinux_lib does not work... complains about ->

/usr/lib/gcc/x86_64-slackware-linux/3.4.4/../../../../x86_64-slackware-linux/bin/ld: unix-lib.o: relocation R_X86_64_32 against `errorJump' can not be used when making a shared object; recompile with -fPIC
unix-lib.o: could not read symbols: Bad value
collect2: ld returned 1 exit status
make[1]: *** [default] Error 1



So I edited the makefile_linux64 and added -fPIC... that compiled without errors...

Tehn i edited the makefile_linux_lib and added the -fPIC..that compiled also BUT with lots of warnings -> warning: cast to pointer from integer of different size

Sooo I have now a newlisp binary and a newlisp.so But with 0-fPIC ?? what is the impact of that on newlisp ?


seems the lib is 64 bits ->
@zeep:~/prog/nl/newlisp-8.7.4$ ldd newlisp.so
libm.so.6 => /lib64/tls/libm.so.6 (0x00002aaaaac09000)
libdl.so.2 => /lib64/tls/libdl.so.2 (0x00002aaaaad91000)
libc.so.6 => /lib64/tls/libc.so.6 (0x00002aaaaae94000)
/lib64/ld-linux-x86-64.so.2 (0x0000555555554000)


seems newlisp bin is 32 bits->
@zeep:~/prog/nl/newlisp-8.7.4$ ldd ./newlisp
linux-gate.so.1 => (0xffffe000)
libdl.so.2 => /lib/tls/libdl.so.2 (0x55588000)
libc.so.6 => /lib/tls/libc.so.6 (0x5558c000)
/lib/ld-linux.so.2 (0x55555000)


regards, Norman.

[newdep]

Hi Lutz,

See above how i compiled newlisp on the amd64...
make linux64 with -fPIC and the make linux_lib also...

BUT !!!! it seems newlisp is unable to import 64bits compiled Library's....
(See above...for dependencies)

So a simple import results in this ->

(import "/newlisp-8.7.4/newlisp.so" "newlispEvalStr")
/newlisp-8.7.4/newlisp.so: cannot open shared object file: No such file or directory
problem loading library in function import : "/newlisp-8.7.4/newlisp.so"


Trying to import other libs made with -fPIC -m32 also did not work...

So my question is... How do I build a lib on amd64 ?

a Make linux gives pointer warnings..and I cant make any 32b lib out of those .o files...

a Make linux64 does only work with -m32 and -fPIC (whatever that might be)
and does create a good newlisp.so.. but unsuitable for import..

any idea?

Regards, Norman.

[Lutz]

Current newLISP is a 32 bit application and cannot be compiled as a 64 bit app or library, neither can it be used to import 64 bit libraries.

Lutz

ps: I will try the 32bit compile to a lib on Amd64 later.

[Lutz]

On Sourceforge the following makefile made a full functioning library running the qa suite:

Code: Select all

# makefile for newLISP v. 8.x.x on 64 bit LINUX tested on AMD64 as a shared library
#
# Note, that readline support may require different libraries on different OSs
#

OBJS = newlisp.o nl-symbol.o nl-math.o nl-list.o nl-liststr.o nl-string.o nl-filesys.o \
        nl-sock.o nl-import.o nl-xml.o nl-web.o nl-matrix.o nl-debug.o pcre.o unix-lib.o

#CFLAGS = -m32 -Wall -pedantic -Wno-uninitialized -c -O2 -g -DREADLINE -DLINUX
CFLAGS = -m32 -Wall -pedantic -Wno-uninitialized -c -O2 -g -DLINUX -DLIBRARY

CC = gcc

default: $(OBJS)
#       $(CC) $(OBJS) -m32 -g -lm -ldl -lreadline -ltermcap -o newlisp
#       $(CC) $(OBJS) -m32 -g -lm -ldl -lreadline -lncurses -o newlisp
        $(CC) $(OBJS) -m32 -g -lm -ldl -shared -o newlisp.so
        strip newlisp.so

.c.o:
        $(CC) $(CFLAGS) $<

$(OBJS): primes.h protos.h makefile_linux64_lib

Code: Select all

> (import "./newlisp.so" "newlispEvalStr")
newlispEvalStr <F7FC6490>
> (get-string (newlispEvalStr {(load "qa_dot")}))            
[text]
Testing built-in functions ...

!              !=             $              %              &              
*              +              -              /              <              
<<             <=             =              >              >=             
>>             NaN?           ^              abs            acos           
add            address        amb            and            append         
.....
.....
ALL FUNCTIONS FINISHED SUCCESSFULL

nil
[/text]
> (exit)
~/newlisp-8.7.4> uname -a
Linux amd64-linux1.sourceforge.net 2.6.9-1.667smp #1 SMP Tue Nov 2 15:09:11 EST 2004 x86_64 x86_64 x86_64 GNU/Linux
~/newlisp-8.7.4> 

Here some info about the compiler used:

Code: Select all

~/newlisp-8.7.4> cc -v
Reading specs from /usr/lib/gcc/x86_64-redhat-linux/3.4.2/specs
Configured with: ../configure --prefix=/usr --mandir=/usr/share/man --infodir=/usr/share/info --enable-shared --enable-threads=posix --disable-checking --with-system-zlib --enable-__cxa_atexit --disable-libunwind-exceptions --enable-languages=c,c++,objc,java,f77 --enable-java-awt=gtk --host=x86_64-redhat-linux
Thread model: posix
gcc version 3.4.2 20041017 (Red Hat 3.4.2-6.fc3)
~/newlisp-8.7.4> 

Lutz

ps: it seems in your last (import ....) it did not find the library/path

[newdep]

Hi Lutz,

The linux64 binary and library compile fine!

They both compile without errors.
Im not using the -fPIC and only with the -m32.
Library import works...!
im using slamd64 10.2 slackware/amd64!


im using gcc and no problems..->
$ cc -v
Reading specs from /usr/lib/gcc/x86_64-slackware-linux/3.4.4/specs
Configured with: ../gcc-3.4.4/configure --prefix=/usr --enable-shared --enable-threads=posix --enable-__cxa_atexit --disable-checking --with-gnu-ld --verbose --target=x86_64-slackware-linux --host=x86_64-slackware-linux
Thread model: posix
gcc version 3.4.4



/newlisp-8.7.4$ ldd ./newlisp
linux-gate.so.1 => (0xffffe000)
libdl.so.2 => /lib/tls/libdl.so.2 (0x55588000)
libc.so.6 => /lib/tls/libc.so.6 (0x5558c000)
/lib/ld-linux.so.2 (0x55555000)
/newlisp-8.7.4$ ldd ./newlisp.so
linux-gate.so.1 => (0xffffe000)
libdl.so.2 => /lib/tls/libdl.so.2 (0x555b4000)
libc.so.6 => /lib/tls/libc.so.6 (0x555b8000)
/lib/ld-linux.so.2 (0x56555000)



Thanks, Norman.

[newdep]

Though now I have newlisp running its not without problems ->

during my rss-ticker.lsp program -->

*** glibc detected *** double free or corruption (!prev): 0x080a5290 ***
Aborted

--- and it core dumped ---

Mmmmmm..odd?? kernel problem? gcc problem?? or newlisp?

Norman.

[Lutz]

Until 64 bit gets main stream I cannot support these kind of issues, you have to debug these kined of problems yourself ;)

Pehaps Peter with more experience on ont TRUE64 compile has some insights.

Lutz

[nigelbrown]

Until 64 bit gets main stream ...
Lutz

I've recently bought 2 middle range computers and a barebox and even the barebox one was 64bit - they're very quickly going to become all you can get.

Nigel

[newdep]

Seems the newlisp.so compiled on amd64 is hehaving a little strange ->
(compiled using the makefile_linux64_lib, no errors!)

newLISP v.8.7.5 on linux, execute 'newlisp -h' for more info.

> (import "./newlisp.so" "newlispEvalStr")
newlispEvalStr <556E2240>
> (newlispEvalStr "(+ 2 2)")
134923456
> (get-string (newlispEvalStr "(+ 2 2)"))
""
>


> (unpack "s2" (newlispEvalStr "(+ 2 2)"))
("\000\000")
>


I just tested this under a fresh 32b slackware and does also not run, so probably I doing something wrong ;-)

[Lutz]

Again, this seems to be specific to your compile on the slcakware amd64 platform. I cannot repeat this on the amd64 sourceforge machine.

Lutz

[newdep]

Hi Lutz,

I dropped the slamd64 for now... this error is on a slackware 10.2 32Bits machine.!!

newLISP v.8.7.5 on linux, execute 'newlisp -h' for more info.

> (import "./newlisp.so" "newlispEvalStr")
newlispEvalStr <40241D10>
> (get-string (newlispEvalStr "(+ 2 2)"))
""
>

/newlisp-8.7.5$ldd ./newlisp
libm.so.6 => /lib/libm.so.6 (0x4002d000)
libdl.so.2 => /lib/libdl.so.2 (0x40051000)
libreadline.so.4 => /usr/lib/libreadline.so.4 (0x40055000)
libncurses.so.5 => /lib/libncurses.so.5 (0x40081000)
libc.so.6 => /lib/libc.so.6 (0x400c0000)
/lib/ld-linux.so.2 (0x40000000)
libgpm.so.1 => /lib/libgpm.so.1 (0x401d9000)

/newlisp-8.7.5$ldd ./newlisp.so
libm.so.6 => /lib/libm.so.6 (0x40052000)
libdl.so.2 => /lib/libdl.so.2 (0x40076000)
libc.so.6 => /lib/libc.so.6 (0x4007a000)
/lib/ld-linux.so.2 (0x80000000)

/newlisp-8.7.5$cat makefile_linux
# makefile for newLISP v. 8.x.x on LINUX
#
# Note, that readline support may require different libraries on different OSs
#

OBJS = newlisp.o nl-symbol.o nl-math.o nl-list.o nl-liststr.o nl-string.o nl-filesys.o \
nl-sock.o nl-import.o nl-xml.o nl-web.o nl-matrix.o nl-debug.o pcre.o

CFLAGS = -Wall -pedantic -Wno-uninitialized -c -O2 -g -DREADLINE -DLINUX
#CFLAGS = -Wall -pedantic -Wno-uninitialized -c -O2 -g -DLINUX

CC = gcc


default: $(OBJS)
# $(CC) $(OBJS) -g -lm -ldl -lreadline -ltermcap -o newlisp
$(CC) $(OBJS) -g -lm -ldl -lreadline -lncurses -o newlisp
# $(CC) $(OBJS) -g -lm -ldl -o newlisp
strip newlisp

.c.o:
$(CC) $(CFLAGS) $<

$(OBJS): primes.h protos.h makefile_linux



/newlisp-8.7.5$cat makefile_linux_lib
# makefile for newLISP v. 8.x.x on LINUX as a shared library - newlisp.so -
#
# Note, that readline support may require different libraries on different OSs
#

OBJS = newlisp.o nl-symbol.o nl-math.o nl-list.o nl-liststr.o nl-string.o nl-filesys.o \
nl-sock.o nl-import.o nl-xml.o nl-web.o nl-matrix.o nl-debug.o pcre.o unix-lib.o

CFLAGS = -Wall -pedantic -Wno-uninitialized -c -O2 -DLINUX -DLIBRARY

CC = gcc

default: $(OBJS)
$(CC) $(OBJS) -lm -ldl -shared -o newlisp.so
strip newlisp.so

.c.o:
$(CC) $(CFLAGS) $<

$(OBJS): primes.h protos.h makefile_linux_lib



--- no errors during compilation...


Hope you have a hint ;-)


regards, Norman

[Lutz]

I just cannot repeat this, everything is fine on Debian and Fedora:

Code: Select all

Linux 2.6 on Debian 3.1

newLISP v.8.7.5 on linux, execute 'newlisp -h' for more info.

> (import "./newlisp.so" "newlispEvalStr")
newlispEvalStr <401B39F0>
> (newlispEvalStr "(+ 3 4)")
134902984
> (get-string (newlispEvalStr "(+ 3 4)"))
"7\n"
> 

Linux 2.6 on Fedora FC2

newLISP v.8.7.5 on linux, execute 'newlisp -h' for more info.

> (import "./newlisp.so" "newlispEvalStr")
newlispEvalStr <710928>
> (get-string (newlispEvalStr "(+ 3 4)"))
"7\n"
> (get-string (newlispEvalStr "(+ 2 2)"))
"4\n"
> 

I wonder if this is specific to Slackware, which is hard to imagine.

Lutz

[Lutz]

... and FreeBSD 4.9 is fine too:

Code: Select all

newLISP v.8.7.5 on BSD, execute 'newlisp -h' for more info.

> (import "./newlisp.so" "newlispEvalStr")
newlispEvalStr <281D6C28>
> (get-string (newlispEvalStr "(+ 3 4)"))
"7\n"
> 

Lutz

[newdep]

I cant explain the problem, Pjot looked at my machine and had the same
problem in MY newlisp directory but not in His directory..

Problem is solved it works..BUT problem source is unknown...

Lets have a beer! ;-)

regards, Norman.

[newdep]

'make clean' does wonders inbetween linux and linux_lib...

That explains why i had troubles with my last codes and import.. I was
always running an old newlisp.so... Snif snif...

[Lutz]

... have one for me too ;)

Lutz

ps: sipping on my 'Cuba Libre'

[newdep]

Your a Lucky Man !!!

Last year I was in Cuba for a month and I could not resist the Cuba-Libres ;-)

Enjoy!

[pjot]

This was a nice problem! Enjoyed hacking it.... now I'm on a "Forbidden Fruit' beer, which I still owe you... ;-)