Compiling on Tru64Unix 4.0f

[pjot]

Hi all,

Today I tried to compile version 8.5.7 on Tru64Unix 4.0f (formerly known as Digital Unix). I have adapted the makefile using the makefile for Solaris as a template:

Code: Select all

OBJS = newlisp.o nl-symbol.o nl-math.o nl-list.o nl-liststr.o nl-string.o nl-filesys.o \
        nl-sock.o nl-import.o nl-xml.o nl-web.o nl-matrix.o nl-debug.o pcre.o

CFLAGS = -pedantic -c -O2 -DNANOSLEEP -DSOLARIS -D_POSIX_PII_SOCKET

CC = cc

default: $(OBJS)
        $(CC) $(OBJS) -lm -lrt -ldb -o newlisp

.c.o:
        $(CC) $(CFLAGS) $<

$(OBJS): primes.h protos.h makefile_solaris

Also I made some minor changes in in sourcefiles:

- Put the #define, #elsif, #endif to the first column in the sourcefiles nl-string.c and nl-sock.c
- Removed "RTLD_GLOBAL" from nl-import.c
- Commented the line #define FIONREAD I_NREAD in nl-sock.c since Tru64 knows the FIONREAD

Now there is only 1 thing left which the Compaq C compiler cannot pass:

pamela_vaspadm> make -f makefile_tru64
cc -pedantic -c -O2 -DNANOSLEEP -DSOLARIS newlisp.c
cc -pedantic -c -O2 -DNANOSLEEP -DSOLARIS nl-symbol.c
cc -pedantic -c -O2 -DNANOSLEEP -DSOLARIS nl-math.c
cc -pedantic -c -O2 -DNANOSLEEP -DSOLARIS nl-list.c
cc -pedantic -c -O2 -DNANOSLEEP -DSOLARIS nl-liststr.c
cc -pedantic -c -O2 -DNANOSLEEP -DSOLARIS nl-string.c
cc: Error: nl-string.c, line 499: In this statement, "args" is of type "pointer to signed char", and cannot be converted to "struct declared without a tag". (noconvert)
vasprintf(&ptr, format, args);
------------------------^
*** Exit 1
Stop.

So the 'vasprintf' is known, but it cannot accept the 3rd argument which indeed appears to be a "char*".

If I comment this 'vasprintf' then everything compiles, but the resulting 'newlisp' binary cores.

Any ideas?

Peter

[pjot]

Hm, probably newLisp will not run anyway because of the 64-bit architecture of Tru64Unix.

Still it is nice to know that it *almost* compiles... :-)

[Lutz]

First, thanks for doing this work Peter, every new platform for newLISP is always very welcome by somebody.

For the vasprintf just change the line 499 in nl-string.c to

vasprintf(&ptr, format, (va_list)args)

va_list and char* seem to be equivalent on all OS ( hopefully ;-) )

I will do the same in the next version and check that all is OK on the other platforms.

Lutz

[Lutz]

Yes, it might not run, but if it compiles/links? why not. I also have a makefile_linux64 for the AMD64 but compile with a special flag -m32, which makes the platform know that it's really a 32bit app.

Lutz

[pjot]

Unfortunately, that doesn't work either, now I receive the following error:

pamela_vaspadm> make -f makefile_tru64
cc -pedantic -c -O2 -DNANOSLEEP -DSOLARIS -D_POSIX_PII_SOCKET newlisp.c
cc -pedantic -c -O2 -DNANOSLEEP -DSOLARIS -D_POSIX_PII_SOCKET nl-symbol.c
cc -pedantic -c -O2 -DNANOSLEEP -DSOLARIS -D_POSIX_PII_SOCKET nl-math.c
cc -pedantic -c -O2 -DNANOSLEEP -DSOLARIS -D_POSIX_PII_SOCKET nl-list.c
cc -pedantic -c -O2 -DNANOSLEEP -DSOLARIS -D_POSIX_PII_SOCKET nl-liststr.c
cc -pedantic -c -O2 -DNANOSLEEP -DSOLARIS -D_POSIX_PII_SOCKET nl-string.c
cc: Error: nl-string.c, line 499: In this statement, "va_list" is a struct type, which is not scalar. (needscalartyp)
vasprintf(&ptr, format, (va_list)args);
------------------------^
*** Exit 1
Stop.

I also tried the 'my_vasprintf' in your nl-filesys.c, but also this did not work... :-(

[pjot]

By the way, I compiled the "types.c" file on Tru64 as well, using the Compaq C compiler, maybe you are interested in the result:

type bytes
---------------
char 1
char * 8
short int 2
int 4
long 8
long long 8
float 4
double 8
wchar_t 4
jmp_buf 672

[Lutz]

The types.c file reveals that it wouldn't work because of the size-8 of char*. And that is probably the reason that va_list is not accepted, which seems to be a struct type on Tru64 but is equivalent to a char* scalar on GCC 32bit.

Perhaps there is a switch similar to -m32 on the AMD64, which lets you compile as a 32 bit application? But it still could be necessary to rewrite my_vasprintf() used in 'format', if workiing differently on that platform.

As soon as 64bit goes mainstream with good GCC support over several platforms newLISP will move to 64bit, but this day seems to be still in the future. It basically means changing the structure of the lisp cell to accomodate 64bit pointers.

Lutz

[pjot]

There is such an option: -protect_headers default -xtaso_short

If I compile the types.c with this option, then this is the result:

type bytes
---------------
char 1
char * 4
short int 2
int 4
long 8
long long 8
float 4
double 8
wchar_t 4
jmp_buf 672

So this seems to be better. Now, if I compile newlisp, again I receive the same result:

....
cc -pedantic -protect_headers default -xtaso_short -c -O2 -DNANOSLEEP -DSOLARIS -D_POSIX_PII_SOCKET nl-math.c
cc -pedantic -protect_headers default -xtaso_short -c -O2 -DNANOSLEEP -DSOLARIS -D_POSIX_PII_SOCKET nl-list.c
cc -pedantic -protect_headers default -xtaso_short -c -O2 -DNANOSLEEP -DSOLARIS -D_POSIX_PII_SOCKET nl-liststr.c
cc -pedantic -protect_headers default -xtaso_short -c -O2 -DNANOSLEEP -DSOLARIS -D_POSIX_PII_SOCKET nl-string.c
cc: Error: nl-string.c, line 499: In this statement, "va_list" is a struct type, which is not scalar. (needscalartyp)
vasprintf(&ptr, format, (va_list)args);
------------------------^
*** Exit 1
Stop.

So compiling 32bits is possible, but still it stumbles over this vasprintf argument... ?

[Lutz]

I would have to study their header files to figure out how "va_list" works on that machine. On the GCC compiler its just the memory addres to the parameters packed one ofter the other in memory.

The types file compile looks good this way, so it should be possible to compile on that machine after the vasprintf obstacle.

Compiling with -DSOLARIS means that the compile is already taking my_vasprintf from nl-filesys.c. You could redefine to take char* on the last argument, but then it might trip over vsnprintf() later in that function and I have no idea how this is defined in your machine'sheader files.

It's tedious to figure out this stuff if you are not on the machine ;)

Lutz

[pjot]

Yes, I understand, if you're not on the machine yourself, it is hard... the Alphaservers belong to my employer, and unfortunately I cannot just take one home and open a line for you... :-(

Thanks for the compile hints again, tomorrow I'll see if I can pass the vasprintf problem.

Peter

[Lutz]

You could just out-comment 'format' in primes.h just to see if the whole thing runs after compliling/linking and if it's worth it. Anyway thanks for working on this. The whole vasprintf() and vsnprintf() thing has caused me some head-aches in the past, because it works differently on different compilers. Good luck tomorrow!

Lutz

[pjot]

Well, good news and bad news.

---> Good news: newLisp compiles and runs on Tru64 4.0f and also Tru64 5.1! <---

Bad news: the 'format' statement delivers "Unaligned access" errors and wrong output.

The reason for the unaligned access is caused by the way I created a workaround for the 'vasprintf' trouble. In the file nl-filesys.c I created an extra function called "tru64_vasprintf" which is a copy of the existing 'my_vasprintf':

Code: Select all

int tru64_vasprintf(char * * buffer, const char * format, char* argptr)
{
UINT size = MAX_STRING;
int pSize;

while(TRUE)
        {
        *buffer = allocMemory(size + 2);
        pSize = vsnprintf(*buffer, size + 1, format, argptr);

#ifndef MINGW
        if(pSize > size)
                {
                freeMemory(*buffer);
                size = pSize;
                continue;
                }
#else
        if(pSize < 0)
                {
                freeMemory(*buffer);
                size = size + size / 2;
                continue;
                }
#endif
        break;
        }

return(pSize);
}

The file 'nl-string.c' was changed to point to this function (line 499). Now everything compiles without errors. (Also 'protos.h' was adjusted, adding a prototype.)

However, the 'vsnprinf' takes also a 'va_list' as 3rd argument (why the compiler does not complain now, I don't know). If instead a char* is feeded here, the data alignment error appears.

It might be of your interest to know that the va_list in Tru64 is defined as:

Code: Select all

typedef struct {
        char    **_a0;          /* pointer to first homed integer arg */
        int     _offset;                /* byte offset of next param */
} va_list;

Except for the format statement, all other newlisp functions seem to work fine. I will perform some tests today and let you know the results.

Peter

[pjot]

Adjusted the 'tru64_myvasprinf' function to wrap around the va_list:

Code: Select all

int tru64_vasprintf(char * * buffer, const char * format, ...)
{
UINT size = MAX_STRING;
int pSize;
va_list ap;

while(TRUE)
        {
        *buffer = allocMemory(size + 2);
        va_start(ap, format);
        pSize = vsnprintf(*buffer, size + 1, format, ap);
        va_end(ap);

#ifndef MINGW
        if(pSize > size)
                {
                freeMemory(*buffer);
                size = pSize;
                continue;
                }
#else
        if(pSize < 0)
                {
                freeMemory(*buffer);
                size = size + size / 2;
                continue;
                }
#endif
        break;
        }
return(pSize);
}

Now the alignment errors are gone, but the output is wrong. At every string I run through format, the result is 0 (or "0").

[Lutz]

Its the difference between GNU CC and Tru64 CC how arguments are layed out in memory for vsnprintf().

In Gnu CC all args are packes one after the other in a memory area, in Tru64 after each arg piece comes an int describing the offset. I order to make it work p_format() in nl-string.c would have to be rewritten to pack arguments for the format string in a way necessary for Tru64.

The change you did using va_list and va_start was the right thing to do, but would also require to change the function doing the preperation of it's last args ('ap' in your code) parameter

I wonder if you can get GCC for the Tru64 platform? Perhaps that would be easier then trying to work around the peculiarities of the CC found on Tru64?

This is one of those typical cases where porting gets a head-ache. Another problem is how vsnprintf() returns different values in case of failure, as you can see on the MINGW switch, and this is also something that would have to be looked into.

Anyway, good luck in your porting efforts ... perhaps somebody else here on the port as some 'quick' fix hoew to deal with this problem?

Lutz

[pjot]

Indeed I already have been busy to change the code in nl-string.c... the memcpy's create a mem layout which indeed is not compatible with Tru64Unix va_list.

I am able to print the first argument though, like (format "c" 65) -> "A". But more complicated formats deliver problems.

The latest GCC for Tru64Unix I am aware of, is version 2.95, which lacks the '-m32' switch.

So I need to hack the memory layout of the 'va_list' anyway, in order to get the format properly working.

BTW: other things, like lists, signals, read-key and sockets, work. It is *only* this format statement which gives a headache. This weekend I will work on the port.

Peter

[pjot]

The Tru64Unix port is ready.

Peter

[newdep]

newLISP v.8.5.8 on Tru64Unix, execute 'newlisp -h' for more info.

> (setq a 1)
1
> (inc 'a -15)
4294967282
> (setq b -16)
-16
> (inc 'b)
-15
> (inc 'b -17)
8589934560
> (inc 'b -255)
1.28849016e+10
>

[pjot]

Strange, there is an issue with negative numbers:

newLISP v.8.5.8 on Tru64Unix, execute 'newlisp -h' for more info.

> (set 'a 10)
10
> (inc 'a)
11
> (inc 'a 3)
14
> (inc 'a -1)
4294967309
>

-----------------------

But normal expressions work:

> (- -1 -5)
4
> (+ 1 -10)
-9
>

-----------------------

I'll look into 'nl-match.c' to see what is going on. Obviously, negative numbers are rounded wrong, though the format works:

> (format "%x %X" -1 -1)
"ffffffff FFFFFFFF"

As a workaround you can use a float:

> (set 'b 5)
5
> (inc 'b 0.1)
5.1
> (inc 'b -0.1)
5
> (inc 'b -6.5)
-1.5

Peter

[Lutz]

I wonder if true64 formats numbers as IEE 754 double, 8 bytes long. The function doing the 'inc' and 'dec' is called incDec() and is in nl-math.c.

But I would check first if things like:

(sub 10 -1) or (add 10 1)

work. It may be that the function getFloat() in newlisp.c is not working right on true64-unix. Did you run qa_dot, testing all integer and float arithemetik?

Lutz

[pjot]

Hi Lutz,

Well we are in the process of testing, and indeed I was wondering if there was such a thing as a testsuite.

Here the results of the 'div' and 'add':

newLISP v.8.5.8 on Tru64Unix, execute 'newlisp -h' for more info.

> (sub 10 -1)
-4294967285
> (add 10 1)
11
>

So it seems to be something with negative numbers. A '-1' should be 0xFFFFFFFF right? Probably on Tru64 this is bigger.

If I run the 'qa_dot' program, there is a crash after the 'append':

pamela_vaspadm> newlisp qa_dot

Testing built-in functions ...

! != $ % &
* + - / <
<< <= = > >=
>> NaN? ^ abs acos
add address amb and append
Segmentation fault (core dumped)

To avoid this, I have commented out the following functions:
-(test-array-list) which causes this crash.
-(test-dotimes , aList) which blocked newLisp
-(test-pmt) which gave a FLOAT error


If I run again, this is the result:

pamela_vaspadm> newlisp qa_dot

Testing built-in functions ...

! != $ % &
* + - / <
<< <= = > >=
>> NaN? ^ abs acos
add address amb and append
apply args array array-list array?
asin assoc atan atan2 atom?
base64-dec base64-enc begin beta betai
binomial case catch ceil change-dir
char chop close command-line cond
cons constant context context? copy-file
cos count cpymem crit-chi2 crit-z
current-line date date-value debug dec
def-new define define-macro delete delete-file
device difference directory directory? div
do-until do-while dolist dotimes dotree
dump dup empty? encrypt ends-with
env error-event error-number error-text eval
eval-string exec exit exp expand
explode factor fft file-info file?
filter find first flat float
float? floor flt for fork
format fv gammai gammaln get-char
get-float get-int get-integer get-string get-url
global if ifft import inc
index int integer integer? intersect
invert irr join lambda? last
legal? length let letn list
list? load log lookup lower-case
macro? main-args make-dir map match
max member min mod mul
multiply name net-accept net-close net-connect
net-error net-listen net-local net-lookup net-peek
net-peer net-receive net-receive-from net-receive-udp net-select
net-send net-send-to net-send-udp net-service net-sessions
new normal not now nper
npv nth nth-set open or
pack parse pipe pack parse pipe pmt pop
post-url pow pretty-print primitive? print
println prob-chi2 prob-z process println prob-chi2 prob-z process push
put-url pv quote quote? rand
random read-buffer read-char read-file read-key
read-line ref regex remove-dir rename-file
replace replace-assoc reset rest reverse
rotate save search seed seek
select semaphore sequence series set
set! set-locale set-nth setq share
signal silent sin sleep --- sleep duration: 1000 ---
slice
sort source sqrt starts-with string
string? sub swap sym symbol
symbol? symbols sys-error sys-info tan
throw throw-error time time-of-day timer
trace trace-highlight transpose trim unique
unless unpack until upper-case wait-pid
while write-buffer write-char write-file write-line
xml-error xml-parse xml-type-tags | ~


Testing contexts as objects and scoping rules ...


TESTING FINISHED WITH ERRORS:

>>>> / failed nil
>>>> < failed nil
>>>> <= failed nil
>>>> > failed nil
>>>> NaN? failed nil
>>>> abs failed nil
>>>> array-list failed nil
>>>> array? failed nil
>>>> binomial failed nil
>>>> dotimes failed nil
>>>> exec failed nil
>>>> fv failed nil
>>>> int failed nil
>>>> integer failed nil
>>>> net-receive-udp failed nil
>>>> net-send-udp failed nil
>>>> npv failed nil
>>>> pmt failed nil
>>>> semaphore failed nil
>>>> share failed nil

newLISP v.8.5.8 on Tru64Unix, execute 'newlisp -h' for more info.

> ring...

Most of these failed tests use negative numbers or floats.

[pjot]

In this situation:

Code: Select all

newLISP v.8.5.8 on Tru64Unix, execute 'newlisp -h' for more info.

> (set 'a 5)
5
> (inc 'a 1)
6
> (inc 'a -1)
4294967301

The function 'CELL * getFloat(CELL * params, double * floatNumber)' determines the '-1' wrongly.

This is caused by the fact that "cell = evaluateExpression(params);" determines the cell->type as a CELL_FLOAT for the number '-1'.

So the cause seems to be in the routine 'evaluateExpression'. Or is the cell->type previously determined by a parser? Continuing...

[Lutz]

Looks like for doing this true64 port we need to look in the Compaq 'C' compiler documentation to find out about floating point formats etc. Can you compile types.c again and show the output? Perhaps we missed something there.

Lutz

[pjot]

I found a compiler switch '-ieee' to ensure the IEEE754 compatibility, but the resulting newLisp binary delivers the same results with 'qa_dot'.

Compiled the 'types.c' again:

pamela_vaspadm> cc -o types -ieee -taso types.c
pamela_vaspadm> ./types
type bytes
---------------
char 1
char * 8
short int 2
int 4
long 8
long long 8
float 4
double 8
wchar_t 4
jmp_buf 672

I have the strong feeling it is an issue with the negative sign '-' on a number...

Hacked the routine 'CELL * incDec(CELL * params, int type)' to print info about variables:

newLISP v.8.5.8 on Tru64Unix, execute 'newlisp -h' for more info.

> (set 'a 5)
5
> (inc 'a 1)
adjust: 1.000000
type: 1
6
> (inc 'a -1.0)
adjust: -1.000000
type: 1
5
> (inc 'a -1)
adjust: 4294967295.000000
type: 1
4294967300

[pjot]

Are you using INT's or LONG's for normal numbers? Since on the alpha the LONG is twice as big as an INT.

While on X86 both the INT and also the LONG contain 4 bytes.

So a '-1' as a long will be 0xFFFFFFFFFFFFFFFF instead of 0xFFFFFFFF.

If Alpha sees 0xFFFFFFFF it will treat this as a regular number which is 4294967295.

Jep that's it. Any idea what I need to change in the source?

[Lutz]

There are no 'long's in newlisp.c but in several other files, all these could be replaced with 'int', because int and long are the same for 32bit GCC. Its mostly in older code from Win16 times, I haven't touched for a longer time. You would have to go through all the files for this. I could do this, but not before the weekend.

But there is a bigger problem with pcre.c there are tons of 'long int' in it and I have no idea if that code was written with 'long int' == 8byte in mind. But you might just try and see ;)

Lutz