pop-assoc problem: feels like a bug [fixed in v.10.5.4]

[hartrock]

Hello All,
hello Lutz,

The following shows something, which looks like a bug.

Code:

Code: Select all

(new Tree 'Data)
(push '(1 (k_1 "v_1")) Data)
(push '(k_2 "v_2") (assoc 1 Data) -1)
;; -> OK
;;
(pop-assoc '(1 k_2) Data)
Data:Data
;; -> OK
;;
;;
(push '(k_2 "v_2") (assoc 1 Data) -1)
;; -> fails!
;;
;; repair
(push (pop (assoc 1 Data) -1) (assoc 1 Data) -1)
;;
(push '(k_2 "v_2") (assoc 1 Data) -1)
;; -> works again.

Session:

Code: Select all

newLISP v.10.5.3 64-bit on Linux IPv4/6 UTF-8, options: newlisp -h

> (new Tree 'Data)
Data
> (push '(1 (k_1 "v_1")) Data)
((1 (k_1 "v_1")))
> (push '(k_2 "v_2") (assoc 1 Data) -1)
(1 (k_1 "v_1") (k_2 "v_2"))
> ;; -> OK
> ;;
> (pop-assoc '(1 k_2) Data)
(k_2 "v_2")
> Data:Data
((1 (k_1 "v_1")))
> ;; -> OK
> ;;
> ;;
> (push '(k_2 "v_2") (assoc 1 Data) -1)
(1 (k_1 "v_1"))
> ;; -> fails!
> ;;
> ;; repair
> (push (pop (assoc 1 Data) -1) (assoc 1 Data) -1)
(1 (k_1 "v_1"))
> ;;
> (push '(k_2 "v_2") (assoc 1 Data) -1)
(1 (k_1 "v_1") (k_2 "v_2"))
> ;; -> works again.

It would be nice, if someone could check this at his/her own host, for checking platform differences (I had to compile with a configure-alt step in advance, to get all tests running).

Best regards,
Stephan

[Lutz]

Also happens on OSX and Windows. Will investigate.

[Lutz]

The problem is fixed here: http://www.newlisp.org/downloads/develo ... nprogress/

There will be a development release with installers. If you need a binary now, I could help you for Mac OSX and Windows.

[hartrock]

Hello Lutz,

1. Many thanks for your fast fix!

2. It works for me under Debian wheezy (stable).

3. For getting all tests running:

Code: Select all

./configure-alt 
make
make check
make testall

Tried to upload makefile-build named
- makefile_build makefile_build.text makefile_build.txt
, but always gotten:
- The extension ... is not allowed.
If you want to have this or another file/info, please give me a note.

4. The direct way of doing make without the configure-alt step leads to failure of

Code: Select all

make testall

with (console output):

Code: Select all

make checkall | grep '>>>'
>>>>> ALL FUNCTIONS FINISHED SUCCESSFUL: ./newlisp
>>>>> Dictionary API tested SUCCESSFUL
>>>>> XML API tested SUCCESSFUL
>>>>> XML callback tested SUCCESSFUL
>>>>> JSON translation tested SUCESSFUL
>>>>> Signal testing SUCCESSFUL
>>>>> Network eval and network file functions IPv4 SUCCESSFUL
>>>>> Network eval and network file functions IPv6 SUCCESSFUL
>>>>> The Cilk API tested SUCCESSFUL
>>>>> Reference testing SUCCESSFUL
>>>>> Time per simple message: 18 micro seconds
>>>>> Time per round trip : 50 micros seconds
>>>>> Time per proxy trip: 60 micro seconds
>>>>> Message API tested SUCCESSFUL
>>>>> Memory allocation/deallocation SUCCESSFUL
>>>>> Exception testing SUCCESSFUL
>>>>> Floating point tests SUCCESSFUL
>>>>> FOOP nested 'self' tested SUCCCESSFUL
>>>>> FOOP symbol protection SUCCESSFUL
>>>>> UNIX local domain sockets SUCCESSFUL
>>>>> In-place modification passed SUCCESSFUL
>>>>> UTF-16 path-file-names testing SUCCESSFUL
>>>>> 0.007 ms per write->read pipe/fork (0.0356 ms Mac OSX, 1.83 GHz Core 2 Duo)
make[1]: *** [checkall] Error 255

My system is
- Linux free 3.2.0-4-amd64 #1 SMP Debian 3.2.46-1 x86_64 GNU/Linux
with env var
- LANG=en_US.UTF-8
.

Best regards,
Stephan

[Lutz]

The build made with the normal configure fails because it's making for libffi on LINUX. The test qa-specific-tests/qa-ffi then fails. You could install libffi and/or adjust the makefile_linux* for the correct include file directory.

ps: you could just paste the makefile as code

[hartrock]

1. libffi has been installed.
2. Using configure-all just omits ffi tests, so there has been no failure.

3.
After looking into the problem two patches in the test code have solved the problem (compiled the standard way):

Code: Select all

sr@free:~/tmp/newlisp-10.5.4$ diff qa-specific-tests/qa-ffi qa-specific-tests/qa-ffi_orig 
18c18
<     ("Linux" "/lib/x86_64-linux-gnu/libc.so.6")
---
>     ("Linux" "/lib/i386-linux-gnu/libc.so.6")
sr@free:~/tmp/newlisp-10.5.4$ diff qa-specific-tests/qa-libffi qa-specific-tests/qa-libffi_orig 
20c20
<         (! "gcc -m64 -fPIC util/ffitest.c -shared -o ffitest.dylib"))
---
>         (! "gcc -m64 util/ffitest.c -shared -o ffitest.dylib"))

If you want to have checked other - more generic - variants (don't know what these changes do at other platforms), I could do this.

[hartrock]

PS: Simplified makefile working for me without /usr/local libffi, because it's at some standard loc:

Code: Select all

sr@free:~/tmp/newlisp-10.5.4$ diff makefile_linuxLP64_utf8_ffi makefile_linuxLP64_utf8_ffi_orig 
9c9
< CFLAGS = -fPIC -m64 -Wall -pedantic -Wno-uninitialized -Wno-strict-aliasing -Wno-long-long -c -O2 -g -DREADLINE -DSUPPORT_UTF8 -DNEWLISP64 -DLINUX -DFFI
---
> CFLAGS = -fPIC -m64 -Wall -pedantic -Wno-uninitialized -Wno-strict-aliasing -Wno-long-long -c -O2 -g -DREADLINE -DSUPPORT_UTF8 -DNEWLISP64 -DLINUX -DFFI -I/usr/local/lib/libffi-3.0.13/include 

[rickyboy]

Kludgy fix for one of those issues (in qa-ffi), for Lutz's approval for inclusion into the next version. Or better yet, come up with something better. :)

Code: Select all

(define LIBC (case ostype
               ("Win32" "msvcrt.dll")
               ("OSX" "libc.dylib")
               ("Linux" (if is64bit
                            "/lib/x86_64-linux-gnu/libc.so.6"
                            "/lib/i386-linux-gnu/libc.so.6"))
             ))

[hartrock]

Thanks for your solution: a switch between 32/64-bit seems to be reasonable.
Works for me (Debian 64-bit), but should be checked for other Linuxes, too: don't know, if all are following proposal https://wiki.ubuntu.com/MultiarchSpec.

[rickyboy]

Agreed, hartrock. I called it a "kludgy fix" for this type of reason.

On that note, here's a kludgy fix for qa-libffi. (Again, up to you, Lutz.)

First, add this procedure.

Code: Select all

(define (compile-recover CC-CALL)
  ;; Exec the compiler invocation CC-CALL (which is a string) and, on
  ;; failure, check certain error conditions, amend the invocation and try
  ;; again. This routine only handles re-compiling with -fPIC, for now.
  (let (cc-call-output (exec (string CC-CALL " 2>&1")))
    (if (not (empty? cc-call-output))
        ;; You can expand the following IF to handle more recovery cases.
        (if (find "recompile with -fPIC" (join cc-call-output " "))
            (compile-recover (string CC-CALL " -fPIC"))
            ;; Default case: if you've exhausted your recovery cases, then
            ;; just throw out the compiler's error message to the console.
            (! CC-CALL)))))

Next, change "!" to "compile-recover" in the current code; the expression should then look like this:

Code: Select all

(if (ends-with (real-path) "qa-specific-tests")
    (if (zero? (& 0x100 (sys-info -1)))
        (compile-recover "gcc -m32 ../util/ffitest.c -shared -o ffitest.dylib")
        (compile-recover "gcc -m64 ../util/ffitest.c -shared -o ffitest.dylib"))
    (if (zero? (& 0x100 (sys-info -1)))
        (compile-recover "gcc -m32 util/ffitest.c -shared -o ffitest.dylib")
        (compile-recover "gcc -m64 util/ffitest.c -shared -o ffitest.dylib"))
)