Maintenance Release newLISP v.10.1.6

Notices and updates
Locked
Lutz
Posts: 5289
Joined: Thu Sep 26, 2002 4:45 pm
Location: Pasadena, California
Contact:

Maintenance Release newLISP v.10.1.6

Post by Lutz »

• This release contains bug fixes and new rewrite macro facility

Release notes: http://www.newlisp.org/downloads/newLIS ... lease.html
Downloads: http://www.newlisp.org/index.cgi?page=Downloads

cormullion
Posts: 2038
Joined: Tue Nov 29, 2005 8:28 pm
Location: latiitude 50N longitude 3W
Contact:

Re: Maintenance Release newLISP v.10.1.6

Post by cormullion »

Cool! I think I can have much damageXXXXXX fun with the new reader-event.. :)

I don't think it's as easy to disable it, though:

Code: Select all

> (define (rewrite expr) nil)
(lambda (expr) nil)
> 
> (reader-event rewrite)
$reader-event
> 
> (println hello)
nil
> (reader-event nil)
nil
> (println hello)
nil
> (exit)
>nil
I've painted myself into a corner...

(Actually I didn't type this first off. I made a mistake in my rewrite function...)

itistoday
Posts: 429
Joined: Sun Dec 02, 2007 5:10 pm
Contact:

Crash

Post by itistoday »

I've been getting a crash with versions since at least 10.1.4, and it's still there in 10.1.6. I'm still working on figuring out how to get it to run, but I've been able to at least get a fairly simple, reproducible crash, as well as a stack trace.

Two files are required:

./foo.lsp
./dragonfly-framework/lib/utils.lsp

./dragonfly-framework/lib/utils.lsp

Code: Select all

; this declaration needs to be at the top of the file because of newLISP bizarroness with 'load'
; TODO: bug? report on forum
(define-macro (define-subclass)
	(new (args 0 1) (args 0 0))
	(dolist (method (rest $args))
		(setf (method 0 0) (sym $it (args 0 0)))
		(eval (push 'define method))
	)
)

(context 'Dragonfly)

(define (load-once)
	; check if the last argument is a context (to behave like 'load' does)
	(let (ctx (let (_ctx (last $args)) (if (context? _ctx) _ctx MAIN)))
		(doargs (file)
			(unless (or (context? file) (find file _loaded))
				(push file _loaded)
				(saved-load file ctx)
			)
		)
	)
)

; We define our own module function so that we can easily support
; shared hosting services where the modules directory might not be
; in /usr/share/newlisp/modules.
(define (Dragonfly:module module-to-load)
	(if-not NEWLISP_DIR (throw-error "need value 'NEWLISP_DIR' config.lsp!"))
	(load-once (append NEWLISP_DIR "/modules/" module-to-load))
)

; places the key/value pairs from assoc-list into ctx
(define (into-ctx-assoc ctx assoc-list)
	(dolist (x assoc-list) (ctx (x 0) (x 1))) ; here dolist is slightly faster than map
)

; load all .lsp files directory
(define (load-lsp-files-in-dir dir)
	(dolist (x (directory dir "\.lsp$")) (load-once (append dir "/" x)))
)

(context 'MAIN)

; swap the MAIN functions for ours
(unless Dragonfly:saved-load
	(def-new  'load 'Dragonfly:saved-load)
	(constant 'load Dragonfly:load-once)
	(constant 'module Dragonfly:module)
)
./foo.lsp

Code: Select all

(context 'Dragonfly)
(load "./dragonfly-framework/lib/utils.lsp")
(load-lsp-files-in-dir "./dragonfly-framework/lib")
(context 'MAIN)
Then just do newlisp foo.lsp. Crash (stacktrace for 10.1.6):

Code: Select all

(gdb) r foo.lsp 
Starting program: /usr/bin/newlisp foo.lsp

Program received signal EXC_BAD_ACCESS, Could not access memory.
Reason: KERN_PROTECTION_FAILURE at address: 0x00000000
evaluateExpression (cell=0x0) at newlisp.c:1267
1267	switch(cell->type)
(gdb) bt
#0  evaluateExpression (cell=0x0) at newlisp.c:1267
#1  0x000066b8 in p_unless (params=0x0) at newlisp.c:5182
#2  0x00005802 in evaluateExpression (cell=0x819c80) at newlisp.c:1373
#3  0x00006918 in evaluateBlock (cell=0x819c80) at newlisp.c:5100
#4  0x00009e02 in dolist (params=0x819c60, doType=2) at newlisp.c:5597
#5  0x0000a0a9 in p_doargs (params=0x828000) at newlisp.c:5502
#6  0x00005802 in evaluateExpression (cell=0x819cc0) at newlisp.c:1373
#7  0x0000728b in let (params=0x819de0, type=0) at newlisp.c:4782
#8  0x000074e9 in p_let (params=0x828000) at newlisp.c:4653
#9  0x00005802 in evaluateExpression (cell=0x819e10) at newlisp.c:1373
#10 0x0000a8b8 in evaluateLambda (localLst=0x81a1c0, arg=<value temporarily unavailable, due to optimizations>, newContext=0x103870) at newlisp.c:1686
#11 0x0000583e in evaluateExpression (cell=0x81a470) at newlisp.c:1381
#12 0x00006918 in evaluateBlock (cell=0x81a470) at newlisp.c:5100
#13 0x00009e02 in dolist (params=0x81a260, doType=0) at newlisp.c:5597
#14 0x0000a0e9 in p_dolist (params=0x828000) at newlisp.c:5492
#15 0x00005802 in evaluateExpression (cell=0x81a220) at newlisp.c:1373
#16 0x0000a8b8 in evaluateLambda (localLst=0x81a4b0, arg=<value temporarily unavailable, due to optimizations>, newContext=0x103870) at newlisp.c:1686
#17 0x0000583e in evaluateExpression (cell=0x819ba0) at newlisp.c:1381
#18 0x0000d8bb in evaluateStream (stream=0xbfffefac, outDevice=0, flag=1) at newlisp.c:1103
#19 0x0000dc78 in loadFile (fileName=0xbffff2bd "foo.lsp", offset=0, encryptFlag=0, context=0x100570) at newlisp.c:2983
#20 0x0000efcc in main (argc=2, argv=0xbffff1f0) at newlisp.c:719
The problem seems to be related to the 'unless' block at the bottom of 'utils.lsp'. (Actually, it's the other unless block, the one inside of load-once.) Unfortunately I can't even move that into the Dragonfly context as then I'll get this error:

Code: Select all

(unless saved-load
	(def-new  'MAIN:load 'Dragonfly:saved-load)
	(constant 'MAIN:load Dragonfly:load-once)
	(constant 'MAIN:module Dragonfly:module)
)

=> ERR: symbol not in current context in function constant : load
Also, I was wondering why I have to put 'define-subclass' at the top of utils.lsp? If I try putting it at the bottom (after switching back to the MAIN context), then I get this error:

Code: Select all

ERR: string expected in function append : module-to-load
called from user defined function Dragonfly:load-once
called from user defined function Dragonfly:load-lsp-files-in-dir
Last edited by itistoday on Wed Oct 21, 2009 6:06 pm, edited 1 time in total.
Get your Objective newLISP groove on.

cormullion
Posts: 2038
Joined: Tue Nov 29, 2005 8:28 pm
Location: latiitude 50N longitude 3W
Contact:

Re: Maintenance Release newLISP v.10.1.6

Post by cormullion »

I can't get that error. With just what you posted:

Code: Select all

$ newlisp foo.lsp 

ERR: invalid function in function unless : (def-new  'load 'Dragonfly:saved-load)
Which figures. Don't understand the def-new function anyway.

The manual says "the current context must not be MAIN". And yet:

Code: Select all

(context 'MAIN)

; swap the MAIN functions for ours
(unless Dragonfly:saved-load
   (def-new  'load 'Dragonfly:saved-load)
Looks like you're in MAIN. ?

itistoday
Posts: 429
Joined: Sun Dec 02, 2007 5:10 pm
Contact:

Re: Maintenance Release newLISP v.10.1.6

Post by itistoday »

cormullion wrote:I can't get that error. With just what you posted:
Make sure you have the same directory setup (create a dragonfly-framework folder with a lib folder inside of it with utils.lsp inside of it, etc.).
The manual says "the current context must not be MAIN". And yet:
That's only if you're not specifying the target context.
Get your Objective newLISP groove on.

itistoday
Posts: 429
Joined: Sun Dec 02, 2007 5:10 pm
Contact:

Re: Maintenance Release newLISP v.10.1.6

Post by itistoday »

BTW, this problem is driving me crazy. I've tried just about everything to fix it, even removing the Dragonfly context from the scenario.

Example:

foo.lsp

Code: Select all

(load "./dragonfly-framework/lib/utils.lsp")
(load-lsp-files-in-dir "./dragonfly-framework/lib")
./dragonfly-framework/lib/utils.lsp

Code: Select all

(define-macro (define-subclass)
	(new (args 0 1) (args 0 0))
	(dolist (method (rest $args))
		(setf (method 0 0) (sym $it (args 0 0)))
		(eval (push 'define method))
	)
)

(define (load-once:load-once)
	; check if the last argument is a context (to behave like 'load' does)
	(let (ctx (let (_ctx (last $args)) (if (context? _ctx) _ctx MAIN)))
		(doargs (file)
			(unless (or (context? file) (find file _loaded))
				(push file _loaded)
				(MAIN:__saved-load file ctx)
			)
		)
	)
)

; We define our own module function so that we can easily support
; shared hosting services where the modules directory might not be
; in /usr/share/newlisp/modules.
(define (__module module-to-load)
	(if-not NEWLISP_DIR (throw-error "need value 'NEWLISP_DIR' config.lsp!"))
	(load-once (append NEWLISP_DIR "/modules/" module-to-load))
)

; places the key/value pairs from assoc-list into ctx
(define (into-ctx-assoc ctx assoc-list)
	(dolist (x assoc-list) (ctx (x 0) (x 1))) ; here dolist is slightly faster than map
)

; load all .lsp files directory
(define (load-lsp-files-in-dir dir)
	(dolist (x (directory dir "\.lsp$")) (load-once (append dir "/" x)))
)

; swap the MAIN functions for ours
(unless __saved-load
	(constant '__saved-load load)
	(constant 'load load-once)
	(constant 'module __module)
	(global 'load-lsp-files-in-dir 'into-ctx-assoc)
)
Still crashes at the same place.
Get your Objective newLISP groove on.

itistoday
Posts: 429
Joined: Sun Dec 02, 2007 5:10 pm
Contact:

Re: Crash

Post by itistoday »

itistoday wrote:The problem seems to be related to the 'unless' block at the bottom of 'utils.lsp'.
Actually, I'm wrong, it's the other unless block, the one that's inside of load-once.
Get your Objective newLISP groove on.

itistoday
Posts: 429
Joined: Sun Dec 02, 2007 5:10 pm
Contact:

Re: Maintenance Release newLISP v.10.1.6

Post by itistoday »

Think I've narrowed the crash down to the call to saved-load in load-once. Commenting it out works (but obviously that renders load-once useless). It will crash if it's uncommented, whether saved-load is defined as a symbol in Dragonfly context or a global symbol in main, and whether or not load-once is used as a default function or just a plain old function.
Get your Objective newLISP groove on.

Lutz
Posts: 5289
Joined: Thu Sep 26, 2002 4:45 pm
Location: Pasadena, California
Contact:

Re: Maintenance Release newLISP v.10.1.6

Post by Lutz »

In foo.lsp

Code: Select all

(context 'Dragonfly)
(load "./dragonfly-framework/lib/utils.lsp")
(load-lsp-files-in-dir "./dragonfly-framework/lib")
(context 'MAIN)
when the function (load-lsp-files-in-dir …) is running, it is redefining itself while running because utils.lsp is loaded recursively again. This causes the crash, because after the symbol 'load-lsp-files-in-dir' is redefined loading "utils.lsp" the old contents of symbol 'load-lsp-files-in-dir' is deleted.


PS: Also, why not in the utils.lsp simply redefine the built-in NEWLISPDIR to your custom module directory:

Code: Select all

(env "NEWLISPDIR" "/home/mypath")
the built-in 'module' function will then work on that path.

itistoday
Posts: 429
Joined: Sun Dec 02, 2007 5:10 pm
Contact:

Re: Maintenance Release newLISP v.10.1.6

Post by itistoday »

Lutz wrote:when the function (load-lsp-files-in-dir …) is running, it is redefining itself while running because utils.lsp is loaded recursively again. This causes the crash, because after the symbol 'load-lsp-files-in-dir' is redefined loading "utils.lsp" the old contents of symbol 'load-lsp-files-in-dir' is deleted.
Ah! Thanks Lutz! That seems to have fixed it.
PS: Also, why not in the utils.lsp simply redefine the built-in NEWLISPDIR to your custom module directory:

Code: Select all

(env "NEWLISPDIR" "/home/mypath")
the built-in 'module' function will then work on that path.
Good point, I didn't realize that it used NEWLISPDIR like that, but I guess I should have. Done. :-)

Also, it seems like define-subclass now can be put after the (context 'MAIN) switch at the end as well. Not sure why, but it works now.

Here's the updated utils.lsp (sans most comments):

Code: Select all

(context 'Dragonfly)

; protect against situation where one of the load functions is used to
; load this file, thereby redefining the function itself while it's running
; and causing newlisp to crash.
(unless load-once
	
	(define (load-once)
		(let (ctx (let (_ctx (last $args)) (if (context? _ctx) _ctx MAIN)))
			(doargs (file)
				(unless (or (context? file) (find file _loaded))
					(push file _loaded)
					(saved-load file ctx)
				)
			)
		)
	)

	(define (into-ctx-assoc ctx assoc-list)
		(dolist (x assoc-list) (ctx (x 0) (x 1))) ; here dolist is slightly faster than map
	)
	
	(define (load-files-in-dir dir regex-match)
		(dolist (x (directory dir regex-match))
			(load-once (append dir "/" x))
		)
	)
	
)

(context 'MAIN)

(define-macro (define-subclass)
	(new (args 0 1) (args 0 0))
	(dolist (method (rest $args))
		(setf (method 0 0) (sym $it (args 0 0)))
		(eval (push 'define method))
	)
)

; swap the MAIN functions for ours
(unless Dragonfly:saved-load
	(def-new 'load 'Dragonfly:saved-load)
	(constant 'load Dragonfly:load-once)
)
Get your Objective newLISP groove on.

Lutz
Posts: 5289
Joined: Thu Sep 26, 2002 4:45 pm
Location: Pasadena, California
Contact:

Re: Maintenance Release newLISP v.10.1.6

Post by Lutz »

newLISP by itself will only define NEWLISPDIR as /usr/share/newlisp, if not defined already. So you could also put this in an Apache .htaccess file

Code: Select all

setEnv NEWLISPDIR /blah/foo/mypath
now your Apache CGI processes will see NEWLISPDIR as that path, and newLISP when starting up will not change it.

itistoday
Posts: 429
Joined: Sun Dec 02, 2007 5:10 pm
Contact:

Re: Maintenance Release newLISP v.10.1.6

Post by itistoday »

Hmm... I just remembered the second reason that I wanted to have a custom module function and that was to prevent modules from being loaded twice. Does the current module function already behave this way or will it re-load the module? I'd like to avoid reloads for speed and also safety.
Get your Objective newLISP groove on.

Lutz
Posts: 5289
Joined: Thu Sep 26, 2002 4:45 pm
Location: Pasadena, California
Contact:

Re: Maintenance Release newLISP v.10.1.6

Post by Lutz »

If you enter 'module' in an interactive console, you see how it is defined (its just embedded as source in the C files):

Code: Select all

> module
(lambda (m) (load (append (env "NEWLISPDIR") "/modules/" m)))

> (global? 'module)
true
> 
Many schemes to avoid double loading are possible. You could redefine 'module' to push every file loaded on a module-list.

Code: Select all

(define (module m)
    (unless (find m module-list)
        (load (append (env "NEWLISPDIR") "/modules/" m))
        (push m module-list))
)

itistoday
Posts: 429
Joined: Sun Dec 02, 2007 5:10 pm
Contact:

Re: Maintenance Release newLISP v.10.1.6

Post by itistoday »

Lutz wrote:If you enter 'module' in an interactive console, you see how it is defined (its just embedded as source in the C files):

Code: Select all

> module
(lambda (m) (load (append (env "NEWLISPDIR") "/modules/" m)))

> (global? 'module)
true
> 
Many schemes to avoid double loading are possible. You could redefine 'module' to push every file loaded on a module-list.
Actually that's what my load-once does, and since I'm replacing newLISP's load with load-once, and module is defined in that way then module should also load modules only once. I should have checked before asking, I thought that because it was a built-in function it would be defined in C, but I'm glad it's not! :-)
Get your Objective newLISP groove on.

Locked