4 newLISP bugs in 10.1.9-dev and older

Notices and updates
Locked
itistoday
Posts: 429
Joined: Sun Dec 02, 2007 5:10 pm
Contact:

4 newLISP bugs in 10.1.9-dev and older

Post by itistoday »

While working on various newLISP code I've encountered several bugs.

Bug #1: context? reporting true for non-context

Code: Select all

> (new-class 'Foo)
Foo
> (setf f (instantiate Foo))
Foo#1
> (deallocate f)
true
> (context? f)
true
> (symbols f)

ERR: context expected in function symbols : f
You need ObjNL.lsp to run that example.

Bug #2: Memory leak

For this one ObjNL.lsp is also needed, simply because it's much easier to demonstrate the leak by using auto-release pools:

Code: Select all

> (new-class 'Foo)
Foo
> (define (inflate-memory) (push-autorelease-pool) (dotimes (_ 10000) (autorelease (instantiate Foo))) (pop-autorelease-pool))
(lambda () (push-autorelease-pool) 
 (dotimes (_ 10000) 
  (autorelease (instantiate Foo))) 
 (pop-autorelease-pool))
> (inflate-memory)
true
> (inflate-memory)
true
On my system using the 10.1.9-dev build, each time 'inflate-memory' is called an additional 0.1MB of memory is used up even though 'inflate-memory' cleans up after itself.

Bug #3: nil? evaluates its argument twice

This one is quite significant:

Code: Select all

> (nil? (inc a))
nil
> a
2
Bug #4: REPL can't properly handle [cmd][/cmd] tags

Take a look at this REPL session:

Code: Select all

> [cmd]
(define-macro (epln what)
	(when (nil? (println "\n" what "\n -> " (eval what)))
		(exit)
	)
)[/cmd]
[/cmd]
(lambda-macro (what) 
 (when (nil? (println "\n" what "\n -> " (eval what))) 
  (exit)))
nil
That shows one variant of the problem, where the ending [/cmd] tag isn't respected and must be entered again.

Note the 'nil' at the very end. This shows that newLISP is evaluating something... Here's a bizarre variant that shows some more insight to the problem:

Code: Select all

> [cmd](define-macro (epln what)
	(when (nil? (println "\n" what "\n -> " (eval what)))
		(exit)
	)
)[/cmd]
[/cmd]

nil
 -> nil
macbookpro:~$ 
Even though that should work, that will actually causes the 'epln' function to be evaluated, which is why it quits to the shell (because 'what' is nil).
Get your Objective newLISP groove on.

Lutz
Posts: 5289
Joined: Thu Sep 26, 2002 4:45 pm
Location: Pasadena, California
Contact:

Re: 4 newLISP bugs in 10.1.9-dev and older

Post by Lutz »

#1 and #2

Please show these problems in short but complete snippets. Try to boil it down to a minimum sequence of statements and expressions, which still shows the problem. I have no time, I go through user code myself.

#3

is fixed and updated in development/latest 10.1.9-dev

#4

The tags [cmd] and [/cmd] need to go on their own lines. These tags are primarily implemented to be used internally by 'net-eval' and by editors and IDEs interfacing with newLISP's REPL. If you work a lot in the REPL, you could use util/nls from the source distribution and customize it further for multi-line editing. Cormullion has an example on his site how to do this. What nls primarily does, is integratung the BASH and newLISP REPL and giving online syntax help for built-in functions.

itistoday
Posts: 429
Joined: Sun Dec 02, 2007 5:10 pm
Contact:

Re: 4 newLISP bugs in 10.1.9-dev and older

Post by itistoday »

Lutz wrote:#1 and #2

Please show these problems in short but complete snippets. Try to boil it down to a minimum sequence of statements and expressions, which still shows the problem. I have no time, I go through user code myself.
For #1:

Code: Select all

> (setf os (sym "blah"))
blah
> (setf o (new Class os))
blah
> (set 'o:self o 'o:os os)
blah
> (let (s o:os) (delete s) (delete s))
true
> (context? o)
true
For #2, here's a simplified version:

Code: Select all

(define (instantiate class)
	(letn (	obj-sym	(sym (string class "#" (inc class:@instance-counter)))
			obj		(new class obj-sym)
		)
		(set 'obj:@self obj 'obj:@self-sym obj-sym)
		obj
	)
)

; The existence of these functions causes a memory leak!
(context Class)
(define (Class:Class) true)
(define (Class:dealloc))
(define (Class:equals obj) (= obj @self))
(context MAIN)

(define (inflate-memory)
	(push '() autorelease)
	(dotimes (_ 10000)
		(push (instantiate Class) (first autorelease))
	)
	(dolist (obj (pop autorelease))
		(let (s obj:@self-sym)
			(delete s nil)
			(delete s nil)
		)
	)
)
I have figured out what causes it: simply defining functions inside of 'Class' will cause the memory leak. If you comment that part out there will be no memory leak.
#3

is fixed and updated in development/latest 10.1.9-dev
Great! :-)

Actually, there's one more bug that I forgot to mention:

Bug #5:

Take a look at this function:

Code: Select all

(define (new-class sym-class (super ObjNL) (interfaces '()) , class)
	(set 'class            (new super sym-class)
	     'class:@super     super
	     'class:@class     class
	     'class:@self      class
	     'class:@self-sym  sym-class
	)
	; NOTE: newLISP Bug? Why does pushing to the back result in odd behavior?
	; (push class class:@interfaces -1)
	(push class class:@interfaces)
	(dolist (iface interfaces)
		(setf iface (eval iface))
		(new  iface sym-class)
		(push iface class:@interfaces)
	)
	class
)
Take a look at the comment, it refers to "odd behavior". If the -1 is used then this is what will happen (it's hard to describe):

Code: Select all

> (define (protocol:test) "hello!")
(lambda () "hello!")
> (new-class 'Foo ObjNL '(protocol))
Foo
> (new Foo 'Bar)
Bar
> Bar:@interfaces
(protocol ObjNL)
That last output should be (protocol Foo ObjNL), not (protocol ObjNL). If you remove the -1 then it works fine.

Lutz
Posts: 5289
Joined: Thu Sep 26, 2002 4:45 pm
Location: Pasadena, California
Contact:

Re: 4 newLISP bugs in 10.1.9-dev and older

Post by Lutz »

#1 is fixed in development/latest dev-9

for #2, #4 and #5, I need more of your help.

Let me show you what I mean with "minimum sequence of statements and expressions producing the bug":

Basically I started with the snippet you gave me and put it in a file. Then I would take out statements and simplify expressions in new file copies until the error would either disappear or I had the minimum form producing the error.

You can see the sequence of elimination and simplification here:

Code: Select all

; the original
(setf os (sym "blah"))
(setf o (new Class os))
(set 'o:self o 'o:os os)
(let (s o:os) (delete s) (delete s))
(println "->" (context? o) " o ->" o)

; eliminate let
(setf os (sym "blah"))
(setf o (new Class os))
(set 'o:self o 'o:os os)
(set 's o:os) (delete s) (delete s)
(println "->" (context? o) " o ->" o)

; eliminate os:self
(setf os (sym "blah"))
(setf o (new Class os))
(set 'o:os os)
(set 's o:os) (delete s) (delete s)
(println "->" (context? o) " o ->" o)

; eliminate o:os
(setf os (sym "blah"))
(setf o (new Class os))
(set 's os) (delete s) (delete s)
(println "->" (context? o) " o ->" o)

; eliminate indirection of s to os
(setf os (sym "blah"))
(setf o (new Class os))
(delete os) (delete os)
(println "->" (context? o) " o ->" o)

; eliminate second delete
(setf os (sym "blah"))
(setf o (new Class os))
(delete os) 
(println "->" (context? o) " o ->" o)

; the final minimal form showing the core of
; the problem: the context cell in ctx does
; not get re-typed to nil, after blah is demoted
; to a normal symbol
(define blah:blah)
(set 'ctx blah)
(delete 'blah)
(context? ctx) ;=> true

; should result in nil, because 'blah gets demoted to
; a normal symbol, so blah is no context anymore
Only after doing this I could exclude programmer error and see that this was a bug. I also could pinpoint the error to the exact location in the code, because now the bug was well defined. Beforeit could have been all kinds of problems: was it the let? or the second delete? ar the double indirection to s? or the additional symbols in the context?

Doing this procedure on somebody else's code is time consuming. If the programmer does it himself, it is quicker, because one understands better the own code written and the critical elements to eliminate. One has also a clear picture of the problem and how to circumvent it.
Last edited by Lutz on Mon Dec 14, 2009 8:19 am, edited 1 time in total.

Kazimir Majorinc
Posts: 388
Joined: Thu May 08, 2008 1:24 am
Location: Croatia
Contact:

Re: 4 newLISP bugs in 10.1.9-dev and older

Post by Kazimir Majorinc »

I'll add two small bugs on top.

(legal? "\"")
(legal? "{")

Priority: lowest.

Lutz
Posts: 5289
Joined: Thu Sep 26, 2002 4:45 pm
Location: Pasadena, California
Contact:

Re: 4 newLISP bugs in 10.1.9-dev and older

Post by Lutz »

Thanks for catching this. There is a fixed development update 10.1.9-dev in the development/latest directory.

itistoday
Posts: 429
Joined: Sun Dec 02, 2007 5:10 pm
Contact:

Re: 4 newLISP bugs in 10.1.9-dev and older

Post by itistoday »

Lutz wrote:#1 is fixed in development/latest dev-9
Great!
for #2, #4 and #5, I need more of your help.
I assume you mean for #2 and #5.

For #2:

Code: Select all

(context Class)
(define (Class:equals obj) (= obj @self))
(context MAIN)

(define (inflate-memory)
	(push '() autorelease)
	(dotimes (_ 10000)
		(setf obj-sym (sym (string "Class#" (inc obj-counter))))
		(setf obj (new Class obj-sym))
		(setf obj:@self-sym obj-sym)
		(push obj (first autorelease))
	)
	(dolist (obj (pop autorelease))
		(let (s obj:@self-sym)
			(delete s nil)
			(delete s nil)
		)
	)
)
Enter that code into a REPL session, then run 'inflate-memory' and watch the memory grow each time. You can easily simply that down if you want to, I wrote it that way so that the memory leak is visible.

I've narrowed down the memory leak to the existence of this line:

Code: Select all

(define (Class:equals obj) (= obj @self))
Bug #5:

Code: Select all

(setf Class:@interfaces '(Class))
(new Class 'Foo)
(push Foo Foo:@interfaces -1)
(println Foo:@interfaces)
That should print (Class Foo) but it only prints (Class). Removing the '-1' results in correct behavior.
Get your Objective newLISP groove on.

itistoday
Posts: 429
Joined: Sun Dec 02, 2007 5:10 pm
Contact:

Re: 4 newLISP bugs in 10.1.9-dev and older

Post by itistoday »

Hmm... I'm not sure if this is a bug, but I think it should be as it's not what the documentation states and makes it less convenient to use.

'push' does not return a list if the symbol being pushed to does not exist:

Code: Select all

> (push 1 blah)
1
This means that one must ensure the list exists before combining 'push' with a list manipulating function, i.e.:

Code: Select all

(setf blah (unique (push foo blah)))
Get your Objective newLISP groove on.

Lutz
Posts: 5289
Joined: Thu Sep 26, 2002 4:45 pm
Location: Pasadena, California
Contact:

Re: 4 newLISP bugs in 10.1.9-dev and older

Post by Lutz »

This and other bugs are fixed in the current development/latest 10.1.9-dev. Currently only the cell-leak #2 is pending when using 'delete' on copied contexts.

Locked