memory leak in print?

Q&A's, tips, howto's
Locked
Sammo
Posts: 180
Joined: Sat Dec 06, 2003 6:11 pm
Location: Loveland, Colorado USA

memory leak in print?

Post by Sammo »

Hello Lutz & Hans-Peter,

Using newLisp.dll with NeoBook via the hpwNewLisp plug-in, I am leaking memory at the rate of (I think) two bytes per call. With this test function:

(define (donothing) "test string")

and these calls:

a) hpwNewLispCall "(donothing)" "[LispResult]"
b) hpwNewLispCall "(silent (donothing))" "[LispResult]"
c) hpwNewLispCall "(print (donothing))" "[LispResult]"
d) hpwNewLispCall "(silent (print (donothing)))" "[LispResult]"

the c) and d) calls (both containing 'print') are leaking memory; the other calls are not.

The '(silent (print (myfunction)))' idiom is necessary to properly return values to NeoBook. I found this problem after writing a NeoBook program that invokes hpwNewLispCall every 200 ms. After running for several hours, Win32 complained about "running low on virtual memory." I found the two-bytes-per-call memory leak by experimentation and by watching Task Manager's Mem Usage window.

Can you help? Is the memory leak in newLISP's 'print' function, in the hpwNewLispCall interface, or elsewhere?

Thanks,
-- Sam

HPW
Posts: 1390
Joined: Thu Sep 26, 2002 9:15 am
Location: Germany
Contact:

Post by HPW »

Hello Sam,

from the logical point it seems to me that it should be the 'print' command, since the interface only pass the string through. And this happens with every command in the same way. So when a+b does not leak, then it must be related to the lisp-command. So let's Lutz have a look at it.

Thanks for the observation. This could only be noticed by someone who is a 'bytecounter'. (exact, precise, correct)

Have you cross-tested it with the TK-enviroment?
Hans-Peter

Sammo
Posts: 180
Joined: Sat Dec 06, 2003 6:11 pm
Location: Loveland, Colorado USA

Post by Sammo »

Hans-Peter,

Being somewhat unfamiliar with Win32's Task Manager, I misunderstood what it was telling me. The leak is more serious than I reported; I'm actually losing 2K bytes per call! Since my program is calling newLisp.dll 20 times per minute (50 ms intervals instead of 200 ms as previously stated), that's 72,000 calls per hour. It's no wonder that Win32's virtual memory system complained and took a 700MB bite out of my HDD in the course of about five hours.

I tried to but could not duplicate the problem in the newlisp-tk and newlisp command-line environments.

HPW
Posts: 1390
Joined: Thu Sep 26, 2002 9:15 am
Location: Germany
Contact:

Post by HPW »

I crosstested BCC and GCC DLL's but both have the problem.

(dotimes (x 100)(print (donothing)))

eats about 192KB.
Hans-Peter

HPW
Posts: 1390
Joined: Thu Sep 26, 2002 9:15 am
Location: Germany
Contact:

Post by HPW »

You can reproduce it with newLISP:
newLISP v.8.2.0 Copyright (c) 2004 Lutz Mueller. All rights reserved.

> (import "newlisp.dll" "newlispEvalStr")

newlispEvalStr <100A150>
> (get-string (newlispEvalStr "(define (donothing) \"test string\")"))
"(lambda () \"test string\")"
> (get-string (newlispEvalStr "(donothing)"))
"\"test string\""
> (get-string (newlispEvalStr "(dotimes(x 100)(donothing))"))
"\"test string\""
> (get-string (newlispEvalStr "(dotimes(x 100)(donothing))"))
"\"test string\""
> (get-string (newlispEvalStr "(dotimes(x 100)(donothing))"))
"\"test string\""
> (get-string (newlispEvalStr "(dotimes(x 100)(print(donothing)))"))
Hans-Peter

HPW
Posts: 1390
Joined: Thu Sep 26, 2002 9:15 am
Location: Germany
Contact:

Post by HPW »

Also this way:
> (get-string(newlispEvalStr "(dotimes(x 100)(print \"Test\"))"))
Hans-Peter

HPW
Posts: 1390
Joined: Thu Sep 26, 2002 9:15 am
Location: Germany
Contact:

Post by HPW »

Problem was also in 7.500 and 8.000!
Hans-Peter

Lutz
Posts: 5289
Joined: Thu Sep 26, 2002 4:45 pm
Location: Pasadena, California
Contact:

Post by Lutz »

This is a known problem internal to Windows loosing system resources, it also happens when opening/closing files. I have now way to fix this in newLISP. The only workaround would be not to 'print' from inside the DLL but only from the parent application.

Instead of using 'print' use 'string' to return a string presentation to the parent process calling the DLL, i.e when calling from newLISP itself:

(print (get-string (newlispEvalStr "(string 1234 \"test\")")))


Lutz
Last edited by Lutz on Wed Sep 29, 2004 5:51 pm, edited 1 time in total.

Lutz
Posts: 5289
Joined: Thu Sep 26, 2002 4:45 pm
Location: Pasadena, California
Contact:

Post by Lutz »

print uses a 'stdout' system resource in Windows,

Lutz

HPW
Posts: 1390
Joined: Thu Sep 26, 2002 9:15 am
Location: Germany
Contact:

Post by HPW »

Not so good for us on the neobook side.

(string "Test")
returns
"Test"

we use print for formatiing neobook scripts with:

(silent(print "Test"))
returns
Test

Can you think of another command to get the formating of print without this side effekt?
Will also be the case on the other PC-enviroments.
Hans-Peter

HPW
Posts: 1390
Joined: Thu Sep 26, 2002 9:15 am
Location: Germany
Contact:

Post by HPW »

And why only with DLL's?
EXE's print works without this effekt!
Hans-Peter

HPW
Posts: 1390
Joined: Thu Sep 26, 2002 9:15 am
Location: Germany
Contact:

Post by HPW »

The discussion makes me thinking in a different way.

By now I thought that the return string must a ready script-stream for neobook. When only the outer Double-Quote pair is the problem, then i could possibly modify the interface to take away the first and last Double-quote and then feed it into the script-command buffer.

Sam,
what do you think? Should I modify the existing command or add a optional command for this? Then the stuff with (silent(print )) would be obsolet. We could use (string ) as Lutz suggested.

But maybe it is not the wanted behaviour for other PC-enviroments like Powerbasic, VB, VBA etc.
Hans-Peter

Lutz
Posts: 5289
Joined: Thu Sep 26, 2002 4:45 pm
Location: Pasadena, California
Contact:

Post by Lutz »

VBA and VB6 will work fine receiving the string. Using 'print' to pass things back actually sounds exotic to me, you normally would print/display in the host application not from an imported library. But I am not familiar with NeoBook.

Lutz

HPW
Posts: 1390
Joined: Thu Sep 26, 2002 9:15 am
Location: Germany
Contact:

Post by HPW »

This is a current return-line in lisp:

(silent(print "nb-action AlertBox \"Test\" \"This is a 1.Action!\"\nAlertBox \"Test\" \"This is a 2.Action!\"\n"))

It returns:
nb-action AlertBox "Test" "This is a 1.Action!" LF AlertBox "Test" "This is a 2.Action!" LF

The outer double-quotes are taken away with print and all escape-sequences are resolved /" get " and \n gets 'LF'. (chr 10)

So it is not only deleting outer double quotes it is also resolving escape-sequences. A lot more to do in the interface.
Hans-Peter

HPW
Posts: 1390
Joined: Thu Sep 26, 2002 9:15 am
Location: Germany
Contact:

hpwNewLISP 1.11

Post by HPW »

I have a working prototype of my neobook-plugin hpwNewLISP which allow a more flexibel handling with the return-string.

So instead of:
(silent(print "nb-action AlertBox \"Test\" \"This is a 1.Action!\"\nAlertBox \"Test\" \"This is a 2.Action!\"\n"))
you can use optional:
(string "nb-action AlertBox \"Test\" \"This is a 1.Action!\"\nAlertBox \"Test\" \"This is a 2.Action!\"\n")
or at least the string alone:
"nb-action AlertBox \"Test\" \"This is a 1.Action!\"\nAlertBox \"Test\" \"This is a 2.Action!\"\n"
or the symbol containing the return-string is the last statement in the function.

I think it is more consistent to get back a string "Text" as 'Text' inside the neobook-return-variable than the current "Text".
But this could cause some changes in current neobook sources.

So what do I do:
I scan the return string for
first and last char = "
or:
first six and last seven = [text][/text]

This text-flags are then deleted.
Then I do a replace:
\" to "
\r to chr(13)
\n to chr(10)
\t to chr(9)
\\ to \

Demo-pub is running so far. I will do some more testing and release it in the evening when I am back home.

Without the need of print we have a workaround for the memory-leak.
(No more exotic thing to pass things back)
As always: The right tool to get the work done!
;-)
Hans-Peter

HPW
Posts: 1390
Joined: Thu Sep 26, 2002 9:15 am
Location: Germany
Contact:

Post by HPW »

hpwNewLISP 1.11 is released.
Hans-Peter

Sammo
Posts: 180
Joined: Sat Dec 06, 2003 6:11 pm
Location: Loveland, Colorado USA

Post by Sammo »

Thank you, Hans-Peter!

A nice solution, and it works perfectly for me. I stripped away the (silent (print ...)) code from the offending application. With your updated NeoBook plug-in, my application that called the newLisp.dll 20 times per second is no longer leaking memory.

Locked