replace with empty pattern crashes and eats up memory

Q&A's, tips, howto's
Post Reply
HPW
Posts: 1377
Joined: Thu Sep 26, 2002 9:15 am
Location: Germany
Contact:

replace with empty pattern crashes and eats up memory

Post by HPW »

In newlisp 7.314:

Code: Select all

(set 'str "ZZZZZxZZZZyy")
(replace "[x|y]" str "PP" 0)
"ZZZZZPPZZZZPPPP"

(replace "" str "PP" 0)
Empty replace pattern crashes heavily by eating up memory from newlisp.exe. I entered it accidently on testing the DLL.
Hans-Peter

HPW
Posts: 1377
Joined: Thu Sep 26, 2002 9:15 am
Location: Germany
Contact:

Post by HPW »

This should be checked, because neobook users can do this often accidently. In neobook the [] (brackets) mark a neobook variable and in this case they are replaced with a empty string. neobook user have to use [#91] and [#93] instead.
Hans-Peter

Lutz
Posts: 5282
Joined: Thu Sep 26, 2002 4:45 pm
Location: Pasadena, California
Contact:

Post by Lutz »

this will be fixed in 7.3.15 today

Lutz

nigelbrown
Posts: 429
Joined: Tue Nov 11, 2003 2:11 am
Location: Brisbane, Australia

Post by nigelbrown »

Just to follow up on the memory eating behaviour,
would it be possible (in a future release) to have a command line
switch to set a limit on memory use and then give an out of memory
error - rather than trying to consume all of available windows memory?
Sometimes you 'know' any heavy memory use will be a programming
error that you would like to be able to (reset from.
In windows if newlisp grows the swap file to consume all swap space
then windows itself or explorer etc have problems and recovering the system
needs a reboot. To protect one from this windows feature
could we limit total (or subcategory) memory use.

I did a (sequence 2 averybignumber) and my win98 fell to bits.

Regards
Nigel

Lutz
Posts: 5282
Joined: Thu Sep 26, 2002 4:45 pm
Location: Pasadena, California
Contact:

Post by Lutz »

a memory check may be be an expensive thing to implement? One line of code but degrading speed, because it is one of the routines newLISP is calling all the time. But you could check the cell count in a function suspect for potential memory overuse with 'sys-info'. There are also other ways to control memory usage i.e. with memory quotas in Unix systems.

Lutz

nigelbrown
Posts: 429
Joined: Tue Nov 11, 2003 2:11 am
Location: Brisbane, Australia

Post by nigelbrown »

I was wondering (I've not looked at the code yet - sorry a bit lazy of me) if newlisp had a point where occasionally it requested another chunk of memory from windows to put aside as its own free space - then a memory check could be made at those occasional heap growing requests.
Nigel

Lutz
Posts: 5282
Joined: Thu Sep 26, 2002 4:45 pm
Location: Pasadena, California
Contact:

Post by Lutz »

Yes, there is a allocBlock() for a 1000 cells. Of course that would only handle cell memory (the biggest memory portion in most cases), it would not cover string allocations.

Perhaps for 8.0 I do the cell limit.

Lutz

nigelbrown
Posts: 429
Joined: Tue Nov 11, 2003 2:11 am
Location: Brisbane, Australia

Post by nigelbrown »

Hi Lutz
Thank you for adding the -m N memory limiting switch to 7.4.1 in the development stream.
I've tried it with
(define (usemem n)(dotimes (x n) (set (symbol (string "nnn" x)) (sequence 1 1000))))
and newlisp stops very gracefully.

Could you make the memory restriction limit available via (sys-info) so that a program could potentially take action if the memory limit was approaching or so it could check prior to a memory intensive action?
(I realise that newlisp can't say that the memory will actually be available from the system)

Nigel

Post Reply