New thread coming from here:
http://www.alh.net/newlisp/phpbb/viewtopic.php?t=168
>What do others think about memory management style? (Is it time to spawn another thread on this?)
I have not cared about much about the internals until now, because as long things were working quite well with sufficient performance, the job gets done. I have printed out the explanation to read it carefully later.
My large lists are mostly data, which are loaded at startup and are not much modified at runtime. Of cource I have to keep an eye on this when things gets bigger. I agree with Lutz to keep things simple, small and fast.
newLISP memory management
newLISP memory management
Hans-Peter
Much of my own code violates the recommendations I have given for dealing with large lists in newLISP. Specially the statistics functions, many of them take one or two dimensional lists as arguments and passed by value, I have never had performance problems. I should change the recommendations from 'few hundred' to 'few thousand' for large lists.
The processing done on the list itself in the application outweighs the time needed to copy the list. I did the following experiment:
(length (set 'lst (sequence 1 100000))) => 100,000 elements
(define (sum1 l) (apply + l))
(define (sum2) (apply + lst)
(time (sum1 lst) => 125 ms
(time (sum2) => 94 ms
sum1 will pass the lst by value and copy it, sum2 uses a global. On a large list the difference in speed is only about 25%. Lets see a smaller list:
(set 'lst (sequence 1 1000))
(time (dotimes (x 1000) (sum1 lst))) => 219 ms (219 us / iteration)
(time (dotimes (x 1000) (sum2))) => 203 ms (203 us / iteration)
On a smaller list about 10%. The larger list has proportionally more overhead, but overall and for practical purposes this overhead can be neglected. I will change the paper to reflect this.
Lutz
The processing done on the list itself in the application outweighs the time needed to copy the list. I did the following experiment:
(length (set 'lst (sequence 1 100000))) => 100,000 elements
(define (sum1 l) (apply + l))
(define (sum2) (apply + lst)
(time (sum1 lst) => 125 ms
(time (sum2) => 94 ms
sum1 will pass the lst by value and copy it, sum2 uses a global. On a large list the difference in speed is only about 25%. Lets see a smaller list:
(set 'lst (sequence 1 1000))
(time (dotimes (x 1000) (sum1 lst))) => 219 ms (219 us / iteration)
(time (dotimes (x 1000) (sum2))) => 203 ms (203 us / iteration)
On a smaller list about 10%. The larger list has proportionally more overhead, but overall and for practical purposes this overhead can be neglected. I will change the paper to reflect this.
Lutz
-
nigelbrown
- Posts: 429
- Joined: Tue Nov 11, 2003 2:11 am
- Location: Brisbane, Australia