newLisp without collateral effects

Pondering the philosophy behind the language
Locked
ale870
Posts: 297
Joined: Mon Nov 26, 2007 8:01 pm
Location: Italy

newLisp without collateral effects

Post by ale870 »

Hello,

after some time I restarted to use newLisp! I love this language.

Now I wanted to try to use it implementing code to avoid the "famous" collateral effects. Since newLisp supports dynamic scope, it introduces them, but I can use some techniques to avoid it.

For example:

Code: Select all

> (define (myFunc b)(println "SUM:" (+ a b)))
(lambda (b) (println "SUM:" (+ a b)))

> (setq a 10)
10

> (myFunc 5)
SUM:15
15
>
As you can see I introduced an external variable, so I "introduced" a collateral effect. I can simply avoid this by making the program in this way:

Code: Select all

> (define (myFunc a b)(println "SUM:" (+ a b)))
(lambda (a b) (println "SUM:" (+ a b)))

> (myFunc 1 5)
SUM:6
6
>
Well, now see the following code:

Code: Select all

> (setq counter 0)
(map 
  (fn(x)
    (inc counter)
    (println "Value " counter " --> " x) )

  '("A" "B" "C") )

Value 1 --> A
Value 2 --> B
Value 3 --> C
("" "" "")
>
The problem is the variable "counter" introduces a collateral effect, but I cannot find a way to avoid it. Can you help please?
I'm very interested to solve this class of problems!

Thank you!
--

ale870
Posts: 297
Joined: Mon Nov 26, 2007 8:01 pm
Location: Italy

Re: newLisp without collateral effects

Post by ale870 »

I found this solution using CONTEXT, but I wanted to try something without using them and using "pure" Lisp code:

Code: Select all

(define (CALC:myFunc x)
    (inc CALC:counter)
    (println "Value " CALC:counter " --> " x) )

(map CALC:myFunc '("A" "B" "C"))
I "moved" the counter inside a context, so I can say that I minimize the collateral effect since the functions and values inside the context are "controlled", but it is more "object-oriented" approach, and does not eliminate the collateral-effect.

So... ???
--

ale870
Posts: 297
Joined: Mon Nov 26, 2007 8:01 pm
Location: Italy

Re: newLisp without collateral effects

Post by ale870 »

Another question: can I create a lambda function inside a context, in order to use context-insulation?
--

tomtoo
Posts: 46
Joined: Wed Oct 28, 2009 10:00 pm

Re: newLisp without collateral effects

Post by tomtoo »

"silent" suppresses console output, or you could run your code from a script.

ale870
Posts: 297
Joined: Mon Nov 26, 2007 8:01 pm
Location: Italy

Re: newLisp without collateral effects

Post by ale870 »

Sorry but I don't understand your answer on my question.
Do you know what are "collateral effects" in a programming language? My problem is not text in output, but the capability for a programming language to run functions that take values from outside the function self (based on the variables scoping).
--

tomtoo
Posts: 46
Joined: Wed Oct 28, 2009 10:00 pm

Re: newLisp without collateral effects

Post by tomtoo »

Ok, sorry, disregard. I thought I'd take a stab at it.

m i c h a e l
Posts: 394
Joined: Wed Apr 26, 2006 3:37 am
Location: Oregon, USA
Contact:

Re: newLisp without collateral effects

Post by m i c h a e l »

Hi Alessandro!

About your solution using a context, you say:
Alessandro wrote:. . . does not eliminate the collateral-effect.
Why not? Perhaps I'm not sure what you mean by collateral-effect. You started by saying:
Alessandro wrote:I found this solution using CONTEXT, but I wanted to try something without using them and using "pure" Lisp code
So it sounds like you are referring to pure functional programming (without side effects), but later you say:
Alessandro wrote:My problem is not text in output, but the capability for a programming language to run functions that take values from outside the function self (based on the variables scoping).
So I'm not sure. Do you mean how to pass a variable by reference rather than by value?

m i c h a e l

ale870
Posts: 297
Joined: Mon Nov 26, 2007 8:01 pm
Location: Italy

Re: newLisp without collateral effects

Post by ale870 »

Sorry I made a mistake in the naming: I was talking about "side-effects" not "collateral-effects" :-)

In fact I wanted to use MAP function without using an external variable, which introduce the side-effects.

Even contexts, do not eliminate side-effects, just like object-oriented programming (even OO has side-effects, since some variables are inside a class but outside the methods contained in the class self).
Contexts are "similar" to OO class: they minimize side effects but not eliminate them.

I wish to find a method to eliminate side-effects using MAP.

Thank you for your help!
--

johu
Posts: 143
Joined: Mon Feb 08, 2010 8:47 am

Re: newLisp without collateral effects

Post by johu »

If my understanding to this discussion might be wrong, I am sorry.
It means the following ?

Code: Select all

> (setq counter 0)
0
> (map (fn(x) (inc counter) (println "Value " counter " --> " x)) '("A" "B" "C"))
Value 1 --> A
Value 2 --> B
Value 3 --> C
("A" "B" "C")
> counter
3
> (setq counter 0)
0
> (let (counter 0) (map (fn(x) (inc counter) (println "Value " counter " --> " x)) '("A" "B" "C")))
Value 1 --> A
Value 2 --> B
Value 3 --> C
("A" "B" "C")
> counter
0
> (define (myFunc x) (inc counter) (println "Value " counter " --> " x))
(lambda (x) (inc counter) (println "Value " counter " --> " x))
> (setq counter 0)
0
> (map myFunc '("A" "B" "C"))
Value 1 --> A
Value 2 --> B
Value 3 --> C
("A" "B" "C")
> counter
3
> (setq counter 0)
0
> (let (counter 0) (map myFunc '("A" "B" "C")))
Value 1 --> A
Value 2 --> B
Value 3 --> C
("A" "B" "C")
> counter
0
> 

ale870
Posts: 297
Joined: Mon Nov 26, 2007 8:01 pm
Location: Italy

Re: newLisp without collateral effects

Post by ale870 »

As per my knowledge, every example you made "suffers" the side-effects.
In order to eliminate a side-effect, one must only call a function passing all values as parameters, and get the function results. No global variable should be used.

For example, this is the wrong approach ("counter" introduces the side-effects):

Code: Select all

(setq counter 0)

(define (myFunc)
	(dotimes (i 3)
  	(inc counter)
  	(println "VALUE [" i "]: " counter) ) )

(myFunc)
This is the correct approach (no side-effect):

Code: Select all

(define (myFunc)
  (setq counter 0)
	(dotimes (i 3)
  	(inc counter)
  	(println "VALUE [" i "]: " counter) ) )

(myFunc)
As you can see, in order to eliminate the side-effects, one must eliminate every reference, inside functions, to external variables (global variables).
This approach is very useful in order to avoid that, another function (XYZ) may change the value of the global variable, by introducing unexpected results (and a big BUG!) inside (myFunc).
In fact, if one function uses global variables (or variables external to the function self), this case may happen:

Code: Select all

(setq counter 0)

(define (anotherFunc)
  (dec counter 5) )

(define (myFunc)
	(dotimes (i 3)
  	(inc counter)
		(anotherFunc)
  	(println "VALUE [" i "]: " counter) ) )

(myFunc)
As you can see, (myFunc) makes a job, but there is the function (anotherFunc) that modify the value of counter, so I introduced a bug (difficult to discover in a complex application), and I get unexpected results.

Another correct approach for this problem could be the following code:

Code: Select all

(define (myFunc myI myCounter)
	(inc myCounter)
	(println "VALUE [" myI "]: " myCounter)
	myCounter)

(setq counter 0)

(dotimes (i 3)
	(setq counter (myFunc i counter)) )

(myFunc)
As you can see, the function (myFunc) never works on "counter" directly, but over a controlled copy.

The concept is: a function must get some values, can create local (controlled) variables, then it will return the results. A function should never work on variables external to its scoping.
--

johu
Posts: 143
Joined: Mon Feb 08, 2010 8:47 am

Re: newLisp without collateral effects

Post by johu »

I see.
My understanding to this discussion is wrong.
I'm sorry.

By the way,
In fact I wanted to use MAP function without using an external variable, which introduce the side-effects.
In newLISP, the internal system variable $idx can be used within map.
If myFunc will be used only within map,

Code: Select all

> (define (myFunc x) (println "Value " $idx " --> " x))
(lambda (x) (println "Value " $idx " --> " x))
> (map myFunc '("A" "B" "C"))
Value 0 --> A
Value 1 --> B
Value 2 --> C
("A" "B" "C")
>  
Thank you for the explanation.

ale870
Posts: 297
Joined: Mon Nov 26, 2007 8:01 pm
Location: Italy

Re: newLisp without collateral effects

Post by ale870 »

That variable is an internal loop counter, and its usage is really limited. In our example it works, but I don't think that is a good generic solution.
That is a tricky workaround however!
--

ale870
Posts: 297
Joined: Mon Nov 26, 2007 8:01 pm
Location: Italy

Re: newLisp without collateral effects

Post by ale870 »

I found a workaround to limit the "damages" that a global function can do.
See the following code:

Code: Select all

(setq counter 0)

(define (bad-boy)
	(setq counter 2000) )

(map
	(fn (x) 
		(let ( (myCounter counter)		)
			(inc myCounter)
			(bad-boy)
			(println "Value: " myCounter " --> " (upper-case x)) 
			(setq counter myCounter) ) )
		 '("a" "b" "c") )
I use a trick: I use a global variable "counter" to maintain the counter, but when the lambda function starts, I save counter value in a local variable, so in the function I work with the local value. It means that, even if the function (bad-boy) change the value of the global counter, my function still works correctly.
It is not a good procedure, since if (bad-boy) changes the value means there is a reason to do that, but in this way the side-effects are under control.

I'm still looking for the definitive solution.
We need a way to create a variable inside the lambda function, but that value must exist across different function calls.

Another possible solution is using a mix of contexts (dynamically generated inside the function self) and the variable $idx ;-)

Code: Select all

(map
	(fn (x) 
		(if (= $idx 0) (setq MY_CONTEXT:myCounter 1) (inc MY_CONTEXT:myCounter) )
			(println $idx " - " "Value: "  MY_CONTEXT:myCounter " --> " (upper-case x))  )
		 '("a" "b" "c") )
Practically speaking, I use $idx=0 to inizialize a variable inside a context to manage the counter. Even if that variable is potentially accessible from externally of the function, we can say that symbol is "private" to the function self (since that context is not referenced anywhere outside the function self).
Then after the context variable was initialized, I use it as usual.

Even this approach is not fully side-effect free, but I really cannot find other ways to avoid side-effects to solve this algorithm using the (map) function.
the only definitive solution seems eliminating (map) and using a more "classic" LOOP (e.g. a (dotimes)).

This problem is interesting since MAP is a key function in LISP and it seems it goes against the logical approach of the language self: that function does not allow to allow us to make an algorithm side-effect free!
--

ale870
Posts: 297
Joined: Mon Nov 26, 2007 8:01 pm
Location: Italy

Re: newLisp without collateral effects

Post by ale870 »

Ok, I have another solution. Even this one is not so clean, but I like it:

Code: Select all

(define (bad-boy)
	(inc counter 100) )

(let ( (counter 0) )
	(map
		(fn (x)
			(inc counter)
			(bad-boy)
			(println $idx " - " "Value: " counter " --> " (upper-case x))  )
		'("a" "b" "c") ) )
I used a (let) function to create a local variable "counter". As you can see an external function, caused by dynamic binding, can change a "locally defined variable, but there is one point: who can make an external function (bad-boy) which works on a local defined variable?!
--

cormullion
Posts: 2038
Joined: Tue Nov 29, 2005 8:28 pm
Location: latiitude 50N longitude 3W
Contact:

Re: newLisp without collateral effects

Post by cormullion »

not sure what the issue is here, but you could do this:

Code: Select all

(map
      (fn (x)
         (inc (copy counter))  ; -<<<<<<
         (bad-boy)
         (println $idx " - " "Value: " counter " --> " (upper-case x))  )
      '("a" "b" "c") )

ale870
Posts: 297
Joined: Mon Nov 26, 2007 8:01 pm
Location: Italy

Re: newLisp without collateral effects

Post by ale870 »

Ciao @cormullion, I tried it but does not solve side-effects problems.
Maybe you need to use (copy) but assigning the result to a new local variable:

Code: Select all

(map
      (fn (x)
         (setq myCounter (inc (copy counter)) ) ; @@@@@@@@@@@@@@@@@@@
         (bad-boy)
         (println $idx " - " "Value: " myCounter " --> " (upper-case x))  )
      '("a" "b" "c") )
But this approach is similar to (let) usage I tried before.
--

Lutz
Posts: 5289
Joined: Thu Sep 26, 2002 4:45 pm
Location: Pasadena, California
Contact:

Re: newLisp without collateral effects

Post by Lutz »

Code: Select all

(define (counter:counter x) 
  (println $idx "-->" (upper-case x)) (inc counter:cnt)
  (bad-boy))

(define (bad-boy) 
  (println "count: " counter:cnt))

(map counter '("a" "b" "c"))
I also added two subchapters to: http://www.newlisp.org/downloads/CodePa ... html#toc-9 showing "Functions with memory" and "Functions using self modifying code". Both are applicable to your problem.

ale870
Posts: 297
Joined: Mon Nov 26, 2007 8:01 pm
Location: Italy

Re: newLisp without collateral effects

Post by ale870 »

Nice info @Lutz, thank you! I will check them immediately!
--

ale870
Posts: 297
Joined: Mon Nov 26, 2007 8:01 pm
Location: Italy

Re: newLisp without collateral effects

Post by ale870 »

@Lutz can you explain me this one:

Code: Select all

(define (sum (x 0)) (inc 0 x))
OK I tried it and works, but I don't understand HOW it works!
--

johu
Posts: 143
Joined: Mon Feb 08, 2010 8:47 am

Re: newLisp without collateral effects

Post by johu »

Code: Select all

> (define (sum (x 0)) (inc 0 x))
(lambda ((x 0)) (inc 0 x))
> (sum 1)
1
> sum
(lambda ((x 0)) (inc 1 x))
> (sum 3)
4
> sum
(lambda ((x 0)) (inc 4 x))
> 
Note that the lambda expression has been changed after executing sum.
The built-in function inc is destructive.
But this way might not be used in map.

Code: Select all

> (define (myFunc x) (println "Value " (inc 0 1) " --> " x))
(lambda (x) (println "Value " (inc 0 1) " --> " x))
> (map myFunc '("A" "B" "C"))
Value 1 --> A
Value 1 --> B
Value 1 --> C
("A" "B" "C")
> (dolist (x '("A" "B" "C")) (myFunc x))
Value 1 --> A
Value 2 --> B
Value 3 --> C
"C"
> (let (res) (dolist (x '("A" "B" "C")) (push (myFunc x) res -1)))
Value 4 --> A
Value 5 --> B
Value 6 --> C
("A" "B" "C")
> 

Kazimir Majorinc
Posts: 388
Joined: Thu May 08, 2008 1:24 am
Location: Croatia
Contact:

Re: newLisp without collateral effects

Post by Kazimir Majorinc »

I think that appropriate approach in this situation would be:

Code: Select all

(local(counter)
  (map (lambda(x)(inc counter)(print counter "-->" x ";  "))
       '(A B C))) 
(print counter)
1-->A; 2-->B; 3-->C; nil

because side effects are really needed here, just you do not need it for long. Without some side effects, there would be 1-->A; 1-->B; 1-->C; nil. If you want to protect yourself against accidental name clashes, it is more general problem. One really must care about unique names. For example, if you want that in

(define 'bad-boy (lambda()(setq counter 2000)))

some specific counter is used, not counter that can be supplied from environment that calls bad-boy, you must give it specific name. In Common Lisp, there is convention to use globals on this way:

(define 'bad-boy (lambda()(setq *counter* 2000)))

but you can use your own notation. For example, I wrote two helper functions set-protected1 and set-protected2. http://kazimirmajorinc.blogspot.com/200 ... exprs.html

(set-protected1 'bad-boy (lambda()(setq counter 2000)) '(counter))

==>

(lambda () (setq .<.bad-boy____counter.>. 2000))

So, variable counter is renamed to the form that cannot be accidentally rewritten. It actually works:

Code: Select all

(load "http://instprog.com/Instprog.default-library.lsp")

(set-protected1 'bad-boy 
                (lambda()(setq counter 'ruin)
                         (setq temp 'destroy)) 
                '(counter temp))
 
(map (begin (set-protected1 'temp 
                            (lambda(x)
                                (inc counter)
                                (bad-boy)
                                (println " Value " counter "===>" x))
                             '(x counter))
             temp)
      '("A" "B" "C"))
      
(println counter)
Value 1===>A
Value 2===>B
Value 3===>C
nil


However, contexts were designed exactly for that purpose, and I defined my functions only to see how it can be done without contexts, as proof-of-the-concept. Itistoday (Greg Slepak) defined the function similar to protected1 that uses contexts to ensure uniqueness of the used variables.

Lutz
Posts: 5289
Joined: Thu Sep 26, 2002 4:45 pm
Location: Pasadena, California
Contact:

Re: newLisp without collateral effects

Post by Lutz »

Re: self modifying functions
But this way might not be used in map.
Actually you can, if you quote myFunc when mapping:

Code: Select all

> (define (myFunc x) (println "Value " (inc 0 1) " --> " x))
(lambda (x) (println "Value " (inc 0 1) " --> " x))
> (map 'myFunc '("A" "B" "C")) ; <-- note the quote before myFunc
Value 1 --> A
Value 2 --> B
Value 3 --> C
("A" "B" "C")
> 
Without the quote map will construct expressions like this when iterating:

Code: Select all

((lambda (x) …) "A")
((lambda (x) …) "B")
((lambda (x) …) "C")
When quoting myFunc, map will construct this kind:

Code: Select all

(myFunc "A")
(myFunc "B")
(myFunc "C")
The symbol reference to myFunc causes the same function to be used over and over and modified.

All built-in destructive functions in newLISP can do in place modification, if you want to (*). If this is a good programming style, is an entirely different topic. Just like the underlying question if code=data is a good thing or not. I think it's a good thing in the right circumstances.

(*) this is also tested in newlisp-x.x.x/qa-specific-tests/qa-inplace

johu
Posts: 143
Joined: Mon Feb 08, 2010 8:47 am

Re: newLisp without collateral effects

Post by johu »

Thank you, Lutz.

I know the difference between quoted and unquoted in map, also apply.

ale870
Posts: 297
Joined: Mon Nov 26, 2007 8:01 pm
Location: Italy

Re: newLisp without collateral effects

Post by ale870 »

Hello,

your posts are really interesting.

I think @Kazimir's solution is good, since it creates variables only for the needed time, and it is a more "classic programming" approach.
About @Lutz's solution, that is the one I was looking for, in fact the only way to eliminate side-effects was to implement a kind of "memory" inside the function self.
Some languages do not apply the concept of self-modifying code ( I remember self-modifying code when I made programs in assembler for MC68000!). Some languages use a "special" kind of local variables which can maintain their value across multiple function calls, even if the variable is not visible externally of the function self.
I think that self modifying code is a good approach due to the mature of lisp in general. This is a clear example sue to push to the limit of the dual nature of lisp (data<->programs). This concept remember me the dual nature of the photons (energy<->matter) ;-)

I never thought to use destructive functions in this way, and I like it! I will spend some time to better learn about them!
--

Locked