development release newLISP version 9.9.2

[Lutz]

This is the first development release of the new v.10.0 series of newLISP.

Several changes in this version are incompatible with the previous 9.4.x series of releases.

At the end of the CHANGES notes for 9.9.2 is a "Short conversion guide".

For files and CHANGES notes: http://www.newlisp.org/downloads/development

[Jeff]

Wonderful! Thanks, Lutz. Couple of things:

1. Can we return references from our own functions?

Code: Select all

(define (my-first lst)
  (expand '(lst 0) 'lst))

2. Lookup should accept the standard index syntax:

Code: Select all

(set 'foo '((a 1) (b 2) (c ((a 2) (b 3)))))
(assoc (foo 'c 'b)) ; => (b 3)
(lookup (foo 'c 'b)) ; => 3

As opposed to:

Code: Select all

(set 'foo '((a 1) (b 2) (c ((a 2) (b 3)))))
(assoc (foo 'c 'b)) ; => (b 3)
(lookup '(b c) foo) ; => 3

[Lutz]

Only flat syntax in 9.9.2 for all functions (see CHANGES notes). When using previous form: (assoc (aList key1 key2)) then the inner parenthesis are syntax and you can neither 'apply', 'curry' or 'map'. That form also created ambiguities in all 'ref' functions and was generally to slow to parse and distinguish by the evaluator.

And nested associations are and always have been formed like this:

Code: Select all

(set 'foo '((a 1) (b 2) (c (a 2) (b 1 2 3))))

(assoc '(c b) foo) => (b 1 2 3)

(lookup '(c b) foo -1) => 3

This way it conforms to nested objects in FOOP.

Pass references in and out of user defined functions using contexts/default functors:

Code: Select all

(set 'db:db '(a b c d e f g))

(define (my-reverse data)
	(reverse data)
	data)

(pop (my-reverse db)) 

db:db => (f e d c b a)

ps: in this version apply on built-in functions doesn't yet respect ref returns.

[Kazimir Majorinc]

Lutz,

I hope I'm not too boring, but if you make some syntactical construct that force evaluation of the result of the function in caller environment, you get "return by reference" and CL macros for free.

Say, f = (elambda(x) .... 'result)

And if interpreter came across such function, he does

(f arg1 ... argn) = (eval (f0 arg1 ... argn)) where f0=(lambda(x)...'result)

This solves practically all problems.

I already proposed that (and implemented support for that through preprocessing in some 30 lines of Newlisp code) but it is better if interpreter can do it automatically.

If you do not want to specialize kinds of functions further - it is possible to define generalized function

(elambda pre post (x) ... 'result)

(f expr1 ... exprn) =(post (apply f0 (pre (expr1 ... exprn)))
(or something like that) in caller environment

Returning by reference, ordinary functions, CL macros, NL macros and lot more are just special cases of such generalized functions. With this syntax you get some support of "aspect oriented programming" for free.

[HPW]

I am glad to report that my product configurator (neobook/newlisp.dll) runs unchanged with 9.9.2 !
;-)

[Lutz]

Kazimir:

Any of these solutions must be able to recover the original variable environment without cell leaks and multiple memory-object references in case of early functions return when errors occur, or when 'catch' and 'throw' is involved. Only symbols and contexts are exempted.

The problem is always in those details. There seems to be no solution which doesn't represent a big performance hit on lambda-function overhead.

Passing symbols is the traditional way to do it, but is not hygienic, prone to variable capture. The best solution is using context symbols defaulting to their default functor.

The current solution for references into and out from lambda expression via default functors is a nice solution, which does not add overhead and lets you use the same function in both copying or reference passing mode:

Code: Select all

(set 'L:L '(a b c d e f g)) 
(set 'l '(a b c d e f g))

(define (my-reverse data) 
   (reverse data) 
   data) 

; pass in and out by reference
(pop (my-reverse L)) 
L:L => (f e d c b a)

; pass in and out by copy
(pop (my-reverse  l))
l => (a b c d e f g)

The caller of 'my-reverse' decides the mode by calling either with a normal variable or a context.


HPW:

congratulations, hopefully others have a similar experience. There are still some details to take care of though, and everybody should be careful when deploying the new version and test everything. There will be regular development version updates all through September and October to have it rock-solid.

[cormullion]

from 9.9.1 and 9.9.2 towards newLISP v10.0It also obsoletes the functions 'set-assoc', 'assoc-set', 'set-nth 'nth-set', 'ref-set'. As these function don't occur much in average newLISP, source changes necessary in existing code are minimal.

How do you know? You never asked. I stopped counting when I got to 100 occurrences of one of them. And yet you sent me an email in January which mentions changing the flat syntax of functions to the new nested form for 9.3:

I am not sure yet what to do about it, these are used very frequently and there is a lot of legacy code.

?

[Lutz]

How do you know? You never asked

My assessment of the situation in January was not based on actually counting, just an estimation I made. Now after actually looking into this issue in a more systematic fashion, I know better.

A couple of weeks ago unhappy with the current situation of multiple syntax forms incompatible with 'map', 'apply' and 'curry' and the ambiguities it was sometimes causing, I started to count and analyze code. First my own then other people's code. E.g. I went through all of the modules published and linked to on http://www.newlisp.org/modules

To my surprise others had as little occurrences of these functions as I do. About 90% of the files are not affected at all. When files where affected often a fix was not required (i.e. already using a flat syntax in simple 'nth' and 'assoc') or the eliminated setter functions where not used at all.

ps: if any body needs help converting, come and ask!

[cormullion]

ps: if any body needs help converting, come and ask!

If you write us a newlisp 9to10 (like Python's 2to3) I'll be happier... I can't face going through all my code for so little gain.

[HPW]

Our last newlisp-tk and it's demo.lsp also runs unchanged with 9.9.2!

[Lutz]

Thanks for the report HPW.


Ps: FOOP seems to be fine in 9.9.2 but problems in the FOOP assoc examples in the manual.

[xytroxon]

Errata:

newlisp-9.9.2 source code file "makefile"

help:
...
@echo " make linux_lib_utf8
@echi " make debian <<<<<<<<<< change @echi to @echo
@echi " make debian_utf8 <<<<<< change @echi to @echo
@echo " make linux64ILP32
----------------

File "newlisp_manual.html"

HTTP-only server mode
...
file extension media type
.jpg image/jpg
.pgn image/png <<<<<<< Change .pgn to .png
.gif image/gif
.pdf application/pdf
.mp3 image/mpeg
.mov image/quicktime
.mpg image/mpeg
any other text/html
...

Suggest adding mime formats:

.wmv .avi ( windows movie formats )
.zip .gz .bz2 .7z .rar etc. ( compressed files ) (Note: .7z format -> http://www.7-zip.org/)

And shouldn't the "any other" behavior be text/plain?
Then would need .htm .html

-------
End of errata
----------------------------
Some history from Common Lisp...

Re: SETQ vs SETF
http://coding.derkeiler.com/Archive/Lis ... /2218.html
From: Christopher C. Stacy (cstacy_at_news.dtpq.com)
Date: 03/24/04

Date: Wed, 24 Mar 2004 06:03:14 GMT

>>>>> On Wed, 24 Mar 2004 04:15:23 GMT, David Steuber ("David") writes:

David> Is there ever a reason to use setq in preference to setf?

David> Since setf can do what setq can do, and more,
David> why should anyone use setq?

David> There is also set, but the CLHS says it is deprecated.

SETQ and SET are historical legacies from the original Lisp
languages that directly preceeded Common Lisp. SETF was new,
and was not used by the large body of existing programs for
which Common Lisp was designed to be portability target.

Note that SETQ is "a special form of SET", and so if you are
writing code that sets variables by calling the SET function,
you might want nearby code to employ the contrasting SETQ function.
But people don't really write SET anymore, because most variables
are lexical variables. And if you want to set a value cell,
nowadays you can SETF the SYMBOL-VALUE instead.

The only reason to write SETQ anymore is to highlight the fact that
you're setting a variable. This is a pretty weak argument. since it's
syntactically obvious that you're setting a variable. (And it's not
even guaranteed to be true, anyway, in the face of symbol macros.)

If we were inventing a new Lisp dialect today, willing to break
compatability with old Lisp programs, we would get rid of SET, SETQ,
and SETF -- and we would just have a form named SET (meaning SETF).

When hacking variables, I always write SETQ rather than SETF.
Mostly this is because I am a geezer who has been writing it
that way since before Common Lisp was a twinkle in the eye.
And when I think about modernizing my personal style to use SETF
instead of SETQ, I get as annoyed by the letter "F" as by the "Q".
I am then tempted to start writing in my own dialect, using the
package system to make "SET" mean "SETF".
And I can't quite bring myself to do that.

Yet.
-------

And from this forum discussing SETF vs SETQ:
http://www.cocreateusers.org/forum/show ... php?t=5109

Thanks Calus & Jones...
Both of your replies are quite useful to me... in short i have concluded

Setq=set Quantity
Setf= set Field
--------

Maybe it should be SETI - for SET (list) Item...

That would then make it an out of this world function ;)

http://en.wikipedia.org/wiki/SETI

-- xytroxon

[cormullion]

Is newlispdoc having some problems? - see the definition of Time in http://unbalanced-parentheses.nfshost.c ... s.lsp.html - it says 'true' lots of times...I don't think it did that before.

[Lutz]

I uploaded newlispdoc version 1.8 which fixes a problem of "true" appearing before mutliple @syntax lines in documentation.

http://newlisp.nfshost.com/downloads/newlispdoc-18

ps: works also with 9.3 on nfshost

[xytroxon]

Errata:

"manual_frame.html"

------------
List processing, flow control and integer arithmetic
...
set-ref-all
setf
setf <<<<<<<<<<<<< remove second setf link
silent
...
------------
String and conversion functions
...
name
nth
nth-set <<<<<<<<<<<<< remove nth-set link
pack
...
select
set-nth <<<<<<<<<<<<< remove set-nth link
slice
...
------------
Array functions
...
nth
nth-set <<<<<<<<<<<<< remove nth-set link
rest
set-nth <<<<<<<<<<<<< remove set-nth link
transpose
...

[Lutz]

Thanks for all the corrections and errata.

Regarding setq/setf: In newLISP 9.9.2 'setq' and 'setf' point to the same internal function. I still think the normal 'set' is needed for things like this:

Code: Select all

> (set 'l '(a b c))
(a b c)
> (set (first l) 123)
123
> a
123

with only setf/setq you would have to do:

Code: Select all

> (setf (eval (first l)) 456)
456
> a
456
> 

using the extra 'eval'.

'set setq setf' is one of those things you can discuss eternally without finding agreement ;-) Perhaps its not a bad idea to have several of them. As syntax is almost non-existent in Lisp, the different versions help to communicate your intentions. In the manual I also recommend to use 'setq' for variables and 'setf' for references.

You know, you always can do: (constant 'set setf)
Or in order not to confuse everybody else: (constant 'assign setf)
Or how about this: (constant '<- setf)

The way newLISP works, all of these are equally fast.

ps: for several versions newLISP also had 'set!' but nobody ever used it.

[newBert]

As for me I like the way NewLISP is developing. Some of my scripts, after all quickly modified, are really faster ! I realized this with my own eyes, without the help of any benchmarking :-)

It's sure that all three set(s) (set, setq, setf) must be preserved, at least for readability and expressiveness, but not only ... as Lutz explained.

Thank you for these changes, that also gave rise to review and optimize some of my scripts ;-)

[Kazimir Majorinc]

Kazimir:

Any of these solutions must be able to recover the original variable environment without cell leaks and multiple memory-object references in case of early functions return when errors occur, or when 'catch' and 'throw' is involved. Only symbols and contexts are exempted.

If you introduce new kind of function, and these functions do not cooperate well with some elements of language, like catch and throw - so what? Those who need catch and throw lose nothing, those who do not need catch and throw get something. If in the future you find the way to extend catch and throw on new functions, it is additional bonus.

It is usually that way with generalizations. They have the price.