Page 1 of 1

fexpr issues

Posted: Fri Feb 28, 2014 9:42 am
by Astrobe
Consider this dumbed-down version of do-until:

Code: Select all

(module "macro.lsp")
(define-macro (DO-UNTIL test B)
			  (do-while (not (eval test)) (eval B)))

(macro (DO-UNTIL* T B)
	(do-while (not T) B))


(println "test 1")
(define (foo k)
  (DO-UNTIL (<= k 0) (begin (println k) (dec k))))
(foo 3)

(println "test 2")
(define (foo* k) 
	(DO-UNTIL* (<= k 0) (begin (println k) (dec k))))
(foo* 3)

(println "test 3")
(define (bar*)
  (let (k 0)
	(DO-UNTIL* (>= k (args 0)) (begin (println k) (inc k)))))
(bar* 3)

(println "test 4")
(define (bar)
  (let (k 0)
	(DO-UNTIL (>= k (args 0)) (begin (println k) (inc k)))))
(bar 3)
First problem: foo doesn't behave correctly under debug; doing (debug (foo 3)) doesn't do the same as (foo 3)

Second problem: bar doesn't work at all, because (args 0) is evaluated in the context fo the macro DO-UNTIL.

Those kind of problems makes it really tricky to use fexprs. On the other hand, rewrite macros provided by macro.lsp don't have these problems, and are also faster because the substitution is done at compile-time. They however increase the code size, but this can be mitigated by factoring out the "big parts" of macros in external functions.

macro.lsp notes that it increases the load-time of scripts. Is the impact significant? What about native and improved support for rewrite macros?

Re: fexpr issues

Posted: Fri Feb 28, 2014 5:10 pm
by Lutz
First problem: foo doesn't behave correctly under debug; doing (debug (foo 3)) doesn't do the same as (foo 3)

I see (debug (foo 3)) behaving correctly doing the same thing as (foo 3). May be if you replace (println k) with (println “=====>” k), you can better follow the debug behavior. There is a lot of output from the debugger in-between the println expressions, so perhaps you just missed it visually, scrolling off the terminal screen.
Second problem: bar doesn't work at all, because (args 0) is evaluated in the context fo the macro DO-UNTIL.
fexprs are functions, so I would argue: the behavior is expected. args should always work in the context of the currently executing function and nested functions/fexprs should all keep their local versions of args, this makes code more readable too.

To your last question: “should macro be implemented natively”. I frequently have thought about this and keep looking into it. At least at the moment, I have not found a way to do this efficiently without increasing load speed substantially. Even when implemented natively, for every functor expression symbol parsed, you would have to do the lookup, if a reader-event is defined.

Re: fexpr issues

Posted: Fri Feb 28, 2014 7:07 pm
by Astrobe
May be if you replace (println k) with (println “=====>” k), you can better follow the debug behavior
Indeed I thought about this possibilty and did something like that. It seems I didn't put enough dashes in my println.
fexprs are functions, so I would argue: the behavior is expected. args should always work in the context of the currently executing function and nested functions/fexprs should all keep their local versions of args, this makes code more readable too.
The problem I see is that one can easily forget that some definition is actually a macro and pass it an expression that contains (args) or $args. This situation typically happens when one nests macros, id est when one uses macros inside macros.
Even when implemented natively, for every functor expression symbol parsed, you would have to do the lookup, if a reader-event is defined.
What about doing it like Forth? Macro symbols could have a special mark, that triggers their execution (after the arguments have been parsed) at compile-time.

Re: fexpr issues

Posted: Sat Mar 01, 2014 8:31 pm
by TedWalther
You mean like , and ,@ and ` operators in Common Lisp?
Both Common Lisp and Scheme also support the backquote operator (known as quasiquote in Scheme), entered with the ` character (grave accent). This is almost the same as the plain quote, except it allows expressions to be evaluated and their values interpolated into a quoted list with the comma , unquote and comma-at ,@ splice operators. If the variable snue has the value (bar baz) then `(foo ,snue) evaluates to (foo (bar baz)), while `(foo ,@snue) evaluates to (foo bar baz). The backquote is most frequently used in defining macro expansions.[36][37]
If there are several layers of function calls, how do you know if you need to expand (args 0) with quasiquote or not?

Re: fexpr issues

Posted: Sat Mar 01, 2014 10:03 pm
by Lutz
It would be like the current macro facility from the module macro.lisp but built into newLISP. The goal is to maintain the current macro-method but faster w/o impact on newLISP's speed in loading source code.

To get rid of back quotes and other special characters in LISP was/is one of newLISP's goals ;)

The reader-event function would stay as it has many other possibilities besides using it to implement expansion macros.

See also here: http://www.newlisp.org/downloads/newlis ... ader-event

Ps: I think Astrobe is talking about the way it it implemented, not the way it works for the programmer. When doing it natively, I could handle things at a lower level not possible with the current implementation as shown in the reader-event example.

Re: fexpr issues

Posted: Sun Mar 02, 2014 8:32 am
by TedWalther
I was referring to this statement by Astrobe:
The problem I see is that one can easily forget that some definition is actually a macro and pass it an expression that contains (args) or $args. This situation typically happens when one nests macros, id est when one uses macros inside macros.
It is a bit of a "gotcha" that fexprs are function calls and not just inline expansions.g Although it makes it nicer for debugging.

Re: fexpr issues

Posted: Mon Mar 03, 2014 8:47 pm
by Lutz
The expansion macro function is now a built-in primitive :)

http://www.newlisp.org/downloads/develo ... 10.5.8.txt

and runs Astrobe's macro examples fine.

Re: fexpr issues

Posted: Mon Mar 03, 2014 9:30 pm
by TedWalther
Very nice. Did you lift the limitation that macro arguments must be uppercase?

Re: fexpr issues

Posted: Tue Mar 04, 2014 5:40 pm
by Astrobe
Oh, nice!

Do I have to wait 10.5.8 to officially come out or can I grab it right now?

Re: fexpr issues

Posted: Tue Mar 04, 2014 6:21 pm
by Lutz
You can grab a source package right now here:

http://www.newlisp.org/downloads/develo ... nprogress/

also check the new manual entry:

http://www.newlisp.org/downloads/develo ... html#macro

Ps: if you cannot compile yourself, I can compile for you an executable for either Windows, OSX or Ubuntu Linux for Windows and Ubunto specify UTF8 or non-UTF8. Let me know.

Re: fexpr issues

Posted: Tue Mar 04, 2014 6:43 pm
by rickyboy
Image

Re: fexpr issues

Posted: Tue Mar 04, 2014 11:32 pm
by TedWalther
Would it be hard to make delete work on a macro, if I wanted to reuse the symbol as a non-macro?

Re: fexpr issues

Posted: Wed Mar 05, 2014 7:48 am
by bairui
Lutz, re: the manual entry for (macro ...)

1. The signature still has 'define' instead of 'macro'.
2. "But macro definitions cannot be repeated for the same symbol during the sane newLISP session." (sane)

Re: fexpr issues

Posted: Wed Mar 05, 2014 11:54 am
by Astrobe
Also, I spotted by chance a misuse of define-macro in the "apply" entry: my-gcd is defined as a macro; but if you try (my-gcd 12 18 (+ 1 2 3)) one gets an error.

Re: fexpr issues

Posted: Wed Mar 05, 2014 2:46 pm
by Lutz
Thanks to Barui and Astrobe. Spelling is fixed and the my_gcd define-macro now evaluates its arguments:

http://www.newlisp.org/downloads/develo ... nprogress/

There will be a development 10.5.8 release next week and probably an official v.10.6.0 in April.

Re: fexpr issues

Posted: Wed Mar 05, 2014 5:26 pm
by Astrobe
I know this is different from what I initially asked for, but...

if you remove lines from 4912 to 4921 (the ones that insert the expansion code), one gets interesting options for forging the replacement expression.
Like for instance this improved curry:

Code: Select all

(macro (curry*)
  (case (length $args)
    (2 (letex (F (args 0) A (args 1)) (fn(x) (F A x))))
    (3 (letex (F (args 0) A (args 1) B (args 2)) (fn(x) (F A B x))))))

(set 'K (curry* + 1))
> (lambda (x) (+ 1 x))
(set 'L (curry* + 1 2))
>(lambda (x) (+ 1 2 x))
(copied by hand from another screen; parens might be missing)

What I'm thinking is the functionality of "macro" already exists in a module, and the term "macro" can be confusing together with define-macro, so... What if we call our new feature "inline" (for instance), and we don't restrict it to the functionality of its ancestor?

Re: fexpr issues

Posted: Wed Mar 05, 2014 9:15 pm
by Lutz
The module implementation of macro in macro.lsp added more than 100% of load-time for each macro defined. So when you had 10 macros defined, the whole load time of source files, would increase by more than 10 times.

The new native implementation has no measurable impact on load times, even when defining many macros.

You are still able to do, what you are showing in your post. The default expansion function in C in lines 4912 to 4921 can easily be overwritten once you have marked the symbol as a macro symbol:

Code: Select all

; let newLISP know that curry* contains a reader expansion function
> (macro (curry*))
(lambda-macro () (expand 'nil))

; now assign your own for that same symbol
(define curry* (lambda-macro ()
  (case (length $args)
    (2 (letex (F (args 0) A (args 1)) (fn(x) (F A x))))
    (3 (letex (F (args 0) A (args 1) B (args 2)) (fn(x) (F A B x)))))))

> (set 'K (curry* + 1))
(lambda (x) (+ 1 x))
> (set 'L (curry* + 1 2))
(lambda (x) (+ 1 2 x))
> 
What the native implementation of macro gives us, is more reading speed. But we still have the possibility to play our own tricks. The real work - reader-time expansion - happens in lines 3505 to 3515.

You also could code your example without the technique shown above in one step like this:

Code: Select all

(macro (curry* F A B)
    (if B (lambda (x) (F A B x)) (lambda (x) (F A x))))

> (curry* + 1)
(lambda (x) (+ 1 x))
> (curry* + 1 2)
(lambda (x) (+ 1 2 x))

Re: fexpr issues

Posted: Fri Mar 07, 2014 8:03 pm
by Lutz
More explanations how the new macro works with map and apply and differences to define-macro:

http://www.newlisp.org/downloads/develo ... html#macro

Any corrections are appreciated.

Re: fexpr issues

Posted: Sat Mar 08, 2014 8:53 am
by bairui
Hi, Lutz,

In the last example box for (macro ...):

Code: Select all

(define-macro (fexpr-double X) i
Trailing i a vim oopsie? :-)