fexpr issues

Pondering the philosophy behind the language

fexpr issues

Postby Astrobe » Fri Feb 28, 2014 9:42 am

Consider this dumbed-down version of do-until:

Code: Select all
(module "macro.lsp")
(define-macro (DO-UNTIL test B)
           (do-while (not (eval test)) (eval B)))

(macro (DO-UNTIL* T B)
   (do-while (not T) B))


(println "test 1")
(define (foo k)
  (DO-UNTIL (<= k 0) (begin (println k) (dec k))))
(foo 3)

(println "test 2")
(define (foo* k)
   (DO-UNTIL* (<= k 0) (begin (println k) (dec k))))
(foo* 3)

(println "test 3")
(define (bar*)
  (let (k 0)
   (DO-UNTIL* (>= k (args 0)) (begin (println k) (inc k)))))
(bar* 3)

(println "test 4")
(define (bar)
  (let (k 0)
   (DO-UNTIL (>= k (args 0)) (begin (println k) (inc k)))))
(bar 3)


First problem: foo doesn't behave correctly under debug; doing (debug (foo 3)) doesn't do the same as (foo 3)

Second problem: bar doesn't work at all, because (args 0) is evaluated in the context fo the macro DO-UNTIL.

Those kind of problems makes it really tricky to use fexprs. On the other hand, rewrite macros provided by macro.lsp don't have these problems, and are also faster because the substitution is done at compile-time. They however increase the code size, but this can be mitigated by factoring out the "big parts" of macros in external functions.

macro.lsp notes that it increases the load-time of scripts. Is the impact significant? What about native and improved support for rewrite macros?
Astrobe
 
Posts: 39
Joined: Mon Jan 11, 2010 9:41 pm

Re: fexpr issues

Postby Lutz » Fri Feb 28, 2014 5:10 pm

First problem: foo doesn't behave correctly under debug; doing (debug (foo 3)) doesn't do the same as (foo 3)



I see (debug (foo 3)) behaving correctly doing the same thing as (foo 3). May be if you replace (println k) with (println “=====>” k), you can better follow the debug behavior. There is a lot of output from the debugger in-between the println expressions, so perhaps you just missed it visually, scrolling off the terminal screen.

Second problem: bar doesn't work at all, because (args 0) is evaluated in the context fo the macro DO-UNTIL.


fexprs are functions, so I would argue: the behavior is expected. args should always work in the context of the currently executing function and nested functions/fexprs should all keep their local versions of args, this makes code more readable too.

To your last question: “should macro be implemented natively”. I frequently have thought about this and keep looking into it. At least at the moment, I have not found a way to do this efficiently without increasing load speed substantially. Even when implemented natively, for every functor expression symbol parsed, you would have to do the lookup, if a reader-event is defined.
Lutz
 
Posts: 5279
Joined: Thu Sep 26, 2002 4:45 pm
Location: Pasadena, California

Re: fexpr issues

Postby Astrobe » Fri Feb 28, 2014 7:07 pm

May be if you replace (println k) with (println “=====>” k), you can better follow the debug behavior


Indeed I thought about this possibilty and did something like that. It seems I didn't put enough dashes in my println.

fexprs are functions, so I would argue: the behavior is expected. args should always work in the context of the currently executing function and nested functions/fexprs should all keep their local versions of args, this makes code more readable too.


The problem I see is that one can easily forget that some definition is actually a macro and pass it an expression that contains (args) or $args. This situation typically happens when one nests macros, id est when one uses macros inside macros.

Even when implemented natively, for every functor expression symbol parsed, you would have to do the lookup, if a reader-event is defined.


What about doing it like Forth? Macro symbols could have a special mark, that triggers their execution (after the arguments have been parsed) at compile-time.
Astrobe
 
Posts: 39
Joined: Mon Jan 11, 2010 9:41 pm

Re: fexpr issues

Postby TedWalther » Sat Mar 01, 2014 8:31 pm

You mean like , and ,@ and ` operators in Common Lisp?

Both Common Lisp and Scheme also support the backquote operator (known as quasiquote in Scheme), entered with the ` character (grave accent). This is almost the same as the plain quote, except it allows expressions to be evaluated and their values interpolated into a quoted list with the comma , unquote and comma-at ,@ splice operators. If the variable snue has the value (bar baz) then `(foo ,snue) evaluates to (foo (bar baz)), while `(foo ,@snue) evaluates to (foo bar baz). The backquote is most frequently used in defining macro expansions.[36][37]


If there are several layers of function calls, how do you know if you need to expand (args 0) with quasiquote or not?
Cavemen in bearskins invaded the ivory towers of Artificial Intelligence. Nine months later, they left with a baby named newLISP. The women of the ivory towers wept and wailed. "Abomination!" they cried.
TedWalther
 
Posts: 605
Joined: Mon Feb 05, 2007 1:04 am
Location: Abbotsford, BC

Re: fexpr issues

Postby Lutz » Sat Mar 01, 2014 10:03 pm

It would be like the current macro facility from the module macro.lisp but built into newLISP. The goal is to maintain the current macro-method but faster w/o impact on newLISP's speed in loading source code.

To get rid of back quotes and other special characters in LISP was/is one of newLISP's goals ;)

The reader-event function would stay as it has many other possibilities besides using it to implement expansion macros.

See also here: http://www.newlisp.org/downloads/newlis ... ader-event

Ps: I think Astrobe is talking about the way it it implemented, not the way it works for the programmer. When doing it natively, I could handle things at a lower level not possible with the current implementation as shown in the reader-event example.
Lutz
 
Posts: 5279
Joined: Thu Sep 26, 2002 4:45 pm
Location: Pasadena, California

Re: fexpr issues

Postby TedWalther » Sun Mar 02, 2014 8:32 am

I was referring to this statement by Astrobe:

The problem I see is that one can easily forget that some definition is actually a macro and pass it an expression that contains (args) or $args. This situation typically happens when one nests macros, id est when one uses macros inside macros.


It is a bit of a "gotcha" that fexprs are function calls and not just inline expansions.g Although it makes it nicer for debugging.
Cavemen in bearskins invaded the ivory towers of Artificial Intelligence. Nine months later, they left with a baby named newLISP. The women of the ivory towers wept and wailed. "Abomination!" they cried.
TedWalther
 
Posts: 605
Joined: Mon Feb 05, 2007 1:04 am
Location: Abbotsford, BC

Re: fexpr issues

Postby Lutz » Mon Mar 03, 2014 8:47 pm

The expansion macro function is now a built-in primitive :)

http://www.newlisp.org/downloads/develo ... 10.5.8.txt

and runs Astrobe's macro examples fine.
Lutz
 
Posts: 5279
Joined: Thu Sep 26, 2002 4:45 pm
Location: Pasadena, California

Re: fexpr issues

Postby TedWalther » Mon Mar 03, 2014 9:30 pm

Very nice. Did you lift the limitation that macro arguments must be uppercase?
Cavemen in bearskins invaded the ivory towers of Artificial Intelligence. Nine months later, they left with a baby named newLISP. The women of the ivory towers wept and wailed. "Abomination!" they cried.
TedWalther
 
Posts: 605
Joined: Mon Feb 05, 2007 1:04 am
Location: Abbotsford, BC

Re: fexpr issues

Postby Astrobe » Tue Mar 04, 2014 5:40 pm

Oh, nice!

Do I have to wait 10.5.8 to officially come out or can I grab it right now?
Astrobe
 
Posts: 39
Joined: Mon Jan 11, 2010 9:41 pm

Re: fexpr issues

Postby Lutz » Tue Mar 04, 2014 6:21 pm

You can grab a source package right now here:

http://www.newlisp.org/downloads/develo ... nprogress/

also check the new manual entry:

http://www.newlisp.org/downloads/develo ... html#macro

Ps: if you cannot compile yourself, I can compile for you an executable for either Windows, OSX or Ubuntu Linux for Windows and Ubunto specify UTF8 or non-UTF8. Let me know.
Lutz
 
Posts: 5279
Joined: Thu Sep 26, 2002 4:45 pm
Location: Pasadena, California

Re: fexpr issues

Postby rickyboy » Tue Mar 04, 2014 6:43 pm

Image
(λx. x x) (λx. x x)
rickyboy
 
Posts: 594
Joined: Fri Apr 08, 2005 7:13 pm
Location: Front Royal, Virginia

Re: fexpr issues

Postby TedWalther » Tue Mar 04, 2014 11:32 pm

Would it be hard to make delete work on a macro, if I wanted to reuse the symbol as a non-macro?
Cavemen in bearskins invaded the ivory towers of Artificial Intelligence. Nine months later, they left with a baby named newLISP. The women of the ivory towers wept and wailed. "Abomination!" they cried.
TedWalther
 
Posts: 605
Joined: Mon Feb 05, 2007 1:04 am
Location: Abbotsford, BC

Re: fexpr issues

Postby bairui » Wed Mar 05, 2014 7:48 am

Lutz, re: the manual entry for (macro ...)

1. The signature still has 'define' instead of 'macro'.
2. "But macro definitions cannot be repeated for the same symbol during the sane newLISP session." (sane)
bairui
 
Posts: 64
Joined: Sun May 06, 2012 2:04 am
Location: China

Re: fexpr issues

Postby Astrobe » Wed Mar 05, 2014 11:54 am

Also, I spotted by chance a misuse of define-macro in the "apply" entry: my-gcd is defined as a macro; but if you try (my-gcd 12 18 (+ 1 2 3)) one gets an error.
Astrobe
 
Posts: 39
Joined: Mon Jan 11, 2010 9:41 pm

Re: fexpr issues

Postby Lutz » Wed Mar 05, 2014 2:46 pm

Thanks to Barui and Astrobe. Spelling is fixed and the my_gcd define-macro now evaluates its arguments:

http://www.newlisp.org/downloads/develo ... nprogress/

There will be a development 10.5.8 release next week and probably an official v.10.6.0 in April.
Lutz
 
Posts: 5279
Joined: Thu Sep 26, 2002 4:45 pm
Location: Pasadena, California

Re: fexpr issues

Postby Astrobe » Wed Mar 05, 2014 5:26 pm

I know this is different from what I initially asked for, but...

if you remove lines from 4912 to 4921 (the ones that insert the expansion code), one gets interesting options for forging the replacement expression.
Like for instance this improved curry:

Code: Select all
(macro (curry*)
  (case (length $args)
    (2 (letex (F (args 0) A (args 1)) (fn(x) (F A x))))
    (3 (letex (F (args 0) A (args 1) B (args 2)) (fn(x) (F A B x))))))

(set 'K (curry* + 1))
> (lambda (x) (+ 1 x))
(set 'L (curry* + 1 2))
>(lambda (x) (+ 1 2 x))


(copied by hand from another screen; parens might be missing)

What I'm thinking is the functionality of "macro" already exists in a module, and the term "macro" can be confusing together with define-macro, so... What if we call our new feature "inline" (for instance), and we don't restrict it to the functionality of its ancestor?
Astrobe
 
Posts: 39
Joined: Mon Jan 11, 2010 9:41 pm

Re: fexpr issues

Postby Lutz » Wed Mar 05, 2014 9:15 pm

The module implementation of macro in macro.lsp added more than 100% of load-time for each macro defined. So when you had 10 macros defined, the whole load time of source files, would increase by more than 10 times.

The new native implementation has no measurable impact on load times, even when defining many macros.

You are still able to do, what you are showing in your post. The default expansion function in C in lines 4912 to 4921 can easily be overwritten once you have marked the symbol as a macro symbol:

Code: Select all
; let newLISP know that curry* contains a reader expansion function
> (macro (curry*))
(lambda-macro () (expand 'nil))

; now assign your own for that same symbol
(define curry* (lambda-macro ()
  (case (length $args)
    (2 (letex (F (args 0) A (args 1)) (fn(x) (F A x))))
    (3 (letex (F (args 0) A (args 1) B (args 2)) (fn(x) (F A B x)))))))

> (set 'K (curry* + 1))
(lambda (x) (+ 1 x))
> (set 'L (curry* + 1 2))
(lambda (x) (+ 1 2 x))
>


What the native implementation of macro gives us, is more reading speed. But we still have the possibility to play our own tricks. The real work - reader-time expansion - happens in lines 3505 to 3515.

You also could code your example without the technique shown above in one step like this:

Code: Select all
(macro (curry* F A B)
    (if B (lambda (x) (F A B x)) (lambda (x) (F A x))))

> (curry* + 1)
(lambda (x) (+ 1 x))
> (curry* + 1 2)
(lambda (x) (+ 1 2 x))
Lutz
 
Posts: 5279
Joined: Thu Sep 26, 2002 4:45 pm
Location: Pasadena, California

Re: fexpr issues

Postby Lutz » Fri Mar 07, 2014 8:03 pm

More explanations how the new macro works with map and apply and differences to define-macro:

http://www.newlisp.org/downloads/develo ... html#macro

Any corrections are appreciated.
Lutz
 
Posts: 5279
Joined: Thu Sep 26, 2002 4:45 pm
Location: Pasadena, California

Re: fexpr issues

Postby bairui » Sat Mar 08, 2014 8:53 am

Hi, Lutz,

In the last example box for (macro ...):

Code: Select all
(define-macro (fexpr-double X) i


Trailing i a vim oopsie? :-)
bairui
 
Posts: 64
Joined: Sun May 06, 2012 2:04 am
Location: China


Return to Whither newLISP?

Who is online

Users browsing this forum: No registered users and 1 guest

cron