newlispfanclub.alh.net

Posted: **Thu Nov 14, 2013 3:19 am**

.. with a simple patch for a first solution and further ideas for possibly better ones...
[4. update] Removed some stupid patch ideas.
[3. update]
Solution for 10.5.4: viewtopic.php?f=16&t=4416#p21792
Solution for later versions: viewtopic.php?f=16&t=4416#p21794
'[HowTo]' added to title.
[2. update] Changed title away from feature request: some discussion revealed a very good solution (see posts thereafter); and the patches are not a good idea compared with it.
[update] Better patch - IMHO - in self-reply to this post.

Doing more and more scripting with newLISP, it's important to me to have interactive debugging facilities.
Furthermore it's good to have some separation between CLI args targeted to the interpreter and others targeted to a script interpreted by it.

Assumed there is the following code:

Code: Select all

#!/usr/bin/newlisp

(module "getopts.lsp")

(shortopt "p" (println "players: " getopts:arg) "int" "num of players")
(shortopt "h" (println (getopts:usage)) nil "usage")

(getopts (2 (main-args)))

;; some code here: if something goes wrong, or just for interactive development
;; no (exit) could be a good idea here...
(exit)

This gives:

Code: Select all

sr@free:~/NewLisp$ ./game.lsp -p 10
players: 10
sr@free:~/NewLisp$

So far so good.

But if something goes wrong, or we just want to inspect some symbols before (exit), commenting out (exit) does not work:

Code: Select all

#!/usr/bin/newlisp

(module "getopts.lsp")

(shortopt "p" (println "players: " getopts:arg) "int" "num of players")
(shortopt "h" (println (getopts:usage)) nil "usage")

(getopts (2 (main-args)))

;; some code here: if something goes wrong, or just for interactive development
;; no (exit) could be a good idea here...
;(exit)

We get:

Code: Select all

sr@free:~/NewLisp$ ./game.lsp -p 10
players: 10
newLISP server setup on 10 failed.
sr@free:~/NewLisp$

instead of entering newLISP interpreter.

[4. update]: Removed some stupid patch ideas.

What do you think?

Posted: **Thu Nov 14, 2013 7:24 am**

Posted: **Thu Nov 14, 2013 2:57 pm**

If you define and error handler, you can skip command line processing and are left in interactive mode:

Code: Select all

#!/usr/bin/newlisp

(error-event (fn () (println (last-error))))

(module "getopts.lsp")

;(shortopt "-" (println "'-' opt") nil "stops parsing CLI opts")
(shortopt "p" (println "players: " getopts:arg) "int" "num of players")
(shortopt "h" (println (getopts:usage)) nil "usage")

(getopts (2 (main-args)))

(throw-error "debugging mode")

(exit) ; <-- is never reached

The throw-error statement will put you in interactive mode:

Code: Select all

~> ./game -p 10
players: 10
(58 "ERR: user error : debugging mode")
newLISP v.10.5.4 64-bit on OSX IPv4/6 UTF-8 libffi, options: newlisp -h

>

any other error in you code would also send you to interactive mode, but you can force it using throw-error

Posted: **Fri Nov 15, 2013 1:49 am**

Thanks for the hint: this is a solution. I even haven't thought about exceptions for this use case, though some assert macros of mine use them, too.
Though there is more to do for the developer than just to comment out (exit); for debugging this should be fine.

What about having some kind of non-error exception for this use case?
Because there needn't be an error, it's just to keep the interpreter off from parsing of command line args and entering it instead for an interactive session.

[4. update]: Removed some stuff about a bad idea of checking the exec bit of interpreter loaded scripts for stopping interpreter CL parsing.

Posted: **Fri Nov 15, 2013 9:37 pm**

To summarize: the only reason for newLISP doing unwanted command line processing, is programmer error.

There is no other normal use case for a built-in mechanism to stop processing the command line. If processing occurs and is unwanted, it always means either a code error or a missing exit statement during code development and for debugging. Somehow the program lost control and finishes at the top level without exiting, causing newLISP to continue processing command line parameters.

To help debugging in future versions, a simple (reset) will also interrupt command line processing and enter interactive mode. No error handler will be required. So this will do it too:

Code: Select all

#!/usr/bin/newlisp

(module "getopts.lsp")

(shortopt "p" (println "players: " getopts:arg) "int" "num of players")
(shortopt "h" (println (getopts:usage)) nil "usage")

(getopts (2 (main-args)))

(reset) ; stop processing - now also including command line parsing

(exit) ; <--- never gets here because reset interrupts processing

Ps: A scripting language checking the execution bit would be very unusual, no scripting language does that. The exe bit should only be required by the OS/shell for direct execution of scripts via the shebang #! mechanism.

Posted: **Fri Nov 15, 2013 11:54 pm**

Lutz wrote:To summarize: the only reason for newLISP doing unwanted command line processing, is programmer error.

This is not the only reason: there is also the interest in interactive development: no error, but wishing to inspect things from inside of the interpreter after the script code has run.

Lutz wrote: There is no other normal use case for a built-in mechanism to stop processing the command line. If processing occurs and is unwanted, it always means either a code error or a missing exit statement during code development and for debugging. Somehow the program lost control and finishes at the top level without exiting, causing newLISP to continue processing command line parameters.

See above.

Lutz wrote: To help debugging in future versions, a simple (reset) will also interrupt command line processing and enter interactive mode. No error handler will be required. So this will do it too:
Code: Select all
#!/usr/bin/newlisp

(module "getopts.lsp")

(shortopt "p" (println "players: " getopts:arg) "int" "num of players")
(shortopt "h" (println (getopts:usage)) nil "usage")

(getopts (2 (main-args)))

(reset) ; stop processing - now also including command line parsing

(exit) ; <--- never gets here because reset interrupts processing

I like this solution!
(reset) with error handler already does exactly what I want: the script runs and afterwards it's possible to inspect symbols and their contents in the interpreter.

Without error handler there are problems:

without (reset) it hangs at an unknown opt - e.g. '-c' - this is my use case -,
with (reset) the interpreter exits.

Thanks for your will - and work - to improve this!
After this change it's just a change of (exit) to (reset) and vice-versa (just to change one semicolon for commenting out/in (reset)).

Lutz wrote: Ps: A scripting language checking the execution bit would be very unusual, no scripting language does that.

OK: thanks for the info.

Moreover I think now, that the (reset) mechanism:

has lower overhead - no need to check file status after loading it -, and
it gives less suprises - with checking exec bit of the file, it depends from the position of it, if and when CL arg parsing stops (the (reset) mechanism is cristal clear in this respect: stop CL arg parsing, if developer explicitely says so).

Note: I have changed the title of my original post away from [FR} (feature request).

Posted: **Sat Nov 16, 2013 11:48 pm**

I wrote the getopts modules. I have difficulty understanding the issue.

"--" is a GNU standard. getopts is designed to do GNU standard command line parsing.

If an unknown option makes the program hang in the getopts module, that is a bug. Can you please post a short working sample of code so I can duplicate the problem. Unknown options should raise an error, not hang.

Scripts take arguments as part of the POSIX standard, and it is best not to mess with that. For instance, in some of my newlisp scripts I have to use the -m and -s options, to control the memory and stack usage of the interpeter, so it looks like this: #!/usr/bin/newlisp -m 1024 -s 1024

This happens on a script by script basis. When you write your script and set up the command line parsing, just take that into account.

Another issue with command line parsing is that different operating systems parse the command line differently, so you may need to start your argument processing at a different place if you are on Unix versus Windows.

Can you explain your issue a bit better hartrock?

Posted: **Mon Nov 18, 2013 4:21 am**

Ted you are right!
I haven't known this '--' behavior.

From 'man getopts':

Each parameter not starting with a `-', and not a required argument of
a previous option, is a non-option parameter. Each parameter after a
`--' parameter is always interpreted as a non-option parameter. If the
environment variable POSIXLY_CORRECT is set, or if the short option
string started with a `+', all remaining parameters are interpreted as
non-option parameters as soon as the first non-option parameter is
found.

Exactly this is what getopts does:

Code: Select all

#!/usr/bin/newlisp

(module "getopts.lsp")

(shortopt "a" (println "'a' opt") nil "works as expected")
(shortopt "h" (println (getopts:usage)) nil "usage")

;; Do *not* try this (action won't be triggered):
;; (shortopt "-" (println "'-' opt") nil "stops parsing CLI opts") ; Do *not* try this!

(getopts (2 (main-args)))

(println "After calling getopts.")
(exit)

Code: Select all

sr@free:~/NewLisp$ newlisp getopts_bug.lsp -a --
'a' opt
After calling getopts.
sr@free:~/NewLisp$ newlisp getopts_bug.lsp -- -a
After calling getopts.
sr@free:~/NewLisp$

In the first case '-a' triggers its action; in the second case it will be ignored.

Posted: **Mon Nov 18, 2013 4:27 am**

Looks like it does what is supposed to. Here, from the GNU getopt documentation:

http://www.gnu.org/software/libc/manual ... ing-Getopt

getopt has three ways to deal with options that follow non-options argv elements. The special argument ‘--’ forces in all cases the end of option scanning.

Posted: **Mon Nov 18, 2013 6:44 am**

TedWalther wrote: Can you explain your issue a bit better hartrock?

My original motivation for starting this has been twofold:

Being able to enter the interpreter instead of exit:
→ possible by reset or throw (don't know the differences so far) together with error handler (and starting with v10.5.5 reset even without error handler).
Having a separation between CL args for interpreter and script:
→ possible by correctly using exit or the former mechanism for entering the interpreter during development.

So this thread has changed to [HowTo].

Next is to cleanup it a bit to avoid confusion: most important, removing the stupid idea with using special opt '--' for separating CL args between interpreter and script (coming into mind without knowing first mechanism above and not knowing special semantics of '--')).
[4. update]: Cleanup done.

BTW: I have extended/patched getopts.lsp (injected from the outside after loading it) for having shortlongopt: this leads to a grouping of corresponding short and long opt in usage:

Code: Select all

sr@free:~/Ideas/Spiel$ ./gol -h
Usage: gol [options]
  -c, --cycles INT                  num of cycles
  -r, --rounds INT                  num of rounds per cycle
  -a, --players-at-all INT          players at all
  -g, --players-per-game INT        players per game
  -s, --start-balance INT           start balance for each player
  -l, --lower-risk-limit FLOAT      [opt] 0.0 <= FLOAT <= 1.0 (default: 0.0)
  -u, --upper-risk-limit FLOAT      [opt] 0.0 <= FLOAT <= 1.0 (default: 1.0)
  -d, --distribution-per-cycle FLOAT  [opt] 0.0 <= FLOAT <= 1.0 (default: 0.0) fraction of balance to be distributed
  -h, --help                        Print this help message

(printing could be improved further). May be you are interested.

Posted: **Mon Nov 18, 2013 8:43 am**

That looks like a good idea. Can you send the patch for review?

Posted: **Mon Nov 18, 2013 9:45 pm**

Hello Ted,

I've sent you an email with the code via the User Panel. Because it's my first mail here, and it stays in the Outbox - perhaps diminishing there, if you have seen it? - I think it cannot hurt to give you a hint.

Best regards,
Stephan

newlispfanclub.alh.net

[HowTo] stopping newlisp parsing of '#!'-script args ..

[HowTo] stopping newlisp parsing of '#!'-script args ..

[4. update] Removed.

Re: [FR][patches] stopping newlisp parsing of '#!'-script ar

Re: [FR][patches] stopping newlisp parsing of '#!'-script ar

Re: [FR][patches] stopping newlisp parsing of '#!'-script ar

Re: [FR][patches] stopping newlisp parsing of '#!'-script ar

Re: [HowTo] stopping newlisp parsing of '#!'-script args ..

working example for getopts problem

Re: [HowTo] stopping newlisp parsing of '#!'-script args ..

Re: [HowTo] stopping newlisp parsing of '#!'-script args ..

Re: [HowTo] stopping newlisp parsing of '#!'-script args ..

Re: [HowTo] stopping newlisp parsing of '#!'-script args ..