A new web-based library for newLISP

Notices and updates
Locked
Jeff
Posts: 604
Joined: Sat Apr 07, 2007 2:23 pm
Location: Ohio
Contact:

A new web-based library for newLISP

Post by Jeff »

I've written a replacement CGI library, which can optionally work side-by-side with the official CGI module, with some caveats (see the docs for details.)

It has extensive support for URL-encoding, entities, and sessions, as well as all of the functionality of the CGI module.

http://www.artfulcode.net/articles/a-be ... b-library/
Jeff
=====
Old programmers don't die. They just parse on...

Artful code

cormullion
Posts: 2038
Joined: Tue Nov 29, 2005 8:28 pm
Location: latiitude 50N longitude 3W
Contact:

Post by cormullion »

Looks great, Jeff, thanks... I'll be looking to try this out soon.

I was expecting "#!/usr/bin/env newlisp" - is it more portable to use "#!/usr/bin/newlisp" ?

Is it just for newlisp v 10? And all versions of v 10.0.* ?

Jeff
Posts: 604
Joined: Sat Apr 07, 2007 2:23 pm
Location: Ohio
Contact:

Post by Jeff »

I rarely use /usr/bin/env. In fact, it really doesn't need a preamble because it is not a standalone app. I had that in there because I was testing it as a cgi.

As to the nl version, it should probably be 10, since it uses setf internally.
Jeff
=====
Old programmers don't die. They just parse on...

Artful code

Jeff
Posts: 604
Joined: Sat Apr 07, 2007 2:23 pm
Location: Ohio
Contact:

Post by Jeff »

I also added a tutorial on how to write your own session storage module:

http://www.artfulcode.net/articles/cust ... wlisp-web/
Jeff
=====
Old programmers don't die. They just parse on...

Artful code

TedWalther
Posts: 608
Joined: Mon Feb 05, 2007 1:04 am
Location: Abbotsford, BC
Contact:

Post by TedWalther »

I bet this is related to your recent re-release of the JSON module. You released your module at just the right time, I'm migrating a legacy PHP codebase to newLISP.

I'm wondering, when you parse the POST data, do you check the content type? There are two or three content types that should be url-decoded, but anything else should be left as is, and treated as a stream of raw data.

Does your module handle this?

If the POST data is an XML stream, for instance, what do I do when using your module?

Ted

TedWalther
Posts: 608
Joined: Mon Feb 05, 2007 1:04 am
Location: Abbotsford, BC
Contact:

Post by TedWalther »

Jeff wrote:I also added a tutorial on how to write your own session storage module:

http://www.artfulcode.net/articles/cust ... wlisp-web/
I was just about to post that you were wrong about MySQL being best supported in newLISP, then realized I haven't added in the support for auto-converting query return types (int, float, etc). Until that is done, I'll hold my peace. But apart from that, I think the Postgres support in newLISP is pretty damn good. :)

TedWalther
Posts: 608
Joined: Mon Feb 05, 2007 1:04 am
Location: Abbotsford, BC
Contact:

Post by TedWalther »

Why do you use if-not instead of unless?

TedWalther
Posts: 608
Joined: Mon Feb 05, 2007 1:04 am
Location: Abbotsford, BC
Contact:

Post by TedWalther »

Regarding the MIME type of the POST data, I dug up some old code of mine to see what is happening. Here are the results:

If CONTENT_TYPE === application/x-www-form-urlencoded, then we url-decode.

If CONTENT_TYPE === multipart/form-data; then we have to do some mime decoding.

Once the MIME decoding is done, I don't know if we have to url-decode, presumably if one of the mime parts has a content type of x-www-form-urlencoded, then we decode it.

At all other times, the POST should be nil I guess, or at least read into a variable of some sort.

Ted

TedWalther
Posts: 608
Joined: Mon Feb 05, 2007 1:04 am
Location: Abbotsford, BC
Contact:

Post by TedWalther »

In the cookies function, there should also be a secure-only parameter after the http-only parameter. The secure option makes the cookies only be returned by the browser over an SSL connection.

cormullion
Posts: 2038
Joined: Tue Nov 29, 2005 8:28 pm
Location: latiitude 50N longitude 3W
Contact:

Post by cormullion »

TedWalther wrote:Why do you use if-not instead of unless?
They're slightly different. unless has no else part. From what I've seen, he's used mostly if-not with else clauses, but only once without one.

Jeff
Posts: 604
Joined: Sat Apr 07, 2007 2:23 pm
Location: Ohio
Contact:

Post by Jeff »

Ted -

Sorry for taking so long to respond. I was in meetings much of yesterday and am running a bit behind.

I will work on adding the secure cookie functionality and MIME parsing of the POST body, but it may be a little while. Alas, they make me write actual code for my paycheck, like I don't have other things to attend to :)

In the meantime, if you have a patch or some ready-made code for parsing a MIME-encoded POST body, I would love to take a look at it.
Jeff
=====
Old programmers don't die. They just parse on...

Artful code

Jeff
Posts: 604
Joined: Sat Apr 07, 2007 2:23 pm
Location: Ohio
Contact:

Post by Jeff »

In trying to parse the MIME content, I'm having some difficulty reading the entirety of the POST data. Or perhaps outputting the entirety of the binary POST content. The problem appears to be that the binary may contain null values, and that write-buffer and read-buffer both halt when encountering nulls. Any ideas on how to work around this?
Jeff
=====
Old programmers don't die. They just parse on...

Artful code

TedWalther
Posts: 608
Joined: Mon Feb 05, 2007 1:04 am
Location: Abbotsford, BC
Contact:

Post by TedWalther »

Hi Jeff, I've been coding for a paycheck too, sorry I haven't had time to help out with this. I can definitely see with your improvements, I'd like to see your cgi module be part of the base distro.

As for the 0 value bytes in the stream, we'll have to ask Lutz about this one. I know newlisp strings can have 0 value bytes, and there probably is a read- function that will allow it.

Just reading the description of read-buffer, it looks like a BUG in read-buffer, and it also seems like an unecessary limitation that the arg str-wait can't contain a 0 byte either.

To get it working right now, I suppose (while (setq c (read-char 0)) ...) could be used.

Lutz
Posts: 5289
Joined: Thu Sep 26, 2002 4:45 pm
Location: Pasadena, California
Contact:

Post by Lutz »

'read-buffer' and 'write-buffer' both handle 0 bytes and so do almost all other functions handing string buffers. Here is a demo and source for transferring files with HTTP POST:

http://www.newlisp.org/index.cgi?page=F ... oad_Script

There are few functions when used on binary content which will stop at zeros, i.e. 'string', 'print' and 'println'.

I have also used HTTP PUT to transfer binary content.

ps: there is a limitation only when using wait-string syntax for the wait-string itself

TedWalther
Posts: 608
Joined: Mon Feb 05, 2007 1:04 am
Location: Abbotsford, BC
Contact:

Post by TedWalther »

Jeff wrote:In trying to parse the MIME content, I'm having some difficulty reading the entirety of the POST data. Or perhaps outputting the entirety of the binary POST content. The problem appears to be that the binary may contain null values, and that write-buffer and read-buffer both halt when encountering nulls. Any ideas on how to work around this?
Lutz says read-buffer and write-buffer are working, can you give us a test case and localize it to a line of code?

Ted

Jeff
Posts: 604
Joined: Sat Apr 07, 2007 2:23 pm
Location: Ohio
Contact:

Post by Jeff »

Sure. Here is the code that reads the POST data:

Code: Select all

  (let ((post "") (buffer ""))
    (while (read-buffer (device) buffer POST_LIMIT)
      (write-buffer post buffer))
I have tried various POST_LIMIT sizes, although that should not affect the outcome.

I always end up with a smaller number of bytes than (peek (device)) reports in the buffer.
Jeff
=====
Old programmers don't die. They just parse on...

Artful code

Lutz
Posts: 5289
Joined: Thu Sep 26, 2002 4:45 pm
Location: Pasadena, California
Contact:

Post by Lutz »

Strange ... that is exactly what update.cgi does too, and I am not even using the peek, which is probably better. My script is fed by Apache webserver (probably 1.3x) and either Firefox or Safari on the client side. I wonder if those parameters matter (they shouldn't).

hilti
Posts: 140
Joined: Sun Apr 19, 2009 10:09 pm
Location: Hannover, Germany
Contact:

Dragonfly

Post by hilti »

Hi Jeff,

I'm very excited about Your library, so I started to write an extension called Dragonfly. It's going to be a small framework for web applications. I'm just learning the way newLISP works and posted my code thoughts into Google Code

http://code.google.com/p/dragonfly-newlisp/

But beware: it's really a version 0.01!

Best Regards from Germany
Marc

hilti
Posts: 140
Joined: Sun Apr 19, 2009 10:09 pm
Location: Hannover, Germany
Contact:

Post by hilti »

I just opened a blog about the Dragonfly web framework and updated the Roadmap on Google Code. You'll find the blog here http://dragonfly-newlisp.blogspot.com/

And Google Analytics told me, that there were the first visitors :)

cormullion
Posts: 2038
Joined: Tue Nov 29, 2005 8:28 pm
Location: latiitude 50N longitude 3W
Contact:

Post by cormullion »

Cool, I'm downloading the zip file... :)

hilti
Posts: 140
Joined: Sun Apr 19, 2009 10:09 pm
Location: Hannover, Germany
Contact:

Post by hilti »

Dragonfly Version 0.03 is available in Google Code, including a HTML table generator and some small updates to debugging information. See this blog post how it works.

http://dragonfly-newlisp.blogspot.com/2 ... tions.html

What do You think?

Best Regards
Marc

Locked