Request and Response modules for newLISP

Jeff · Post by **Jeff** » Fri Feb 22, 2008 7:32 pm

As promised, I've finished the request and response modules for cgi programming. They are fairly comprehensive (for their purposes), but do not contain the equivalent of "put-page" from the newLISP cgi module and are not compatible with the newLISP cgi module.

One of my next projects is a template module that will replace that functionality.

Here are the links:

Article: http://www.artfulcode.net/articles/requ ... s-newlisp/
Files: http://www.artfulcode.net/projects/newl ... se-module/

cormullion · Post by **cormullion** » Sat Feb 23, 2008 4:35 pm

Cool! I'm going to try this out in the next few days. You may hear from me again... :)

cormullion · Post by **cormullion** » Tue Feb 26, 2008 4:48 pm

Hi again. Tell me, if I replace (CGI:get {string}) with (Request:get {string}), should that work?

I tried it and half of the time it worked, half of the time it didn't. More precisely, half of the methods always worked, and the others never did. I wonder whether this is because I've misused the Get and Post methods in my own script and the problem is only just appearing now.

Jeff · Post by **Jeff** » Tue Feb 26, 2008 4:58 pm

It would depend on your code. (Request:get "foo") should work, because that data is always available through (env "QUERY_STRING"). Post, however, should only work if no other script has read the input buffer first. So, if the official newLISP cgi module is loaded first, it will have the post data, making it inaccessible to the request module.

Also, I have set an arbitrary limit on the size of post data (for safety reasons). You can alter it at the top of the module.

What does your code look like? Is it running from apache on nfshost?

cormullion · Post by **cormullion** » Tue Feb 26, 2008 5:48 pm

I swapped out cgi.lsp and changed the max value first, so I don't think it's those . But I was running it locally using newlisp's webserver mode - newlisp -httpd - so perhaps that makes a difference.

A quick way of seeing the code is http://code.google.com/p/lambdapress/so ... /index.cgi but please remember that I'm not a programmer ... :)

Jeff · Post by **Jeff** » Tue Feb 26, 2008 6:37 pm

I don't know. I have not messed with the built-in http server. I tested this on apache running newlisp as a cgi.

That code is using the cgi.lsp module. I'm assuming that is pre-swapping. You can see all get and post params entered by printing (Request:get) and (Request:post) (without any arguments). Without a key, they output the entire assoc list.

cormullion · Post by **cormullion** » Tue Feb 26, 2008 8:13 pm

Thanks Jeff - I'll try that. Obviously the code on googlecode isn't the same as the version I tested with Request instead of CGI - I like to try stuff out locally out before posting... Trouble is, trying things locally with newlisp server is much easier than setting up Apache (although that's easy enough too).

Jeff · Post by **Jeff** » Tue Feb 26, 2008 8:47 pm

Corm,

I just tested it out. I've updated the template module to include a test page using the request and response modules. Here is the link:

http://www.artfulcode.net/media/release ... .2.tar.bz2

It has a simple form that sends both post and get data to the server, and I tested it against 9.32 in -httpd mode.

itistoday · Post by **itistoday** » Sun Mar 02, 2008 6:42 pm

Sorry, what was the reason for this? Advantages/disadvantages between this and newLISP's CGI module?

cormullion · Post by **cormullion** » Sun Mar 02, 2008 7:31 pm

I think there's something here: http://www.alh.net/newlisp/phpbb/viewtopic.php?t=2131

itistoday · Post by **itistoday** » Sun Mar 02, 2008 7:47 pm

cormullion wrote:I think there's something here: http://www.alh.net/newlisp/phpbb/viewtopic.php?t=2131

Scanned over that... still don't really get it... (I don't do much web development), is it that there's something bad about CGI's GET/POST retrieval/differentiation...?

Jeff · Post by **Jeff** » Sun Mar 02, 2008 10:14 pm

CGI is dangerous enough without knowing where a parameter got set. With the default cgi module, there is no way to distinguish whether a parameter was set via post or get, nor is there a way to determine, once loaded, if there was any post data at all. A common idiom is to check if post data exists. If it does, continue with validation. If not, return a 404 error code. This protects against automatic attacks.

Additionally, the default module does not do any sanity checking on the size of the data coming in from post, which could potentially hang the application completely.

My modules also provide a means to return various types of HTTP headers without having to print them yourself (i.e. 404 not found, 500 error, etc.)

Lutz · Post by **Lutz** » Sun Mar 02, 2008 11:40 pm

When using the Apache web-server the request method and content length can be retrieved by checking the environment variables set by Apache. The check can be done before loading cgi.lsp:

Code: Select all

(env "REQUEST_METHOD") => the string "GET" or "POST"

(int (env "CONTENT_LENGTH")) => the number of bytes in the post request

In both cases 'env' returns a string, so 'int' should be used to convert the content length. CONTENT_LENGTH is only present when when the REQUEST_METHOD is POST. In case of the GET method the environment variable QUERY_STRING can be inspected.

Jeff · Post by **Jeff** » Mon Mar 03, 2008 1:11 pm

Yes, but how can you verify that parameter "foo" came from POST data?