Truncation problem with newlisp web server (-httpd).

oofoe · Post by **oofoe** » Mon Sep 03, 2007 1:18 pm

Hi,

If I start newlisp in web server mode (with "-httpd -d 80") and use the following code, I occassionally get a blank page. It's sort of like newlisp is ignoring the return value of httpd-conf and trying to send a page which doesn't exist. Here's what my httpd-conf looks like at the moment:

Code: Select all

(define (send text)
  (print "HTTP/1.0 200 OK\r\n")
  (print (format "Server: newLISP v.%d (%s)\r\n" (sys-info -1) ostype))
  (print (format "Content-length: %d\r\n" (length text)))
  (print "Content-type: text/html\r\n\r\n")
  (print text))

(define (template text)
  "( text -- nil) Print out filled in template."

  (send (replace "%content%" (read-file  "default.html") text))
  nil ; As specified in CodePatterns.html#distributed
  )

(define (httpd-conf path query)
  (if (= 0 (length path)) (app:home)
    path
    )
)

The idea is that a custom home page is displayed. Anything else (image requests, etc.) is sent as normal. The worst part about it is that it almost works... One of every four (or so) requests fails with a blank page. It may have something to do with request speed since I've tried manually accessing with telnet and it works every time.

Any ideas? Very frustrating.

Thanks!

Lutz · Post by **Lutz** » Mon Sep 03, 2007 4:00 pm

Are you sure you get the right return value from http-conf? What is (app:home)? When http-conf returns nil then this would result in an empty page, the httpd mode would just return without executing the request/query.

For debugging purpose I would put in print statements:

Code: Select all

(define (httpd-conf path query) 
  (println path ":" query)
  (if (= 0 (length path)) 
    (begin
      (set 'page (app:home))
      (println page)
      page)
    (begin
      (println path)
      path) 
    ) 
)

Lutz

ps: what browser are you running? is it IE-7?

oofoe · Post by **oofoe** » Mon Sep 03, 2007 7:57 pm

Hi,

Sorry to leave the (app:home) in there. That just prints the home page content using the send and template function. I modified your httpd-conf as so:

Code: Select all

(define (httpd-conf path query)
  (println path ":" query) ; Delete this line to see anything on IE.
  (if (= 0 (length path))
    (begin
      (set 'page (send (template "This is a simple test.")))
      (println "Page is " page)
      page)
    (begin
      (println "Path is " path)
      path)
    )
)

When I run it, I usually get a result that looks like this:

Code: Select all

:
HTTP/1.0 200 OK
Server: newLISP v.6 (Win32)
Content-length: 169
Content-type: text/html

<DOCTYPE>
<html>
  <head>
    <title>Test</title>
  </head>

  <body>
    This is a simple test.
  </body>
</html>
HTTP/1.0 200 OK
Server: newLISP v.6 (Win32)
Content-length: 0
Content-type: text/html

nilPage is nil

So it seems that NewLisp is ignoring the nil returned by send and running its page processing anyway. It's very wierd. If I telnet to port 80, I get the same thing. I have also tried it in IE 6.0 (don't have 7 installed) and get variations of the same thing. Normally, I use FireFox 1.5 or 2.0.

I'm running NewLisp on Windows XP SP2, serving locally on port 80.

Thanks!

Lutz · Post by **Lutz** » Mon Sep 03, 2007 9:02 pm

I don't see how your (send text) function can return nil. It will return, what the (print text) statement evaluates too, which is the last thing printed, which is text not nil.

Your (send text) function (and httpd-conf) must return nil in order to stop further processing.

Lutz

oofoe · Post by **oofoe** » Mon Sep 03, 2007 10:10 pm

Hi,

I don't see how your (send text) function can return nil. It will return, what the (print text) statement evaluates too, which is the last thing printed, which is text not nil.

OK, I will admit freely and unreservedly, that I am an idiot. That's what I get for trying to cut and paste an example from a larger chunk of code without trying it first.

If you still have any patience left, please take a look at this instead (complete example this time, and tested too!):

Code: Select all

(define (httpd-conf path query)
  (if (= 0 (length path))
    (begin
     (setq text "This is some text.")
     (print "HTTP/1.0 200 OK\r\n")
     (print (format "Server: newLISP v.%d (%s)\r\n" (sys-info -1) ostype))
     (print "Content-type: text/plain\r\n\r\n")
     (print text)
     nil)
    (begin
      (println "Path is " path)
      path)
    )
)

This does terminate properly. However, sometimes I get a completely blank page (which is the problem I was complaining about in the first place). Using NewLisp 9.2.0.

Thanks!

Lutz · Post by **Lutz** » Tue Sep 04, 2007 3:31 pm

I tested this httpd-conf on Mac OS X with the Safari 2.0.4 and FireFox 2.0.0.3 and everything works reliable on both side of the 'if' condition . I don't have a WinXP system available at the moment but will test on one as soon as I can. Latest next Tuesday, when I am back home in Florida

Lutz

oofoe · Post by **oofoe** » Tue Sep 04, 2007 7:33 pm

Hi!

Tried the program out from my machine at work (NewLisp 9.2.0, XP SP2, FireFox 1.5, running with "newlisp test.lsp -httpd -d 80"). Same results, with the occasional "The connection to the server was reset while the page was loading." error page instead of the blank page.

Thanks for looking at it!

Lutz · Post by **Lutz** » Wed Sep 12, 2007 3:21 pm

I tested your http-conf.lsp on Win32. To make it work you have to add a (read-headers) routine to clear out the headers coming from the client browser. On Mac OS X, Firefox and Safari seem to send a shorter header, and the "this is some text" page can be sent right away. On Win32 when returning a page from (httpd-conf ...). You have to clear out the headers first. As shown in the following modification of your http-conf.

Code: Select all

(define (httpd-conf path query)
  (write-file "debug.txt" (string "length:" (length path) " path->" path "<-\n"))
  (if (= 0 (length path))
    (begin
     (read-headers)
     (setq text "This is some text.")
     (print "HTTP/1.0 200 OK\r\n")
     (print (format "Server: newLISP v.%d (%s)\r\n" (sys-info -1) ostype))
     (print "Content-type: text/plain\r\n\r\n")
     (print text)
     nil)
    (begin
      path)
    )
)


(define (read-headers)
    (local (header)
        (while (!= (set 'header (read-line)) "")
            ; add more if required
            (if (starts-with header "Host:" 0) (env "HTTP_HOST" (5 header)))
            (if (starts-with header "User-Agent:" 0) (env "HTTP_USER_AGENT" (11 header)))
        )))

The 'read-haeders' routine is also contained in the http-conf.lsp shipped in newLISP 9.2.0 and installed in either /usr/share/newlisp/util or c:\Program Files\newisp\util. The routine clears out the header and also sets the environment variables for HTTP_HOST and HTTP_USER_AGENT.

Your modified http-conf.lsp works now reliably on Win32 using either Firefox or IE-6.

Lutz

ps: at the moment the only pending error with newLISP httpd mode on Win32 I can repeat, is making it work at all on IE-7. See also earlier response today to Maurizio.

oofoe · Post by **oofoe** » Thu Sep 13, 2007 7:24 pm

Lutz wrote:I tested your http-conf.lsp on Win32. To make it work you have to add a (read-headers) routine to clear out the headers coming from the client browser.

I'll try it as soon as I get home today.

Thanks for checking it out!

oofoe · Post by **oofoe** » Mon Sep 17, 2007 5:19 pm

Hi!

Just to let you know, the (read-headers) seems to do the trick! I guess I had oversimplified...

Thanks very much for looking at it!

newlispfanclub.alh.net

Truncation problem with newlisp web server (-httpd).

Truncation problem with newlisp web server (-httpd).

Newlisp not paying attention to nil...

Another data point...

Thanks!