Modified HTTPD

grable · Post by **grable** » Fri Jun 17, 2005 4:42 pm

I tinkered with the included HTTPD some, and added a modified CGI:put-page into it, which executes an *.lsp file inside the current process and sends the result back.

should be a little faster then starting new processes, and scripts have full acces to the HTTPD context.
(reads <% %> and <%= %> tags.)

heres an example usage:

Code: Select all

<%
	(set 'page-title "Your IP")
%>
<html>
	<head>
		<title><%=page-title%></title>
	</head>
	<body>
		<h1><%=page-title%>: <%=(first (net-peer HTTPD:connection))%></h1>
	</body>
</html>

heres the modified HTTPD file for those interested.

this is just so much fun =)

grable · Post by **grable** » Fri Jun 17, 2005 11:21 pm

added support for query data with GET, but am having problems with POST.
which is wierd since its basicly the same happening on both GET and POST.
(puting data in QUERY_STRING env, and runing cgi-put-page)

maybe i need a fresh pair of eyes on this ;) hehehe

dev version with funky POST HTTPD_dev

Lutz · Post by **Lutz** » Fri Jun 17, 2005 11:35 pm

Look into modules/cgi.lsp, perhaps it can help you. GET and POST isn't really the same. In a GET request variable/value pairs are encoded in the QUERY_STRING but in in a POST request you have to read more to get the post data. cgi.lsp handles this in a transparent fashion. It also contains a routine to do <% %> tag preprocessing, but your approach to have it integrated in the httpd itself is probably the better way to do it.

Lutz

grable · Post by **grable** » Fri Jun 17, 2005 11:37 pm

my bad ;) was just a pathing issue.. hehehe

ive updated the original HTTPD link, for those interested.

grable · Post by **grable** » Fri Jun 17, 2005 11:40 pm

Thats where i "borrowed" this idea from in the first place.. ;) hehehe

bit im just sending the POST data in QUERY_STRING.. thats probably not the right thing to do ? . maybe add a POST_STRING env or something?

am not that familiar with http and cgi.

grable · Post by **grable** » Sat Jun 18, 2005 12:00 am

Well i think i solved it.. theres realy no need to use env variables when i have acess to the HTTPD context.. to i just put things in HTTPD:QUERY_STRING and HTTPD:POST_STRING when the data is present..

heres the recent examle (and previous url has again been updated)
il probaly revork this code, since it pretty much a hack right now to get it to work properly..

Code: Select all

<%
	(set 'page-title "Your IP")
%>
<html>
	<head>
		<title><%=page-title%></title>
	</head>
	<body>
		<h1><%=page-title%>: <%=(first (net-peer HTTPD:connection))%></h1>
		<hr>
		QUERY_STRING: <%=HTTPD:QUERY_STRING%><br>
		POST_STRING: <%=HTTPD:POST_STRING%><br>
		<form method="POST" action="yourip.lsp?testing">
			<input type="text" name="postdata" value="some random data">
			<input type="submit">
		</form>
	</body>
</html>

btw lutz (the newlisp expert ;) , in the loop of the cgi-put-page that evals embeded parts i do this:

Code: Select all

(while ... 
   ...
   (context (sym file-name)) 
   (eval-string ...)
   (context HTTPD)
   ...
) 
 ... 
(delete (sym file-name))

is this the correct way to use a context? or should i put it outside of the loop?

Lutz · Post by **Lutz** » Sat Jun 18, 2005 1:09 am

Thats the right way to do it. Changing the context around the evaluation will protect httpd from stuff happening inside the evaluation. I think because of that same reason I left <% %> tage processing in cgi.lsp used by the spawned cgi process.

Your way of server-side processing is probably more efficient, but also brings the danger of doing something 'evil' to the httpd server. Bracketing in its own context will not protect against everything. Imagine code in the preprocessed page hangs in a loop, then the httpd server is dead ;)

Lutz

grable · Post by **grable** » Sat Jun 18, 2005 1:19 am

Ah.. k.. i was just wondering.. glad i understood it.. hehe

Imagine code in the preprocessed page hangs in a loop, then the httpd server is dead ;)

hehehehe, yeah.. i thought about that, il just have to be very careful ;)

Although an infinite loop could potentialy happen with another process to, since the httpd will wait for its termination anyway.

grable · Post by **grable** » Sat Jun 18, 2005 2:34 am

my hacked version of HTTPD now eats cookies =)

i merged the functions from CGI.lsp with HTTPD, so scripts have roughly the same interface.

script interface:

Code: Select all

# variables
HTTPD:QUERY_STRING  = raw query data
HTTPD:POST_STRING    = raw post data
HTTPD:COOKIES_STRING = raw cookies data
HTTPD:CONTENT_TYPE  = content type of script result (text/html is default)
HTTPD:PUT_COOKIE    = raw cookie data to send to client
HTTPD:PARAMS         = query & post data as (key value) pairs
HTTPD:COOKIES        = cookies data as (key value) pairs

# functions
(HTTPD:get-cookie "name")
(HTTPD:set-cookie "name" "value")
(HTTPD:set-cookie-ex "name" "value" "domain" "path")
(HTTPD:get-param "name")

HTTPD now does the same as CGI.lsp, eg it fill the PARAMS and COOKIES for you before execution of the script, allso you can have multiple cookies in one request.

example usage:

Code: Select all

<%
	(set 'visits (int (HTTPD:get-cookie "VISITS")))
	(if (= visits nil) (set 'visits 1))
	(HTTPD:set-cookie "VISITS" (+ visits 1))
%>
<html>
	<head>
		<title>Test Page</title>
	</head>
	<body>
		<h1>Your IP: <%=(first (net-peer HTTPD:connection))%></h1>
		<hr>
		Number of visits using cookies: <%=visits%><br>
		QUERY_STRING: <%=HTTPD:QUERY_STRING%><br>
		POST_STRING: <%=HTTPD:POST_STRING%><br>
		COOKIES_STRING: <%=HTTPD:COOKIES_STRING%><br>
		<form method="POST" action="yourip.lsp?testing">
			<input type="text" name="postdata" value="some random data">
			<input type="submit">
		</form>
	</body>
</html>

hope someone else allso finds this useful =) i kinda like having full access to my server, and being able to do whatever i want with it .. hehe

newest version of HTTPD

HPW · Post by **HPW** » Sat Jun 18, 2005 6:37 am

Hi grable,

I have problems to access your FTP.
Can you offer another option or send it?

grable · Post by **grable** » Sat Jun 18, 2005 1:21 pm

I have problems to access your FTP.
Can you offer another option or send it?

Its in the mail =)

the ftp is on 24/7 so it "should" work.. dunno why you cant get in..

theres no password for anonymous, so the first one should work,
but some browsers may complain so the second one is more "correct".

ftp://anonymous@grable.cjb.net/HTTPD
ftp://anonymous:@grable.cjb.net/HTTPD

if anyone else cant get it and wants me to send it, just speak up =)

statik · Post by **statik** » Sat Jun 18, 2005 6:55 pm

Very cool.

grable · Post by **grable** » Sun Jun 19, 2005 2:59 pm

I soon found out that having everything in a single process isnt to good either :/

its was a little bit faster then spawning a new process..
bit it could stil only serve 1 client at a time.. so it now spawns
process again ;) LOL

although this time with proper "forking" using (process)
so its can serve many clients.

it sends the "control" code over the commandline, and loads the
html+newlisp template file and runs it... so the control code must send
the result to the client and close the socket. freeing the server up to do other things =)

allmost same interface as before, just removed the HTTPD context from the script
and i removed the old CGI way completly.

heres the updated files:
grb_httpd.lsp
index.lsp

go here to test http://grable.cjb.net/

UPDATE: forks regular file downloads as well, like images/etc.

Lutz · Post by **Lutz** » Sun Jun 19, 2005 3:56 pm

Very interesting, congratulations, I would love to see the code, but cannot connect to your ftp, can you email it to me? lutz@nuevatec.com, and put the word 'newLISP' in the subject line, so my spamfilter doesn't eat it :)

Lutz

grable · Post by **grable** » Sun Jun 19, 2005 4:13 pm

Thanks lutz =)

their in the mail...

Since some people are having trouble geting in to my ftp, ive
put the files on the http server as well..

http://grable.cjb.net/grb_httpd.rar

HPW · Post by **HPW** » Sun Jun 19, 2005 6:43 pm

Hi grable,

Just tried your HTTP download link.

It open the Save_as dialog and show the file-download window.
But there it hangs and get no progress.

When I enter only:

http://grable.cjb.net/

then I get the:

Index - test page for grb_httpd.lsp HTTP server

But there the 'Download Source' link also does not work.

Is this a newLISP-server? Seems to have problems.

grable · Post by **grable** » Sun Jun 19, 2005 6:51 pm

Yes, its on the the modified newlisp server.

Wierd, because i have no trouble downloading myself. I even tested a 50 mb file while constantly refreshing the index page, without any problems.

Seems a few ppl have trouble acessing my ftp too :(

What browser are you using? what os??
Myself am on Windows XP and using the latest FireFox..

HPW · Post by **HPW** » Sun Jun 19, 2005 7:04 pm

Windows XP pro german IE 6.0.2600

It does get the response normal when using:

http://grable.cjb.net/

But clicking on the download link it takes very long to get any more.

?

grable · Post by **grable** » Sun Jun 19, 2005 7:10 pm

hmmm.. it must be IE ;) the ppl having trouble with the ftp use IE to .. hehe

have you tried with newLISP's (get-url) ? that should bypass any problems in the browser.

Code: Select all

(write-file "c:\grb_httpd.rar" (get-url "http://grable.cjb.net/grb_httpd.rar"))

or better yet, switch to firefox perhaps? ;)

or i could just email them to you..

it must be IE's handling of the "application/octet-stream" mime-type or something.. for its the same process in sending a webpage as in sending a file.. its only the Content-Type (mimetype) thats different in the header.

im sorry, but theres not much i can do :(

UPDATE:
i tested in IE myself now, and the index page was slow as hell, and the download didnt work either :( so theres defnetly something IE does or doesnt do that firefox and (get-url) does .. il have look into it later.

HPW · Post by **HPW** » Sun Jun 19, 2005 7:21 pm

I get it with:

Code: Select all

(write-file "c:\grb_httpd.rar" (get-url "http://grable.cjb.net/grb_httpd.rar"))

But it was stored in "c:\programme\newlisp\grb_httpd.rar"

My IE has no problems with all download from newLISP.org or other sites.
So a newLISP server should be able to work with IE because a majority still use it (Mozilla might be better, but we can not change everyone's browser installation)

grable · Post by **grable** » Sun Jun 19, 2005 7:29 pm

Yeah. i forgot the extra \ for paths ;)

although it ended up in C:\ on my pc.. lol

yes yes.. i know it has to work in IE as well ;)..
but for now itl have to be NON-IE only.. cuz i have no clue as to why IE is slow and refuses to work.

i havent changed that much ;) hehehe

grable · Post by **grable** » Sun Jun 19, 2005 7:44 pm

LOL

I forgot to close the socket when done sending data ;) stupid me!

its wierd that this worked on firefox at all! it should atleast have waited for the timeout..

even though i think newLISP/exe's closes open sockets on exit.

but now it works atleast ;) so all you happy IE users can use this shit to.

grable · Post by **grable** » Sun Jun 19, 2005 8:58 pm

I just changed the newLISP WIKI to work with the format i use in the modified httpd, wasnt that much to change. so far so good ;)

I just have to test all the functionality in newLISP wiki before i post it.

And i fixed some stuff in the server aswell.

newest version is here:
http://grable.cjb.net/grb_httpd.rar

HPW · Post by **HPW** » Sun Jun 19, 2005 9:11 pm

A (happy) IE user say: Thanks

;-)

Of cource: No shit here!

grable · Post by **grable** » Sun Jun 19, 2005 9:36 pm

A (happy) IE user say: Thanks

glad to help.. it was stupid of me not to see it earlier ;)

UPDATE:
yes i know theres alot of them.. atleast you get the latest no?

the modified wiki for grb_httpd works, and its much faster this way than running on the old httpd.

just replace the files in this rar with the ones you got. TAKE BACKUP!!

http://grable.cjb.net/moded_wiki.rar

or you can test it here:

http://grable.cjb.net:8080

newlispfanclub.alh.net

Modified HTTPD

Modified HTTPD

small update...

doh!

Cookies!!!

complete turnaround ;)