get-url with redirection?

Machine-specific discussion
Unix, Linux, OS X, OS/2, Windows, ..?
Locked
zxcel
Posts: 1
Joined: Sat Dec 12, 2009 12:18 am

get-url with redirection?

Post by zxcel »

I want to download the file, but get-url can't do it. That's because the url that i have, redirects me to another url. All browsers, Opera, wget - all of them understand that redirection. How can I do it?

hilti
Posts: 140
Joined: Sun Apr 19, 2009 10:09 pm
Location: Hannover, Germany
Contact:

Re: get-url with redirection?

Post by hilti »

Try to use CURL instead

Code: Select all

(exec (string "curl -u "http://www.google.com" --progress-bar"))
CURL handles HTTPS, too.

Cheers
Hilti
--()o Dragonfly web framework for newLISP
http://dragonfly.apptruck.de

hilti
Posts: 140
Joined: Sun Apr 19, 2009 10:09 pm
Location: Hannover, Germany
Contact:

Re: get-url with redirection?

Post by hilti »

Sorry.

That's the correct snippet:

Code: Select all

(exec (string "curl 'http://www.google.com' --progress-bar"))
--()o Dragonfly web framework for newLISP
http://dragonfly.apptruck.de

Fritz
Posts: 66
Joined: Sun Sep 27, 2009 12:08 am
Location: Russia

Re: get-url with redirection?

Post by Fritz »

You can also look into the text "get-url" returns to you. I believe, inside you will find a correct address of the file.

First get-url:

Code: Select all

(get-url "http://lenta.ru/info")
Result:

Code: Select all

"<!DOCTYPE HTML PUBLIC \"-//IETF//DTD HTML 2.0//EN\">\n<html><head>\n<title>302 Found</title>\n</head><body>\n<h1>Found</h1>\n<p>The document has moved <a href=\"http://lenta.ru/info/\">here</a>.</p>\n<hr>\n<address>Apache Server at lenta.ru Port 80</address>\n</body></html>\n"
Parsing correct address:

Code: Select all

(first (regex {(?<=href=")[^"]+} (get-url "http://lenta.ru/info")))
Result:

Code: Select all

"http://lenta.ru/info/"
Final get-url:

Code: Select all

(get-url (first (regex {(?<=href=")[^"]+} (get-url "http://lenta.ru/info"))))

Lutz
Posts: 5289
Joined: Thu Sep 26, 2002 4:45 pm
Location: Pasadena, California
Contact:

Re: get-url with redirection?

Post by Lutz »

In the past 'get-url' only handled re- "Location:" headers together with error codes 301 and 303. I have now added error code 302 for v.10.2.0 to force a second request also on this error code, and it handles the http://lenta.ru/info case correctly.

Locked