newLisp Document Search Site

Notices and updates
Locked
Joe
Posts: 11
Joined: Sat Jan 03, 2009 10:45 pm
Location: PA USA

newLisp Document Search Site

Post by Joe »

Hi everyone,

I put together a site that allows you to search all of the newLisp documentation, including tutorials, related sites, etc.

http://newlispsearch.nfshost.com/

Please let me know what you think. Right now it runs with PHP, but hopefully a future version will be newLisp (patience please, I'm still working though "Hello world").

Please note the slow speed is the fault of the server and not the script.

Enjoy, and please provide feedback.

Joe

ale870
Posts: 297
Joined: Mon Nov 26, 2007 8:01 pm
Location: Italy

Post by ale870 »

Good starting point!
But I think it is too "similar" to a standard google search.
I think you could supply some specific features that I cannot get using a standard search engine.

For example: I think one big missing thing in newLisp shell is an online help. So, why don't you realize a newlisp module to check your site via shell? For example, I could type, in the console, something like:

(help:find "println")

And that module (I called it "help") could send a search query to your engine, showing the results directly in my console!
--

cormullion
Posts: 2038
Joined: Tue Nov 29, 2005 8:28 pm
Location: latiitude 50N longitude 3W
Contact:

Post by cormullion »

I'm impressed - it's pretty cool. I will definitely use it - the sluggishness is occasionally noticeable. That's presumably nfshost? (Or php? :-) ) Or are you loading tons of stuff as well?

I always have comments and suggestions... :)

In the results page, you have headers in the form
"apply.html (1)" or "sys-sym-const.html "... An improvement would be to tell the user what the matching source documents were, rather than the precise name of an html page which (presumably) they need to know nothing about. It could be done as a hover/mouse text, perhaps.

Will you be able to easily update the content when Lutz updates things (which he does often...)?

The links to the Introduction are less than ideal, I know. Ideally the Introduction would be in lots of small sections (for indexing) yet easily combined to make a continuous whole (for printing/downloading) - and I haven't the patience to find out how to do it right. Anyway, the docuwiki's days might be numbered ...

In the CSS: "Sans-Serrif" can't be right, can it?

And it's definitely "newLISP"... :)

Great work - can't wait to see what else you get up to!

Joe
Posts: 11
Joined: Sat Jan 03, 2009 10:45 pm
Location: PA USA

Post by Joe »

cormullion wrote:... the sluggishness is occasionally noticeable. That's presumably nfshost? (Or php? :-) ) Or are you loading tons of stuff as well?
There is some pretty substantial processing in the PHP code, but when I had it on my test server (also remote), it was very fast, so I think nfhost plays a big role. On the other hand, you can't beat the price.
cormullion wrote:
Will you be able to easily update the content when Lutz updates things (which he does often...)?
No, and that is the real downside. Maybe in 2.0. The problem, at least from my limited understanding is getting the user to specific parts of a long document. Even google doesn't help you there.
cormullion wrote:The links to the Introduction are less than ideal...
I agree. On the other hand, that is most dynamic content because it always links to the live document. It was just the best way to get it done.
cormullion wrote:
Great work - can't wait to see what else you get up to!
How about a newLISP code repository (think CPAN)?

BTW, this is a quick & dirty (somewhat experimental) version just to test the waters. I know there is a lot more that can be done, and appreciate all suggestions.

Joe

Joe
Posts: 11
Joined: Sat Jan 03, 2009 10:45 pm
Location: PA USA

Post by Joe »

ale870 wrote:For example: I think one big missing thing in newLisp shell is an online help.
Let me see how this goes. I do appreciate the suggestion, but that is probably a little advanced for my limited newLISP experience. But I will put it on the ToDo list.

Joe

ale870
Posts: 297
Joined: Mon Nov 26, 2007 8:01 pm
Location: Italy

Post by ale870 »

Ok!
About "official documentation" or sites created by us (newLisp fan and supporters!) we can insert, in the html code, some special tags to get: keywords, functions usage, description, "what's new" (documentation updates), etc...
I think in this way we could simplify your parsing process. These tags could be inserted in html comments.
And I think we could make a central code repository and modules installer like "Ruby gems" (I used it, it's great! It's similar to Ubuntu package manager).
I think we only need to define a standard approved by Lutz, then we could work, all together, to implement it.

Joe, if you use php, why don't you look for a faster, free hosting site? There are many providers free and fast.
I think nfshosting is good for newLisp cgi, but if you work in php you are a "world" of opportunities.
--

cormullion
Posts: 2038
Joined: Tue Nov 29, 2005 8:28 pm
Location: latiitude 50N longitude 3W
Contact:

Post by cormullion »

What happened to the Code Repository post? I was going to ask some questions (about approvals, review, updates, longevity, etc) but the post has gone today...

Joe
Posts: 11
Joined: Sat Jan 03, 2009 10:45 pm
Location: PA USA

Post by Joe »

cormullion wrote:What happened to the Code Repository post? I was going to ask some questions (about approvals, review, updates, longevity, etc) but the post has gone today...
I injured my wrist yesterday and won't be able to work on it for awhile. Since there had been no replies, just pulled the post and the site...for now.

Besides, I had completely overlooked (ADD kicked in again) that newlisp.org already has a code collections page.

We can still discuss it if you like, perhaps open a thread to determine if it is really needed?

Off to find a wrist brace that still lets me use a mouse & keyboard.

Joe

Lutz
Posts: 5289
Joined: Thu Sep 26, 2002 4:45 pm
Location: Pasadena, California
Contact:

Post by Lutz »

Actually parsing is currently pretty simple. Here is the function I am using in nls (in newlisp-x.x.x/util/nls):

Code: Select all

(define (help func-name)
  (if (find func-name "|+*-") (push "\\" func-name))
  (set 'html-text (join (find-all (format {(syntax: \(%s.*?\))} func-name)
    (read-file "/usr/share/doc/newlisp/newlisp_manual.html")) "\n"))
  (replace "<.*?>" html-text "" 0)
  (replace "<" html-text "<")
  (replace ">" html-text ">")
  (replace "&" html-text "&")
  (println html-text)
  "")
This displays only the syntax line(s). E.g. when running the nls shell on Unix you see this:

Code: Select all

MAIN:/Users/lutz> help pri
syntax: (primitive? exp)
syntax: (print exp-1 [exp-2 ... ])
syntax: (println exp-1 [exp-2 ... ])
MAIN:/Users/lutz> 
There is another function help in newlisp.x.x.x/examples in the file init.lsp.example:

Code: Select all

(define-macro (help func)
   (if (primitive? (eval func))
       (let (func-name (name func))
            (if (ends-with func-name "?") (replace "?" func-name "p")) 
       (!  (format "lynx /usr/share/doc/newlisp/newlisp_manual.html#%s" func-name)))
   (format "%s is not a built-in function" (name func))))
This display a whole page, but you need the lynx text web browser installed.

Basically paresing from one {<h4>(syntax: \(%s.*?)</h4>} to the next {<h4>(syntax: \(%s.*?)</h4>} will cut you out the reference for one function.
Last edited by Lutz on Wed Aug 05, 2009 12:10 am, edited 1 time in total.

cormullion
Posts: 2038
Joined: Tue Nov 29, 2005 8:28 pm
Location: latiitude 50N longitude 3W
Contact:

Post by cormullion »

Joe wrote:I injured my wrist yesterday and won't be able to work on it for awhile.
No problem - take the time to get better! Some things are more important than coding... :)

Joe
Posts: 11
Joined: Sat Jan 03, 2009 10:45 pm
Location: PA USA

Post by Joe »

Major Update

Filled with newLISPy goodness and new sites added.

http://newlispsearch.nfshost.com/

Go ahead, show me some love.

Joe

HPW
Posts: 1390
Joined: Thu Sep 26, 2002 9:15 am
Location: Germany
Contact:

Post by HPW »

Schouldn't 'newlisp.og' not read 'newlisp.org' in the site-list?
Hans-Peter

Joe
Posts: 11
Joined: Sat Jan 03, 2009 10:45 pm
Location: PA USA

Post by Joe »

HPW wrote:Schouldn't 'newlisp.og' not read 'newlisp.org' in the site-list?
I would certainly think so.

I will find the person responsible for the error and take appropriate action.

Thanks!

cormullion
Posts: 2038
Joined: Tue Nov 29, 2005 8:28 pm
Location: latiitude 50N longitude 3W
Contact:

Post by cormullion »

It looks good! I'll use it in the next few weeks and report back with my experiences...

Joe
Posts: 11
Joined: Sat Jan 03, 2009 10:45 pm
Location: PA USA

Post by Joe »

cormullion wrote:It looks good! I'll use it in the next few weeks and report back with my experiences...
Thanks. BTW, this version is fully dynamic with the search and results coming directly from specific sites.

Another feature to note, is that all results are categorized by site.

I hope it works well for everyone.

Locked