Page 1 of 1
newLisp Document Search Site
Posted: Sat Aug 01, 2009 6:39 pm
by Joe
Hi everyone,
I put together a site that allows you to search all of the newLisp documentation, including tutorials, related sites, etc.
http://newlispsearch.nfshost.com/
Please let me know what you think. Right now it runs with PHP, but hopefully a future version will be newLisp (patience please, I'm still working though "Hello world").
Please note the slow speed is the fault of the server and not the script.
Enjoy, and please provide feedback.
Joe
Posted: Sat Aug 01, 2009 7:27 pm
by ale870
Good starting point!
But I think it is too "similar" to a standard google search.
I think you could supply some specific features that I cannot get using a standard search engine.
For example: I think one big missing thing in newLisp shell is an online help. So, why don't you realize a newlisp module to check your site via shell? For example, I could type, in the console, something like:
(help:find "println")
And that module (I called it "help") could send a search query to your engine, showing the results directly in my console!
Posted: Sat Aug 01, 2009 7:43 pm
by cormullion
I'm impressed - it's pretty cool. I will definitely use it - the sluggishness is occasionally noticeable. That's presumably nfshost? (Or php? :-) ) Or are you loading tons of stuff as well?
I always have comments and suggestions... :)
In the results page, you have headers in the form
"apply.html (1)" or "sys-sym-const.html "... An improvement would be to tell the user what the matching source documents were, rather than the precise name of an html page which (presumably) they need to know nothing about. It could be done as a hover/mouse text, perhaps.
Will you be able to easily update the content when Lutz updates things (which he does often...)?
The links to the Introduction are less than ideal, I know. Ideally the Introduction would be in lots of small sections (for indexing) yet easily combined to make a continuous whole (for printing/downloading) - and I haven't the patience to find out how to do it right. Anyway, the docuwiki's days might be numbered ...
In the CSS: "Sans-Serrif" can't be right, can it?
And it's definitely "newLISP"... :)
Great work - can't wait to see what else you get up to!
Posted: Sat Aug 01, 2009 9:38 pm
by Joe
cormullion wrote:... the sluggishness is occasionally noticeable. That's presumably nfshost? (Or php? :-) ) Or are you loading tons of stuff as well?
There is some pretty substantial processing in the PHP code, but when I had it on my test server (also remote), it was very fast, so I think nfhost plays a big role. On the other hand, you can't beat the price.
cormullion wrote:
Will you be able to easily update the content when Lutz updates things (which he does often...)?
No, and that is the real downside. Maybe in 2.0. The problem, at least from my limited understanding is getting the user to specific parts of a long document. Even google doesn't help you there.
cormullion wrote:The links to the Introduction are less than ideal...
I agree. On the other hand, that is most dynamic content because it always links to the live document. It was just the best way to get it done.
cormullion wrote:
Great work - can't wait to see what else you get up to!
How about a newLISP code repository (think CPAN)?
BTW, this is a quick & dirty (somewhat experimental) version just to test the waters. I know there is a lot more that can be done, and appreciate all suggestions.
Joe
Posted: Sat Aug 01, 2009 9:41 pm
by Joe
ale870 wrote:For example: I think one big missing thing in newLisp shell is an online help.
Let me see how this goes. I do appreciate the suggestion, but that is probably a little advanced for my limited newLISP experience. But I will put it on the ToDo list.
Joe
Posted: Sun Aug 02, 2009 10:50 am
by ale870
Ok!
About "official documentation" or sites created by us (newLisp fan and supporters!) we can insert, in the html code, some special tags to get: keywords, functions usage, description, "what's new" (documentation updates), etc...
I think in this way we could simplify your parsing process. These tags could be inserted in html comments.
And I think we could make a central code repository and modules installer like "Ruby gems" (I used it, it's great! It's similar to Ubuntu package manager).
I think we only need to define a standard approved by Lutz, then we could work, all together, to implement it.
Joe, if you use php, why don't you look for a faster, free hosting site? There are many providers free and fast.
I think nfshosting is good for newLisp cgi, but if you work in php you are a "world" of opportunities.
Posted: Mon Aug 03, 2009 3:45 pm
by cormullion
What happened to the Code Repository post? I was going to ask some questions (about approvals, review, updates, longevity, etc) but the post has gone today...
Posted: Mon Aug 03, 2009 4:26 pm
by Joe
cormullion wrote:What happened to the Code Repository post? I was going to ask some questions (about approvals, review, updates, longevity, etc) but the post has gone today...
I injured my wrist yesterday and won't be able to work on it for awhile. Since there had been no replies, just pulled the post and the site...for now.
Besides, I had completely overlooked (ADD kicked in again) that newlisp.org already has a code collections page.
We can still discuss it if you like, perhaps open a thread to determine if it is really needed?
Off to find a wrist brace that still lets me use a mouse & keyboard.
Joe
Posted: Mon Aug 03, 2009 4:29 pm
by Lutz
Actually parsing is currently pretty simple. Here is the function I am using in nls (in newlisp-x.x.x/util/nls):
Code: Select all
(define (help func-name)
(if (find func-name "|+*-") (push "\\" func-name))
(set 'html-text (join (find-all (format {(syntax: \(%s.*?\))} func-name)
(read-file "/usr/share/doc/newlisp/newlisp_manual.html")) "\n"))
(replace "<.*?>" html-text "" 0)
(replace "<" html-text "<")
(replace ">" html-text ">")
(replace "&" html-text "&")
(println html-text)
"")
This displays only the syntax line(s). E.g. when running the nls shell on Unix you see this:
Code: Select all
MAIN:/Users/lutz> help pri
syntax: (primitive? exp)
syntax: (print exp-1 [exp-2 ... ])
syntax: (println exp-1 [exp-2 ... ])
MAIN:/Users/lutz>
There is another function help in newlisp.x.x.x/examples in the file init.lsp.example:
Code: Select all
(define-macro (help func)
(if (primitive? (eval func))
(let (func-name (name func))
(if (ends-with func-name "?") (replace "?" func-name "p"))
(! (format "lynx /usr/share/doc/newlisp/newlisp_manual.html#%s" func-name)))
(format "%s is not a built-in function" (name func))))
This display a whole page, but you need the lynx text web browser installed.
Basically paresing from one {<h4>(syntax: \(%s.*?)</h4>} to the next {<h4>(syntax: \(%s.*?)</h4>} will cut you out the reference for one function.
Posted: Mon Aug 03, 2009 4:57 pm
by cormullion
Joe wrote:I injured my wrist yesterday and won't be able to work on it for awhile.
No problem - take the time to get better! Some things are more important than coding... :)
Posted: Tue Aug 04, 2009 3:34 pm
by Joe
Major Update
Filled with newLISPy goodness and new sites added.
http://newlispsearch.nfshost.com/
Go ahead, show me some love.
Joe
Posted: Tue Aug 04, 2009 4:54 pm
by HPW
Schouldn't 'newlisp.og' not read 'newlisp.org' in the site-list?
Posted: Tue Aug 04, 2009 4:57 pm
by Joe
HPW wrote:Schouldn't 'newlisp.og' not read 'newlisp.org' in the site-list?
I would certainly think so.
I will find the person responsible for the error and take appropriate action.
Thanks!
Posted: Tue Aug 04, 2009 8:25 pm
by cormullion
It looks good! I'll use it in the next few weeks and report back with my experiences...
Posted: Tue Aug 04, 2009 8:31 pm
by Joe
cormullion wrote:It looks good! I'll use it in the next few weeks and report back with my experiences...
Thanks. BTW, this version is fully dynamic with the search and results coming directly from specific sites.
Another feature to note, is that all results are categorized by site.
I hope it works well for everyone.