query: threads in newLISP?

For the Compleat Fan
Locked

do you want to see thread support in newLISP?

yes, newLISP lacks concurrent processes.
3
75%
nope, threads would add nothing to newLISP.
0
No votes
newLISP-Tk needs threads to sew on its buttons.
1
25%
 
Total votes: 4

kazemori
Posts: 21
Joined: Fri Aug 22, 2003 1:42 am
Location: Vancouver BC | Portland OR

query: threads in newLISP?

Post by kazemori »

hello Lutz!

you have been rather busy! these past days releasing so many updates to newLISP; i have been reluctant to burden you with additional questions.

here is a general interest query: does newLISP support threads? threads are never mentioned in the newLISP documentation, so one could assume the negative.
(and we all know what happens when we assume ;-)

if threads are not supported, would implementation be difficult (like "never-going-to-happen" difficult), and do you have any plans in this direction?

thanks in advance for your time! -- kazemori

Lutz
Posts: 5289
Joined: Thu Sep 26, 2002 4:45 pm
Location: Pasadena, California
Contact:

Post by Lutz »

I thought about this many times, and one day perhaps it will happen. Because of the way memory is managed (no garbage collection) each thread would need it's own stacks for evaluation environments and intermedeate results. They would be able to use the same heap of recycable Lisp cells though and time-wise the overhead would propbably be minimal. I did something similar a decade ago on a Prolog dialect called 'RuleTalk'.

Anyway, you would have an overhead of about 80Kbyte of memory per thread. These threads would share symbols, which includes of course all user defined functions. Threads implemented this way would have no need of threads in the underlying OS and could not block each other except when doing I/O operations (but Ok with sockets, which can be used in a non blocking fashion in newLISP)..

Are you thinking about a specific application? Perhaps there is a solution running multiple instances of newLISP and letting them communicate via sockets (which also can run in a non-blocking fashion).

Lutz

kazemori
Posts: 21
Joined: Fri Aug 22, 2003 1:42 am
Location: Vancouver BC | Portland OR

Post by kazemori »

at the moment i am only thinking about the future (what i may have to do). certainly for limited problem-spaces, i agree that running multiple instances of newLISP (communicating via sockets) is a straight-forward and acceptable solution.

is it possible to limit the "scope" of a thread, so that it is not necessary for all symbols and user-defined functions to be shared? newLISP already implements contexts, in effect creating multiple namespaces (newLISP is a Lisp-n ;-) -- is it possible to limit a thread's "scope" to its calling context? consider a single instance of newLISP with at least n = 4 active contexts (SYS, MAIN, user-context-parent and user-context-child), wherein the "child" context is the calling context:

(a) perhaps this would limit resource allocation on a per thread basis (overhead). 80kb per thread is an objectively modest requirement; however, it is subjectively large (comparable to an instance of newLISP).

(b) by extension, demonstrating concurrency across many threads would be analogous to the object-oriented paradigm operating over simultaneous instance methods.

(c) if a thread "needs" to encompass the entire superset of namespaces, this is (more or less) the functional equivalent to a new instance of newLISP, in that a separate instance provides (relatively) equivalent functionality.

of course, i may just be rambling now... ;-)

thanks! -- kazemori

Lutz
Posts: 5289
Joined: Thu Sep 26, 2002 4:45 pm
Location: Pasadena, California
Contact:

Post by Lutz »

Most of the time with processes you want a high degree of separation of computing resources and enable communications or sharing of memory with special means. With threads you want to share resources to make common use of functions or memory as painless as possible.

I would not want to limit thread access to contexts, it also would not save memory overhead. Using contexts, every symbol from a foreign context has to be prefixed by the context name. So an accidental misuse of a context in a thread would be highly improbable.

Whenever you implement separating of resources quickly the demand for ways to work around it rises. Both, C++ and Java developed schemes of various degrees of separating access to functions or variables in objects, which made C++ IMHO to clumsy to use and overloaded it with theoretical concepts, which have to be learned and understood.

Probably some C++ users will vehemently disagree with me. I love Java because it implements a much easer to understand model of separating object resources.

But so much in using programming languages is just personal style and preference. Just look at the vastly different philosophies of doing a LISP from newLISP to Scheme to Common Lisp.

If I implement threads in newLISP I would want to do it as simple to use and understand as possible.

Lutz

kazemori
Posts: 21
Joined: Fri Aug 22, 2003 1:42 am
Location: Vancouver BC | Portland OR

Post by kazemori »

so, i was just rambling after all... (red-faced with embarrassment)

if you do implement threads, please do it in a simple and transparent way.

meanwhile, i will endeavor to keep correct working definitions of processes and threads separated in my mind (so that they will never again spill out so incorrectly into my posts! ;-)

thanks for your patience! -- kazemori

Lutz
Posts: 5289
Joined: Thu Sep 26, 2002 4:45 pm
Location: Pasadena, California
Contact:

Post by Lutz »

sorry if I sounded like a teacher, I was just looking for an opportunity to use the nice formatting features of this forum software :)

Lutz

eddier
Posts: 289
Joined: Mon Oct 07, 2002 2:48 pm
Location: Blue Mountain College, MS US

Post by eddier »

I would have to agree with Lutz on Java being much more thread friendly. However, M$ will never embrace it because they can't change Java into a Windoze specific language. So it will slowly fade away ... On the other hand, read a book before implementing threads in C++.

I like Java, but for most of my tasks, I have to be honest here, object oriented programming just gets in the way, i.e. small programs suddenly get complicated.

I would prefer newLisp without threads to a complicated newlisp with complicated features.

Eddie

kazemori
Posts: 21
Joined: Fri Aug 22, 2003 1:42 am
Location: Vancouver BC | Portland OR

Post by kazemori »

Lutz -- no offense taken. i enjoy learning! (so please, teach away! ;-)

i used to say, "you will know that i am dead when a beautiful girl walks by and i do not even move to look at her". i think it is more succinct, more realistic (and even more pc) to say, "you will know that i am dead when i stop learning".

for what it is worth, i do think newLISP should have threads (light-weight, transparent threads). i think newLISP is an excellent Lisp and needs to be more widely adopted. threads will help in this regard, as much of the so-called competition come with threads "out-of-the-box" (e.g., Io, Lua, Ruby, etc.).

perceptions count for a lot in choosing among the various programming languages. just look at java: the language that took over the world through the power of marketing (not because of its intrinsic merits ;-).

kazemori

Lutz
Posts: 5289
Joined: Thu Sep 26, 2002 4:45 pm
Location: Pasadena, California
Contact:

Post by Lutz »

>you will know that i am dead when i stop learning

this is very true for me too and one of the enjoyments for me in the computing field

> perceptions count for a lot in choosing among the various programming languages

that is probably so and definetely threads are a nice feature to have in a programming language and could open up the application of newLISP in other fields where this feature is essential, like hardware control, programming games (would make a good demo for the new feature), etc..

The more we talk about, it I think I am going to do it and it seems to me, that it is straightforward to implement.

Lutz

Lutz
Posts: 5289
Joined: Thu Sep 26, 2002 4:45 pm
Location: Pasadena, California
Contact:

Post by Lutz »

Yes Eddie!

it will be light and not burden newLISP in any way, size or speed- resource- performance wise. If that is not possible I will not do it.

Lutz

kazemori
Posts: 21
Joined: Fri Aug 22, 2003 1:42 am
Location: Vancouver BC | Portland OR

Post by kazemori »

although i am a physicist by academic training, and started my professional life as a programmer, i have in fact achieved most of my private sector success in the field of sales & marketing. (hiss! boo!)

of course, it was always high-tech sales & marketing in silicon valley. (hiss! boo!)

many of the things you learn while working on the "dark side" have analogous structures in the world of programming, for example: key-value pairs -- dark side equivalent: feature-benefit pairs

implementing threads in newLISP will...

feature: add more capabilities to a versatile language
benefit: encourage more people to download newLISP

feature: justify a major revision number - newLISP 8.0
benefit: implementing a major revision will potentially allow you to implement / re-implement other projects that you may have held back on until now...

i am not remotely competent to help you with C coding; however, i am wizard writer. (yes, i know how to capitalize words ;-) my pen is at your disposal (for starters, i am already scrubbing the newLISP manual for typo's and other gotcha's. (i even scrubbed 50% of newlisp.c last night)

i also propose to retool the introductory paragraph that you have on the O'Reilly open source directory site. (read the Tcl text, and then read newLISP ;-) please let me know if this is okay with you!

evangelist-by-nature -- kazemori

eddier
Posts: 289
Joined: Mon Oct 07, 2002 2:48 pm
Location: Blue Mountain College, MS US

Post by eddier »

Thanks Lutz for keeping things simple!

Java did a great job with Applets and with threads. But the Java class hierarchys have gotten too complicated. I have to complete many of my programs in a very short time frame. I gave up on Java for many of my jobs because I became bogged down in learning how to use a bunch of classes to perform a simple task. For instance to open a file for reading, I have to understand the IOException class, InputStream, FileInputStream, FilterInputStream, DataInputStream, etc... just too find which classes I need to use for a particular job. Many times, I can write a small script in newLisp, Perl, PHP, Ruby, or Python in a day that would otherwise take several days of looking through a dozen or so Java classes to find what I need.

Eddie

Lutz
Posts: 5289
Joined: Thu Sep 26, 2002 4:45 pm
Location: Pasadena, California
Contact:

Post by Lutz »

Yes, I will keep it simple ... absolutely! That has always been first priority and it will stay this way. It also has been the most challenging part of newLISP, because requests for new features are abundant and it is not always easy to decide what to leave out or put in. What happened to Java is typical for a tool which gets developed by a group, which has grown to big and managed corporate style, trying to please everybody.

There are also several old things in newLISP, I want to get rid of. Some of the functions are marked deprecated (collapsed into a new one, renamed, etc.). I would like to throw those out in the next round!?. There are other functions, I suspect nobody needs them anymore like 'command-line' for disabling/enabling console input. 'current-line' could be implemented using the same sysvars as in regular expressions input etc., etc., etc.

Any suggestions on this kind of issues is greatly appreciated.

Any input on the introductory paragraph on O'Reilly is very welcome. The current wording is about the same as used on SourceForge; and from there it has been copied by many directory services. Some of these services use paragraphs 5 years old, copied from a submission to AltaVista. O'Reilly is at least current, but perhaps the wording is too dry.

As most of you know I am not a native English speaker and apppreciate any help to 'debug' the manual and make it sound more 'English'. A good manual adds much to the usability of a programming language (at least for me).

Soo much to do ... good evening everybody

Lutz

kazemori
Posts: 21
Joined: Fri Aug 22, 2003 1:42 am
Location: Vancouver BC | Portland OR

Post by kazemori »

> Soo much to do ...

soo true !!!

i will be in touch soon -- kazemori

eddier
Posts: 289
Joined: Mon Oct 07, 2002 2:48 pm
Location: Blue Mountain College, MS US

Post by eddier »

Since we have find and slice we could get rid of "member"

I've seen the (cond (bool1 exp1) (bool2 exp2) ....) vs (cond bool1 exp1 bool2 exp2 ...) debated in several books. Would it be bad to implement the if or the cond like (? bool1 exp1 bool2 exp2 ...). Maybe even get rid of either "cond" or the "if" altogether?

I know we've tossed around getting rid of starts-with and ends-with string functions, but, since you also provide for lists???

Why have "first" and "nth 0?"

Do we really need let? Or if we need let shouldn't we also have a let! that allows something like

Code: Select all

(let! ((x 1) (y x)) (func ....))  where the second variable can use the value of the first variable?
Isn't lookup the same as (rest (assoc ....

how about a reverse-assoc?

I know I'm drawing flames here, but, ...

Eddie

Lutz
Posts: 5289
Joined: Thu Sep 26, 2002 4:45 pm
Location: Pasadena, California
Contact:

Post by Lutz »

You will definetely not draw flames from me. This is exactly the kind of comments I was looking for. In detail:

(1) 'member'
I thinks I have never even used it once! It's there only because of historical reasons and because it seems to mentined in all introductions to LISP. If no body else feels strong about it I also would like to get rid of it.

(2) 'cond'
The form with less parenthesis seems attractive to me and as you mention 'if' would then just be a special case of it. We could change 'if' to the newer multi arg form, then (who wants) could just do a (set 'cond if) to get a new 'cond'-binding. After a while we could mark the old 'cond' as deprecated and eventually throw it out. Never thought about it but seems attractive to me. What do others think? Isn't that the way AutoLisp is doing it?

(3) 'first' with 'nth 0'
Its such a frequently used function and the ex- 'car', 'head' etc.. I could imagine there is a lot oppostion. I myself sometimes use 'nth 0' for more readability, when I use a lot of other 'nth x' in the same area of code. I would like to hear more opinion that one.

(4) 'let' and 'let!'
let wasn't there until a year or two ago. Instead I had supplied a macro. Too many people where asking for it. let! alias 'letrec' .. yeah I agree. Personally I use 'let' rarely and use the 'local effect' of unused parameters instead. What I mostly dislike about 'let' is, that it adds to parenthesis clutter. If let! is cheap to do on top of 'let' I'll do it if its just another little bloat, I would rather like to get rid of it again because using the 'local effect' of unused parameters is a much nicer way to do locals when doing dynamic scoping anyway (in Scheme 'let' gives you at least lexical closure).

(5) 'lookup' versus (rest (assoc ...))

'lookup' can specify which position you want in the assoc list i.e.:

(lookup 'a '((x q w e r t y) (a q w e r t y)) 1) => q

I would love to keep it. although of course you could do (nth x (assoc ...)). Its just very handy when getting GET or POST or COOKIE parameters vi cgi.lsp and faster too, if you do a lot of it.

(6) 'assoc-reverse'
what is it? How would it work?

some other thoughts:
what about the way that 'for' works with the always positive step-parameter? Does this confuse everybody , or do you think its nice.

BTW for all Mac OSX users, just got 'import' working for Mac OSX. it will be included in the next distribution. If somebody needs it now, please contact me.


Lutz

HPW
Posts: 1390
Joined: Thu Sep 26, 2002 9:15 am
Location: Germany
Contact:

Post by HPW »

>(1) 'member'

Would it cost so much to keep traditionals like this inside?

>(2) 'cond'
>The form with less parenthesis seems attractive to me ...

Attractive for reading and typing, but modern editor with parenthesis checking and formating makes life easier when programms grow. Then you can jump over code blocks from starting to closing parenthesis.

>Isn't that the way AutoLisp is doing it?

Yes, and people coming from there, would like to have the current cond.
Of course it remains your decision. As you wrote, it is not easy to get it right for all users.


So for my taste, do not leave traditional lisp to far behind.
Hans-Peter

Locked