RFC open on newLISP documentation

[nigelbrown]

Hi Lutz,
The reasion I suggested _questionmark_ is that listp is a likely name for a user to choose for ther own defined predicate function (especilly if they have an oldLisp background) - I was looking for a substitution that was unlikely to result in a naming clash. For a shorter form how about list_qm etc? Or maybe list__p, ie double underscore, and a manual note that double underscore is best not used in user defined names?

Nigel

[Lutz]

I understand, but think they would confuse even more if they chose a 'listp' as a function name, I think I will leave it with the p, it also would leave them the freedom, to do things like:

Code: Select all

> (constant 'listp list?)
list? <97D0>
> (listp '(a s d f))
true
> 

to redefine to the 'old' ways

Lutz

[nigelbrown]

Fair comment. Having P is more standard.

Nigel

[nigelbrown]

I would like to suggest the need for some clarifying text for the Bayesian Chain example. The example as worked requires the assumption that retesting events are statistically independent. For a worked example see the test paper Q4 parts g) and h) http://math.usask.ca/~bickis/courses/stats246/as203.pdf and the answer at http://math.usask.ca/~bickis/courses/st ... as203.html .

The confounding issue is that if there is something that makes the test positive in a person that is unrelated to the disease in question but that is always going to be there then retest results can't be meaningfully chained.

Maybe add qualifying lines to the example so it reads:<start-text>

example:
(bayes-query '(test-positive) Data true)
=> (0.3076923077 0.6923076923)

If the testing is modified so that any negative result is interpreted as a negative but a positive result is retested and considered a positive only if the retest is a positive, and assuming that the results of the test and retest are statistically independent we have:

(bayes-query '(test-positive test-positive) Data true)
=> (0.64 0.36)

(bayes-query '(test-positive test-positive test-positive) Data true)
=> (0.8767123288 0.1232876712)

The example shows that a person may have to be tested several times positive to be classified as sick with confidence.

Regarding the requirement of statistical independence, two tests would be nearly independent if the variation in test results were due to variation in the lab that was not related to the patient. However, they would not be independent if there was variation among patients as to how readily the disease could be diagnosed. So if, for example, the test involved measuring some enzyme in the blood, then independence between retests would be expected if the lab quantitation was imprecise. Independence would not be expected if some other blood component unrelated to the disease was present in some individuals and this component interfered with the measurement of the enzyme.

<end-text>
The last paragraph is a straight copy from the exam answers mentioned above - if you were generally happy with the tone of the suggested wording then I could rework the lifted section or seek permission (although technically it is possibly too small a part of the whole to be a copyright violation that would be the 'right thing to do')
I will also use the working as set out in the web reference to check the results.

Nigel

PS another way of explaining the issue is here http://www.merck.com/mrkshared/mmanual/ ... 5/295e.jsp

viz
the results of the various tests are often assumed to be conditionally independent (ie, the likelihood of a given test result depends on the gold standard diagnosis alone rather than on the gold standard and results of the other test), and the performance characteristics of the second test can be conditioned only on the gold standard for diagnosis.

[Ryon]

I think I've found a small error in the online Manual and Reference.

In 13. Dynamic and lexical scoping, the example

Code: Select all

(define (print-vars)
    (print "X=" x " Y=" y " Z= " z "\n")

is missing a closing parenthesis.

[Lutz]

Thanks Ryon, will be corrected,

Lutz

[Ryon]

The next example seems to be a little off also:

Code: Select all

(define (inc-symbol x y)
    (inc x))

(set 'y 200)
(inc-symbol 'y 123) => 246

y                   => 999   ; y is still 999

Not tryin' to pick on you today!

[Lutz]

The more corrections, the better ;)

Lutz

[newdep]

in 8.8.0 manual (webpage) the "Co" index points to 'cons instead of
'Command-line

http://www.newlisp.org/downloads/newlis ... .html#cons

[Sammo]

May I offer a corrected entry for "real-path" in the manual?
-- Sam

<a NAME="real-path"></a>
<h2><font color="#DD0000">real-path</font></h2>
<b>syntax: (real-path [<i>str-path</i>])</b>

<p>Returns the full path from a relative file path given in <i>str-path</i>.
If path is not given, <tt>"."</tt> (for the current directory) is assumed. </p>

<b>example:</b>
<blockquote>
<pre>
(real-path) => "/usr/home/fred" ; current directory
(real-path "./somefile.txt") => "/usr/home/fred/somefile.txt"
</pre>
</blockquote>

<p>The output length is limited by the OS's maximum allowed path length. If
<tt>real-path</tt> fails, e.g., because of a nonexistent path, <tt>nil</tt> is returned.</p>

[Lutz]

Thanks Sam, for correcting this!

Lutz

[HPW]

8.8.9 doc has 3 occurance of 'default functors' where it should be 'default function' (I think)

[m i c h a e l]

8.8.9 doc has 3 occurance of 'default functors' where it should be 'default function' (I think)

I looked this up in Wikipedia, as I, too, was unsure whether "functors" was correct. My reading led me to believe that "functor" (i.e., a function object) is likely what Lutz is specifying here. We'll see what Lutz says.

m i c h a e l

[m i c h a e l]

3 occurance of 'default functors' where it should be 'default function'

Just got to the part in the manual you reference and you're right. It should be default function and not default functor. I confused an earlier mention of 'functor' for your corrections. Thanks for pointing these out, Hans-Peter.

m i c h a e l

[Lutz]

Both 'function' and 'functor' should be used. Some times the default form of 'foo' for 'foo:foo' is not used as a function but rather as a handle for a data object in those cases it is called the 'default functor'. Use 'default function' only when meaning an executable function. Use 'default functor' when referring to it as a data object.

Lutz

[Lutz]

... sometimes the distinction is blurry:

Code: Select all

(set 'foo:foo '(a b c d e f g))

(foo 3) => d

in this case of implicit indexing the 'default functor' referring to a data object is used as a function overloaded with 'nth' functionality. So in this case its a little bit of both a 'functor' and a 'function'. But also in this case I would prefer to call it 'functor'.

Lutz

[m i c h a e l]

Here are the three places in the manual (before the function reference) where the term "functor" is used:

Like Scheme, newLISP also evaluates the <b>functor</b> part of an expression
Using default <b>functors</b> can simplify the syntax further
;; pass an object by reference with default <b>functor</b>

Based what Lutz has said in the preceding two posts, it seems that <i>all</i> references to functor here should stay as is.

m i c h a e l

[newdep]

Hi Lutz,

Still I found a small issue in the menual of 8.9.0 i could not fix right away...

I picked up my linux machine since 2 months and downloaded 890
opened the manual in FireFox 1.5.0.3 and it seems its unable
to handle the following quotes in HTML...

<span class="arw">&rArr;</span>

This generated a very strange charater instead of an arrow, changing
the document language or page codig did not work..its odd...

Any idea? because i think <span class="arw">&rArr;</span> is just
an official quote.. and under windows it works..

PS: I dont use any default language settnigs on my linux machine,
its pure a default rollout...


Norman.

[m i c h a e l]

Hi Norman,

Thanks for mentioning this. Seems you became a bit of a guinea pig here ;-)

I now see that &rArr; is not supported for all users. We could try &rarr;, &raquo;, or &rsaquo;, as these are all more openly adopted by browsers. Lutz's call.

As the manual is part of someone's first impression of newLISP, it can't be the first thing to cause them a problem! This one's mine, again. Sorry Lutz.

m i c h a e l

[newdep]

aaahh im always the little pig that has to deal with the wolf ;-)

The &rarr;, &raquo;, or &rsaquo; all work fine here..

Personaly i prefer the &rarr; but the &rsaquo; is more newlisp prompt like..

Norman.

[cormullion]

Reading the fine manual - a few minor typos and infelicities.

Section 18: UTF

date converts date number to string (when used with the third argument)/td>

Syntax section

context

A An expression evaluationg to a context or namespace, or a variable symbol holding a context.

multiply multiplies to matrices

bayes-train counts items in lists for Bayesian or frequentist analysis

list? checka if expression is a list
macro? checka if a lambda-macro expression
main-args geta command line arguments

process launches a child process, remap standard I/O and standrad error

Function section

amb

On of the expressions exp-1 ... n is selected at random and the evaluation result is returned.

append

In the first form append works on lists and all list-n ... are appended to form a new lisp. The original lists are unchanged.

append-file

If the file does not exists it gets created, in this case append-file works identical to write-file.

array

Arrays in newLISP are used whenever randomly indexing into a big list gets to slow. Only a subset of list functions can be used with arrays. For a more detailed discussion see the chapter about arrays in the userers manual.

If not enough initializers are found in the list than the list is repeated until all elements of the array are initialized.

atan

The arctangent function of num-radians is calculated an returned.

bayes-query

in the default R.A. Fisher method priors are not not passed on via chaining, but probabilities are compounded using the CHi2 method.

bayes-train

Each token in D is a content addressable symbol in D containing a list of ferquencies of that token describing how often it occurs in each category. String tokens are prepended with an undersocre _ before converting them to a symbol in D. A total symbol is built containing the totals in each categorie. The symbol total should not be part of symbols passed in a categoy Mi.

Note that these examples are for demo purposes. In reality trainings sets may contain thousands or millions of words, especially when training natural language models. But small data sets may be used when then the frequency of symbols just describes already known proportions. In this case it may be better to describe the model data set explicitely without the bayes-train function:

define

The return value of define is the assigned lambda expression. When calling a user defined function, it's return value is the last expression evaluated in the function body.

lambda or fn expressions may be used by itself as an anonymous function without being defined into a symbol:

Trying to redefine ant protected symbol will cause an error message.

define-macro

New functions can be created which behave just like built-in functions delaying evaluation fo certain parameters or accessing parameters inside list expressions passed as parameters like the built-in functions for flow control.

format:

In newLISP format will automatically convert from inter to floating point numbers or from floating point to integer if the format strings requires it:

[cormullion]

continued....

Function reference

define-macro

A function with this chracteristic is called the default function.

A third possibility is too refer to passed parameters using args:

See also expand do do variable expansion explicit without evaluation of the expanded expression.

device

int may also be a the file handle previously obtained from opening a file using open.

dolist

Opionally a condition for early loop exit may be defined in exp-break.

An internal system variable $idx keeps track of the current offset into the list and can be accessed duwing execution of dolist:

do-while

If the evaluation of exp-condition is nil, then the do-while expression is finished, else the the expressions in body get evaluated again.

error-event

See also the also the function catch for a different possibility to handle errors and the function throw-error to throw user defined errors.

eval-string

If the string contains more then one expressions, the result of the last evaluation is returned.

exec

In this example cgiProc could be a cgi processor like Perl or newLISP receiving processing standard input, which would be supplied by a string contained in the variable query.

expand

expand reduces it's parameter list similar to apply.

factor

For numbers smaller than 2 or for numbers which are to big factor returns nil.

file?

The existence of a file does not imply anything about it's read or write permissions.

fn

The function is mapped on to a list of arguments usiung map. The seconnd example shows sorting strings by length.

for

Opionally a condition for early loop exit may be defined in exp-break.

get-string

When giving a string as an argument get-string will take it's address as the argument. Because get-string always breaks off at the first first \000- (zero) character it encounters, it can be used to get a string out of a buffer:

invert

The function will throw and error if the matrix cannot be inverted.

irr

The next line demonstrates the the relation between irr and npv.

In the last example securities where purchased initially for 5000 then for another 2000 3 month later

legal?

The token in string is checked if a legal symbol in newLISP source code. Non legal symbols can be created using the sym function, i.e. symbols containing spaces, quotes or other characters not allowed. Non legal symbols are created frequently when using symbols for for associative data access:

letex

The function args is used to access the un-evalutuated argument list from define-macro.

load

By default files which do not contain context switches with get loaded in to the MAIN context.

member

(member "" "newLISP") ? "newLISP")

name

If the expression in bool evaluates to anything else than nil the name of the context, where the symbol belongs to, is returned.

net-connect

net-select or net-peek can be used to check for incoming commications in a non-blocking fashion.

net-eval

The following example illustrates the difference between the default evaluated and raw mode or net-eval:

The last to statements reveal that in the default evaluated

The example waits for a connections on port 1234,

new-listen

Optionally an interface IP address or name can be specified in str-ip-addr to listen on a specified address on computers with more then one interface card.

See the examples: UDP client and UDP server in the manuals appendix for a working example.

net-ping

Optionally a timeout parameter to wait for an answer can can be specified in int-timeout in milli seconds.

In the seconds syntax net-ping is run in batch-mode.

Usage of this this parameter can speed up return from the function before the timeout specified.

net-ping

While in broadcast mode net-ping sends out only one packet, which is received by multiple addresses, batch-mode explicitely generates multiple packets, one each for a target address.

... to be continued

[cormullion]

I just realised that you might consider this post-release week to be the wrong time to send in a long list of minor corrections! Sorry!

It's just that I didn't have so much time earlier this month, and I spent some of the available time on the Introduction. Also, I gathered that the document was already being edited and didn't want to spend the time going through an older version or having to synchronise editing versions, etc.

[Lutz]

With Michael's help we will publish manual revisions frequently as they come available. The next will be this Sunday.

So just keep those manual bugs coming on this thread as they jump into your eyes and Michael will work them in to his convenience, and make sure to always be based on the last version published.

Lutz

[cormullion]

I've started so I'll finish:

net-receive

When calling gettime The program connects to port 13 of the server netcom.com.

net-receive-udp

On Win32 is more bytes arrive net-receive-udp returns nil.

new

The example refers to contexts in variables and merges context foo into into bar.

normal

In the first form normal returns a list if int-n of random continuously distributed floating point numbers with a mean of float-mean and a standard deviation of float-stdev.

nth-set

Sets the nth in int-nth element in a list

An index out of bound always picks the last or first

See also the set-nth, which works like nth-set but returns the whole changed expression instead of the replaced element and is slower when doing replacements in a larger lists or string buffers.

pack

The > and < specifiers ca be used to switch

post-url

As a last parameter an optional timeout value in milli seconds can be spcified in int-timeout

process

Not all applications usable interactive in a console are suited for remapping their standard I/O channels.

randomize

Use seed to seed the generator at for a different starting point.

read-buffer

If no news bytes have been read sym-buffer will contain nil.

read-key

The last example an be used to check return sequences from navigation and function keys.

replace

Setting the the option bit 0x8000 in in-option will force replace to replace only the first occurrence.

The last form of replace has only two arguments an expression expr and list This form removes all expr found in list.

save

When saving contexts system variables symbols and all other symbols who's name starts with the $ character are not saved.

semaphore

When making a call to a semaphore with a negative value in int-wait, which would try to decrement the semaphore beyond zero, the function call will block until an other process or thread signals the semaphore with a positive value in int-signal.

The -1 is trying to decrement the semaphore, which is not possible because it's value is already zero.

sequence

Generates a sequence of numbers from num-start to num-end and with a optional step size of num-step.

share

Therefore shared strings should be stored after other shared number fields or reside on there own shared memory page.

Instead of spcifying a symbol containing the signal handler a function can be spefied directly.

When importing kill make always sure it receives integer numbers for the signal number.

source

System symbols starting with the $ character are only serialized when mentioned explicitely.

sub

Any floating point calculation with NaNalso returns NaN.

time

In first the example 450 milliseconds were spent to evaluate (myprog x y z).

timer

With default option 0 real time is measured, 1 mesaures only the time the CPU spends processing in the process or thread owning the timer and 3 is a combination of both, called profiling time.

trace

If an expression occurs more then once in a function, always the first occurrence of in the executing function will be highlighted (bracketed).

trace-highlight

Optionally two more string can be specified for str-header str-footer which control the separator and prompt. There is a maximum of 15 characters for the header and 31 for the footer

The first example replace the default "#" with a ">>" and "<<".

unify

Unbound variables start with an upper-case character to distinguish them from symbols. unify returns nil when the unification process fails and returns a list of variable associations on success or and empty lists when no variables where bound, but the match was still successfull.

The above algorithm could be written differently by omitting expand in the definition of prove-rule and pass the environment e as an parameter to the unify and query functions.

unpack

The > and < specifiers ca be used to switch between little endian and big endian byte order when packing or unpacking:

utf8

Converts UCS-4 encoded Unicode of 4 byte integers per character in str into UTF-8 character strings in str to .

write-buffer

This method is much faster then using append when concatenating to a string in place.

xml-parse

When tags are translated into symbols using option 8, a context can be specified in sym-context if no context is specified all symbols will be created in the current context.

Note that using the option number 16 a @ symbol is added to attributes lists.

And that's that. Enough English, time for some newLISP... :-)