nL 9.3.5-development can crash your machine

[lithper]

[1] accidentally came across a sequence that immediately, explosively eats up all memory and CPU and creates effectively a DoS condition, killing the OS.

Code: Select all

> (symbols Aa)
(Aa:Aa)
> (Aa p (d))
()
> (Aa "p" (d))

invalid function : (d)

//CRASH!!//
Terminated
 ->

In fact, this incorrect attempt at putting something as value in hash will murder your machine in 3-5 seconds:

Code: Select all

 0  0 164184 225932   6932  49968    0    0     0     0 1014   162  0  1 99  0
 0  0 164184 225932   6932  49968    0    0     0     0 1011   159  0  0 100  0
 1  0 164184 205068   6932  49968    0    0     0     0 1010   162  6 12 82  0
 1  0 164184  80972   6932  49968    0    0     0     0 1002   125 35 65  0  0
 1  0 122820   3568   6932  49968    0    0     0     0 1003   203 34 66  0  0
 2  1 187264   2392    968   6936    0 70800     8 70800 1294   454 24 76  0  0
procs -----------memory---------- ---swap-- -----io---- --system-- ----cpu----
 r  b   swpd   free   buff  cache   si   so    bi    bo   in    cs us sy id wa
 4  4 236640   3372    980   6804   20 49376   108 49376 1298   528  5 50  0 45
 2  0 250240   2472    980   6804  220 13608   220 13608 1112   629  9 34  0 57
 0  1 267212   2384    980   6244    0 17160     0 17160 1029   283  7 43  0 50
 0  3 267784   1560    996   6304    0  896    72   896 1035   242  5 10  0 85
 0  3 267780   1292   1068   7700   96    0  1560     0 1128   293  0  3  0 97

CPU usage will jump to almost 100%, all memory will be eaten up, the machine will thrash the disk (swapping memory out), it will freeze

[2] A similar crash OR segmentation fault happens if after an incorrect operation a bracket is omitted (usually a benign error condition)

Code: Select all

 -> newlisp
newLISP v.9.3.5 on Linux IPv4 UTF-8, execute 'newlisp -h' for more info.

> (define Aa:Aa)
nil
> (Aa "f" (list p))
(nil)
> (symbols Aa

missing parenthesis : "...(symbols Aa                         "
Terminated
//CRASH!!//

Sometimes in these cases newlisp generated simple segmentation faults.

I do realize the shown usage is incorrect - only if your script generates it in some unexpected case, or if your program can be manipulated into doing something similar, a very serious condition can happen.

[3] It seems that new usage -- (define Aa:Aa), then (Aa "f" "asdf") or (Aa "f" 345) -- is narrower than the one before?
It seemed that with older usage I could avoid restriction on hash key as STRING ONLY, and on data as only STRING or NUMBER ?? Or am I mistaken and this limitation existed all the time ? (seems, not, as one of my older scripts refused to work with new notation without a typecast).

[newdep]

That is correct, there are some ways to eat all memory or cpu within newlisp,
although it is not seen as a bug...

(correct me if im wrong ;-)

[lithper]

although it is not seen as a bug...

Ouuuffff...

[start of rant]
If an application crashes - for example, when you use the interface for talking to libraries in C and use incorrect variable type - it is a security hole, it is something one should guard against in his code, but it is _marginally_ acceptable -- crashing your OS can not be construed as normal behaviour in any circumstances.

Please, take your head off the shelf, dust, and screw it back onto your neck, and say you have just made a bad joke. First of April, haha. Ha.

Newlisp is a security admin's worst nightmare - packing into one 200kB executable all tools a cracker dreamt of packing for a decade or three. Trivial to open sockets, execute whatever one wishes on the target by just starting t with a command-line switch, get raw access to memory, and crash it to get cores at your will.
It as if was specifically written to get reviewed in 2600, tongue-in-cheek, as a "nifty little language", wink-wink. Not a behemoth requiring effort to do an installation right, just one tiny file to drop some place on a machine
If someone makes a prog widely acceptable (say, as a web/CGI application), and ensures quietly that there's a backdoor in the executable, or ability to trigger one of its special behaviours...

I am ready to say, nL gives full freedom to hack, and therefore limits severely security in this inevitable balance - so one has to program with specific considerations and care - just because it's better to live in freedom and take responsibility rather than sit in one cage or another (they actually prefer you not to think of imprisonment, so they use imagery of infantility and talk about "sandboxes" instead), constructed by some inane "security specialist" (on the pay from NSA anyway, "cooperating" with Microsoft, Sun, or one more of the countless Linux distros they produce to achieve the corporate takeover of the thing, like Ubuntu or RedHat (can you believe that thier previous CEO had the last name of Жулик (Matthew Szulik)?!! - check a Polish/Russian-English dictionary on that).
But I am not prepared to make such pronouncements as yours, I am sorry to say, even on the first of April..
[end of rant]

[Lutz]

Expression evaluation for the value part causes the crashes when using the new hash functions. This will be fixed in development version 9.3.6.

Regarding security: Yes, newLISP gives the freedom and power to directly access most computer resources, and it does this in a small package. This is by design. As any powerful tool, it must be handled responsibly.

[cormullion]

Interesting bug - I presume this is in the new development version... It doesn't seem to happen in the current version of newLISP.

I often wonder if anyone has found a good way of running both the latest unstable development version and the current stable released version of newLISP side by side? I occasionally install the latest dev version, but feel much happier going back to the released version for daily work. Besides, Lutz has been known to take functions away, too... :)

Perhaps there should be a bit more of a warning - "this is a development warning which may contain bugs"...? Or is everybody just happily downloading and installing the latest dev versions regardless of what they might contain? Otherwise it would be difficult to test all development versions on all platforms for crashes etc before releasing them...

I can't see the real thrust of your entertaining rant, lithper - many reasonable points seem to bounce around without generating any cumulative effect. newLISP is too powerful? too small? Not secure enough? Too risky to install? Too unstable? Too suitable for hackers? Too easy to program in? All of the above, perhaps? In comparison with other languages, is it more powerful than it should be, smaller than it should be, less reliable..? Is it really impossible to write dangerous code in other languages?

Well, it should be said that it's not compulsory to use newLISP.

Or perhaps you're really just saying that it shouldn't crash your machine. I'd certainly agree with that. I don't use Microsoft Word for the same reason...!

[lithper]

Lutz - yes, thank you.'

cormullion - well, it just boils down to remembering what "development" version is. Changes in any sufficiently complex system can cause non-obvious consequences.

On the other hand, the whole point of open software development is that even if users do not contribute directly, you can always view their experiences as free testing, a fair exchange in some respects. We should not cry and complain, we are part of the process ;)

Regarding newLisp security.. Yes, I'd prefer to have a tool that can do everything, like a knife blade, rather than be "protected" from yourself, like with those plastic-embedded tiny strips of metal they call "safety razors".

But I'd use even small things to increase security of design. For example, I'd not put bare newLisp scripts into a web server cgi directory configuring it to execute newLisp directly.
I'd package the script+200kB into that fake executable, and drop the package into the web server.
It seems (seems, I'd need to check it better), that a packaged newLisp script+binary executes only the included script and does not react to any other command-line options, environment variables, or scripts altogether.
While dropping a bare script might open the setup to a variety of abuses along these lines.

And, secondly, I'd be very restrictive about the input values - no symbols besides alphanumeric etc - and about their pathways inside the program - will some input characters be treated as strings? symbols? form lists?

Same about the variables/symbols introduced inside the prog itself. Can I in some error conditions (e.g. a file missing etc.) create a situation when things will go wrong - or have I caught it early as exceptions?

These are generally known principles, of course, the difficulty is in seeing how this actually operates in the program.

Some langs tried to automate the process ("taint" input in Perl - generally systems of constraints in languages and how they propagate), but..
And some people tried to write automated checking programs, and it might be they wrote them also in Lisp ;))

[cormullion]

It seems (seems, I'd need to check it better), that a packaged newLisp script+binary executes only the included script and does not react to any other command-line options, environment variables, or scripts altogether.
While dropping a bare script might open the setup to a variety of abuses along these lines.

I've wondered about a 'safe' option for newLISP, a bit like the -http option. Making an online newLISP interpreter such as the Lambdalator was quite interesting without one...

[tom]

Would the NOCMD option contribute to a safer newlisp?