[phpBB Debug] PHP Warning: in file [ROOT]/includes/bbcode.php on line 488: preg_replace(): The /e modifier is no longer supported, use preg_replace_callback instead
[phpBB Debug] PHP Warning: in file [ROOT]/includes/bbcode.php on line 488: preg_replace(): The /e modifier is no longer supported, use preg_replace_callback instead
[phpBB Debug] PHP Warning: in file [ROOT]/includes/bbcode.php on line 488: preg_replace(): The /e modifier is no longer supported, use preg_replace_callback instead
[phpBB Debug] PHP Warning: in file [ROOT]/includes/bbcode.php on line 488: preg_replace(): The /e modifier is no longer supported, use preg_replace_callback instead
[phpBB Debug] PHP Warning: in file [ROOT]/includes/bbcode.php on line 488: preg_replace(): The /e modifier is no longer supported, use preg_replace_callback instead
[phpBB Debug] PHP Warning: in file [ROOT]/includes/bbcode.php on line 488: preg_replace(): The /e modifier is no longer supported, use preg_replace_callback instead
[phpBB Debug] PHP Warning: in file [ROOT]/includes/bbcode.php on line 488: preg_replace(): The /e modifier is no longer supported, use preg_replace_callback instead
[phpBB Debug] PHP Warning: in file [ROOT]/includes/bbcode.php on line 488: preg_replace(): The /e modifier is no longer supported, use preg_replace_callback instead
[phpBB Debug] PHP Warning: in file [ROOT]/includes/bbcode.php on line 488: preg_replace(): The /e modifier is no longer supported, use preg_replace_callback instead
[phpBB Debug] PHP Warning: in file [ROOT]/includes/bbcode.php on line 488: preg_replace(): The /e modifier is no longer supported, use preg_replace_callback instead
[phpBB Debug] PHP Warning: in file [ROOT]/includes/bbcode.php on line 488: preg_replace(): The /e modifier is no longer supported, use preg_replace_callback instead
[phpBB Debug] PHP Warning: in file [ROOT]/includes/bbcode.php on line 488: preg_replace(): The /e modifier is no longer supported, use preg_replace_callback instead
[phpBB Debug] PHP Warning: in file [ROOT]/includes/bbcode.php on line 488: preg_replace(): The /e modifier is no longer supported, use preg_replace_callback instead
[phpBB Debug] PHP Warning: in file [ROOT]/includes/bbcode.php on line 488: preg_replace(): The /e modifier is no longer supported, use preg_replace_callback instead
[phpBB Debug] PHP Warning: in file [ROOT]/includes/bbcode.php on line 488: preg_replace(): The /e modifier is no longer supported, use preg_replace_callback instead
[phpBB Debug] PHP Warning: in file [ROOT]/includes/bbcode.php on line 488: preg_replace(): The /e modifier is no longer supported, use preg_replace_callback instead
[phpBB Debug] PHP Warning: in file [ROOT]/includes/bbcode.php on line 488: preg_replace(): The /e modifier is no longer supported, use preg_replace_callback instead
[phpBB Debug] PHP Warning: in file [ROOT]/includes/bbcode.php on line 488: preg_replace(): The /e modifier is no longer supported, use preg_replace_callback instead
[phpBB Debug] PHP Warning: in file [ROOT]/includes/bbcode.php on line 488: preg_replace(): The /e modifier is no longer supported, use preg_replace_callback instead
[phpBB Debug] PHP Warning: in file [ROOT]/includes/bbcode.php on line 488: preg_replace(): The /e modifier is no longer supported, use preg_replace_callback instead
[phpBB Debug] PHP Warning: in file [ROOT]/includes/bbcode.php on line 488: preg_replace(): The /e modifier is no longer supported, use preg_replace_callback instead
[phpBB Debug] PHP Warning: in file [ROOT]/includes/bbcode.php on line 488: preg_replace(): The /e modifier is no longer supported, use preg_replace_callback instead
[phpBB Debug] PHP Warning: in file [ROOT]/includes/bbcode.php on line 488: preg_replace(): The /e modifier is no longer supported, use preg_replace_callback instead
[phpBB Debug] PHP Warning: in file [ROOT]/includes/bbcode.php on line 488: preg_replace(): The /e modifier is no longer supported, use preg_replace_callback instead
[phpBB Debug] PHP Warning: in file [ROOT]/includes/bbcode.php on line 488: preg_replace(): The /e modifier is no longer supported, use preg_replace_callback instead
[phpBB Debug] PHP Warning: in file [ROOT]/includes/bbcode.php on line 488: preg_replace(): The /e modifier is no longer supported, use preg_replace_callback instead
[phpBB Debug] PHP Warning: in file [ROOT]/includes/bbcode.php on line 112: preg_replace(): The /e modifier is no longer supported, use preg_replace_callback instead
[phpBB Debug] PHP Warning: in file [ROOT]/includes/bbcode.php on line 112: preg_replace(): The /e modifier is no longer supported, use preg_replace_callback instead
[phpBB Debug] PHP Warning: in file [ROOT]/includes/bbcode.php on line 112: preg_replace(): The /e modifier is no longer supported, use preg_replace_callback instead
[phpBB Debug] PHP Warning: in file [ROOT]/includes/bbcode.php on line 112: preg_replace(): The /e modifier is no longer supported, use preg_replace_callback instead
[phpBB Debug] PHP Warning: in file [ROOT]/includes/bbcode.php on line 112: preg_replace(): The /e modifier is no longer supported, use preg_replace_callback instead
[phpBB Debug] PHP Warning: in file [ROOT]/includes/bbcode.php on line 112: preg_replace(): The /e modifier is no longer supported, use preg_replace_callback instead
[phpBB Debug] PHP Warning: in file [ROOT]/includes/bbcode.php on line 112: preg_replace(): The /e modifier is no longer supported, use preg_replace_callback instead
[phpBB Debug] PHP Warning: in file [ROOT]/includes/bbcode.php on line 112: preg_replace(): The /e modifier is no longer supported, use preg_replace_callback instead
[phpBB Debug] PHP Warning: in file [ROOT]/includes/bbcode.php on line 112: preg_replace(): The /e modifier is no longer supported, use preg_replace_callback instead
[phpBB Debug] PHP Warning: in file [ROOT]/includes/bbcode.php on line 112: preg_replace(): The /e modifier is no longer supported, use preg_replace_callback instead
[phpBB Debug] PHP Warning: in file [ROOT]/includes/bbcode.php on line 112: preg_replace(): The /e modifier is no longer supported, use preg_replace_callback instead
[phpBB Debug] PHP Warning: in file [ROOT]/includes/bbcode.php on line 112: preg_replace(): The /e modifier is no longer supported, use preg_replace_callback instead
[phpBB Debug] PHP Warning: in file [ROOT]/includes/bbcode.php on line 112: preg_replace(): The /e modifier is no longer supported, use preg_replace_callback instead
[phpBB Debug] PHP Warning: in file [ROOT]/includes/bbcode.php on line 112: preg_replace(): The /e modifier is no longer supported, use preg_replace_callback instead
[phpBB Debug] PHP Warning: in file [ROOT]/includes/bbcode.php on line 112: preg_replace(): The /e modifier is no longer supported, use preg_replace_callback instead
[phpBB Debug] PHP Warning: in file [ROOT]/includes/bbcode.php on line 112: preg_replace(): The /e modifier is no longer supported, use preg_replace_callback instead
[phpBB Debug] PHP Warning: in file [ROOT]/includes/bbcode.php on line 112: preg_replace(): The /e modifier is no longer supported, use preg_replace_callback instead
[phpBB Debug] PHP Warning: in file [ROOT]/includes/bbcode.php on line 112: preg_replace(): The /e modifier is no longer supported, use preg_replace_callback instead
[phpBB Debug] PHP Warning: in file [ROOT]/includes/bbcode.php on line 112: preg_replace(): The /e modifier is no longer supported, use preg_replace_callback instead
[phpBB Debug] PHP Warning: in file [ROOT]/includes/bbcode.php on line 112: preg_replace(): The /e modifier is no longer supported, use preg_replace_callback instead
[phpBB Debug] PHP Warning: in file [ROOT]/includes/bbcode.php on line 112: preg_replace(): The /e modifier is no longer supported, use preg_replace_callback instead
[phpBB Debug] PHP Warning: in file [ROOT]/includes/bbcode.php on line 112: preg_replace(): The /e modifier is no longer supported, use preg_replace_callback instead
[phpBB Debug] PHP Warning: in file [ROOT]/includes/bbcode.php on line 112: preg_replace(): The /e modifier is no longer supported, use preg_replace_callback instead
[phpBB Debug] PHP Warning: in file [ROOT]/includes/bbcode.php on line 112: preg_replace(): The /e modifier is no longer supported, use preg_replace_callback instead
[phpBB Debug] PHP Warning: in file [ROOT]/includes/bbcode.php on line 112: preg_replace(): The /e modifier is no longer supported, use preg_replace_callback instead
[phpBB Debug] PHP Warning: in file [ROOT]/includes/functions.php on line 4762: Cannot modify header information - headers already sent by (output started at [ROOT]/includes/functions.php:3897)
[phpBB Debug] PHP Warning: in file [ROOT]/includes/functions.php on line 4764: Cannot modify header information - headers already sent by (output started at [ROOT]/includes/functions.php:3897)
[phpBB Debug] PHP Warning: in file [ROOT]/includes/functions.php on line 4765: Cannot modify header information - headers already sent by (output started at [ROOT]/includes/functions.php:3897)
[phpBB Debug] PHP Warning: in file [ROOT]/includes/functions.php on line 4766: Cannot modify header information - headers already sent by (output started at [ROOT]/includes/functions.php:3897)
newlispfanclub.alh.net • View topic - "save" makes improper [text] strings; impossible to reload

"save" makes improper [text] strings; impossible to reload

Q&A's, tips, howto's

"save" makes improper [text] strings; impossible to reload

Postby TedWalther » Thu Aug 20, 2015 12:29 am

Cavemen in bearskins invaded the ivory towers of Artificial Intelligence. Nine months later, they left with a baby named newLISP. The women of the ivory towers wept and wailed. "Abomination!" they cried.
TedWalther
 
Posts: 605
Joined: Mon Feb 05, 2007 1:04 am
Location: Abbotsford, BC

Re: "save" makes improper [text] strings; impossible to relo

Postby TedWalther » Thu Aug 20, 2015 1:05 am

As a temporary workaround, before I ever "save" a string, I do (replace {]} my_string {\]}). And then after saving and loading it, I reverse it with (replace {\]} my_string {]})
Cavemen in bearskins invaded the ivory towers of Artificial Intelligence. Nine months later, they left with a baby named newLISP. The women of the ivory towers wept and wailed. "Abomination!" they cried.
TedWalther
 
Posts: 605
Joined: Mon Feb 05, 2007 1:04 am
Location: Abbotsford, BC

Re: "save" makes improper [text] strings; impossible to relo

Postby Lutz » Thu Aug 20, 2015 1:40 pm

The distinction of shorter or bigger-equal than 2048 is done for speed. The internal string representation is the same. What differs is the method used for reading strings from source. For strings shorter than 2048 limited by “” or {}, allocated buffers or the stack and one read() is used. On output newLISP looks at the length than chooses the "" versus [text][/tags]. For longer strings stream reading is used. The short kind, the most used in program source, can also be handled fast on the C stack, while the long cannot.

But the three sets of delimiters also offer the programmer different ways of encoding strings. E.g. for web work [text][/text] are very convenient because they take text as is including line feeds. {} are useful for regular expression for avoiding double backslashing.
Lutz
 
Posts: 5279
Joined: Thu Sep 26, 2002 4:45 pm
Location: Pasadena, California

Re: "save" makes improper [text] strings; impossible to relo

Postby TedWalther » Fri Aug 21, 2015 7:35 pm

Lutz, can you change things so that "save" function works? Is save much good if you can't "load" the output?
Cavemen in bearskins invaded the ivory towers of Artificial Intelligence. Nine months later, they left with a baby named newLISP. The women of the ivory towers wept and wailed. "Abomination!" they cried.
TedWalther
 
Posts: 605
Joined: Mon Feb 05, 2007 1:04 am
Location: Abbotsford, BC

Re: "save" makes improper [text] strings; impossible to relo

Postby Lutz » Sun Aug 23, 2015 11:18 pm

Lutz
 
Posts: 5279
Joined: Thu Sep 26, 2002 4:45 pm
Location: Pasadena, California

Re: "save" makes improper [text] strings; impossible to relo

Postby hartrock » Fri Sep 04, 2015 10:15 pm

hartrock
 
Posts: 136
Joined: Wed Aug 07, 2013 9:37 pm

Re: "save" makes improper [text] strings; impossible to relo

Postby TedWalther » Fri Sep 04, 2015 11:04 pm

I was thinking, from point of view of correctness, easiest just to have "" for strings. Go language has `` for unescaped strings. {} strings are convenient, but can sometimes bite that they can only contain balanced parenthesis. I think overall, if "" worked for any size, I'd rarely use the other kinds, especially if "" strings allow literal characters inside, so I can use them the same was as [text] and {} for multiline strings.

It is a huge "gotcha" to have a string type (two string types) that work only on some content, but don't have any escape mechanism to include all content. And then the one type of string that does allow all content, is limited to 2048 bytes. Aaarrghghh!

Lutz, how about this; when doing "save", if the string is longer than 2048 bytes, instead of converting it to [text], convert it to (list num num num) where each num is a byte, value in the range 0..255 Or else (string str1 str2...) where str1 is a 2048 byte string in "" representation, as is str2, up until all the content is represented.
Cavemen in bearskins invaded the ivory towers of Artificial Intelligence. Nine months later, they left with a baby named newLISP. The women of the ivory towers wept and wailed. "Abomination!" they cried.
TedWalther
 
Posts: 605
Joined: Mon Feb 05, 2007 1:04 am
Location: Abbotsford, BC

Re: "save" makes improper [text] strings; impossible to relo

Postby Lutz » Sat Sep 05, 2015 3:00 am

In Hartrock’s example the \000 characters are still in sb but text in [text],[/text] tags does not escape characters, so \000 is not shown, although part of sb.

Right now in 10.6.4 (in progress) base64 encoding is used for strings > 2047 characters and containing the [/text] tag when using save. I could drop the condition for [/text] and always use base64 for strings longer 2047 in the save function. That would make Hartrock’s sb string usable for the save function.

But I don’t want to give up the [text],[/tags] to use completely unescaped text (not binary content). This is frequently used in web programming. I also don’t want to eliminate the 2047 limit for quoted string for speed in processing.

The need to display code > 2047 and containing non-displayable binary info is very rare. The save now will work with binary contents too if it always uses base64 transformation on strings > 2047 characters.
Lutz
 
Posts: 5279
Joined: Thu Sep 26, 2002 4:45 pm
Location: Pasadena, California

Re: "save" makes improper [text] strings; impossible to relo

Postby TedWalther » Mon Sep 07, 2015 1:00 am

The base64 solution bothers me, because it breaks the human readability of the (save) output. Can you share a bit more info about the speedups you achieved with the three different string types? How about a 4th type of string, L" for "" strings longer than 2048 bytes?

update

Or instead of L"my string", do it Python style: """my string"""
Cavemen in bearskins invaded the ivory towers of Artificial Intelligence. Nine months later, they left with a baby named newLISP. The women of the ivory towers wept and wailed. "Abomination!" they cried.
TedWalther
 
Posts: 605
Joined: Mon Feb 05, 2007 1:04 am
Location: Abbotsford, BC

Re: "save" makes improper [text] strings; impossible to relo

Postby hartrock » Tue Sep 08, 2015 11:35 am

hartrock
 
Posts: 136
Joined: Wed Aug 07, 2013 9:37 pm

Re: "save" makes improper [text] strings; impossible to relo

Postby Lutz » Tue Sep 08, 2015 2:58 pm

Lutz
 
Posts: 5279
Joined: Thu Sep 26, 2002 4:45 pm
Location: Pasadena, California

Re: "save" makes improper [text] strings; impossible to relo

Postby TedWalther » Tue Sep 08, 2015 6:30 pm

Cavemen in bearskins invaded the ivory towers of Artificial Intelligence. Nine months later, they left with a baby named newLISP. The women of the ivory towers wept and wailed. "Abomination!" they cried.
TedWalther
 
Posts: 605
Joined: Mon Feb 05, 2007 1:04 am
Location: Abbotsford, BC

Re: "save" makes improper [text] strings; impossible to relo

Postby hartrock » Tue Sep 08, 2015 10:01 pm

hartrock
 
Posts: 136
Joined: Wed Aug 07, 2013 9:37 pm

two more ideas [Re: "save" makes improper [text] strings; im

Postby hartrock » Wed Sep 09, 2015 2:02 pm

hartrock
 
Posts: 136
Joined: Wed Aug 07, 2013 9:37 pm

Re: "save" makes improper [text] strings; impossible to relo

Postby Lutz » Wed Sep 09, 2015 8:22 pm

After doing a few more benchmarks, I stay with append, which is faster when appending more than a few strings. In the save case you have at least about 30 strings of about 72 characters each. extend does a realloc() on each string, while append allocates memory in bigger chunks.
Lutz
 
Posts: 5279
Joined: Thu Sep 26, 2002 4:45 pm
Location: Pasadena, California

Re: "save" makes improper [text] strings; impossible to relo

Postby hartrock » Thu Sep 10, 2015 2:19 am

hartrock
 
Posts: 136
Joined: Wed Aug 07, 2013 9:37 pm

[backtrace] newlisp-10.6.4.tgz 2015-09-09 20:27

Postby hartrock » Thu Sep 10, 2015 1:04 pm

Last edited by hartrock on Thu Sep 10, 2015 3:52 pm, edited 1 time in total.
hartrock
 
Posts: 136
Joined: Wed Aug 07, 2013 9:37 pm

Re: "save" makes improper [text] strings; impossible to relo

Postby Lutz » Thu Sep 10, 2015 1:53 pm

extend is destructive append is not. Many things in newLISP can be done in either a destructive or non-destructive manner and destructive functions are marked with an ! in the reference.

http://www.newlisp.org/downloads/newlis ... estructive

Most destructive functions - like extend too - can also do in-place modification. See the last sub-chapter about this on this page: http://www.newlisp.org/index.cgi?page=Closures

Bench-marking is a tricky business you can find some benchmarks comparing to Perl and Python here: http://newlisp.nfshost.com/benchmarks/
Small changes how a function is used may cause big differences in benchmarks. Sometimes just moving between OSs or versions of compilers changes everything. That is, why I like the benchmarks in the link, because they always measure function performance in an application context.

ps: people report errors under all topics of the forum, and that has worked well in the passed times.
Lutz
 
Posts: 5279
Joined: Thu Sep 26, 2002 4:45 pm
Location: Pasadena, California

Re: "save" makes improper [text] strings; impossible to relo

Postby hartrock » Thu Sep 10, 2015 3:55 pm

hartrock
 
Posts: 136
Joined: Wed Aug 07, 2013 9:37 pm

Re: "save" makes improper [text] strings; impossible to relo

Postby hartrock » Thu Sep 10, 2015 4:13 pm

hartrock
 
Posts: 136
Joined: Wed Aug 07, 2013 9:37 pm

Re: "save" makes improper [text] strings; impossible to relo

Postby xytroxon » Thu Sep 10, 2015 11:51 pm

"Many computers can print only capital letters, so we shall not use lowercase letters."
-- Let's Talk Lisp (c) 1976
xytroxon
 
Posts: 296
Joined: Tue Nov 06, 2007 3:59 pm

Re: "save" makes improper [text] strings; impossible to relo

Postby hartrock » Fri Sep 11, 2015 12:43 am

hartrock
 
Posts: 136
Joined: Wed Aug 07, 2013 9:37 pm

[backtrace] newlisp-10.6.4.tgz 2015-09-10 14:11

Postby hartrock » Fri Sep 11, 2015 1:00 am

hartrock
 
Posts: 136
Joined: Wed Aug 07, 2013 9:37 pm

Re: "save" makes improper [text] strings; impossible to relo

Postby Lutz » Fri Sep 11, 2015 1:56 pm

Try to boil down the code to the least amount needed to produce the error. Back-tracing does not help here.
Lutz
 
Posts: 5279
Joined: Thu Sep 26, 2002 4:45 pm
Location: Pasadena, California

Re: "save" makes improper [text] strings; impossible to relo

Postby hartrock » Fri Sep 11, 2015 2:38 pm

hartrock
 
Posts: 136
Joined: Wed Aug 07, 2013 9:37 pm

Next

Return to newLISP in the real world

Who is online

Users browsing this forum: No registered users and 2 guests