about (directory)

For the Compleat Fan

about (directory)

Postby qinying » Sun Jul 16, 2017 1:11 pm

The Chinese file name cannot be displayed correctly.on my win7-64-bit computer.
it's a bug?
Attachments
QQ截图20170716210226.png
QQ截图20170716210226.png (75.28 KiB) Viewed 1681 times
qinying
 
Posts: 7
Joined: Sun Jul 16, 2017 11:12 am

Re: about (directory)

Postby TedWalther » Sun Jul 16, 2017 8:48 pm

To help with debugging, can you run this newlisp code and paste the results?

Code: Select all
(define str->bytes (lambda (s) (unpack (dup "b" (length s)) s)))
(str->bytes "the real filename")
(str->bytes "the filename directory returns")


Don't cut and paste the code; when you run str->bytes function, then put in the different strings, the one you see in Windows Explorer, and the one that (directory) returns. Then we can compare the byte strings.
Cavemen in bearskins invaded the ivory towers of Artificial Intelligence. Nine months later, they left with a baby named newLISP. The women of the ivory towers wept and wailed. "Abomination!" they cried.
TedWalther
 
Posts: 602
Joined: Mon Feb 05, 2007 1:04 am
Location: Abbotsford, BC

Re: about (directory)

Postby qinying » Tue Jul 18, 2017 11:51 am

so glad to receive the reply,thanks!
Code: Select all
> (define str->bytes (lambda (s) (unpack (dup "b" (length s)) s)))
(lambda (s) (unpack (dup "b" (length s)) s))
> (directory)
("." ".." "guiserver" "index.html" "newlisp.exe" "鏂板缓鏂囨湰鏂囨。.txt")

> (str->bytes "新建文本文档")
(208 194 189 168 206 196 177 190 206 196 181 181)
> (str->bytes "鏂板缓鏂囨湰鏂囨。")
(230 150 176 229 187 186 230 150 135 230 156 172 230 150 135 230 161 163)
qinying
 
Posts: 7
Joined: Sun Jul 16, 2017 11:12 am

Re: about (directory)

Postby TedWalther » Tue Jul 18, 2017 5:32 pm

Can you run
Code: Select all
(set-locale)
and tell us the results?
Cavemen in bearskins invaded the ivory towers of Artificial Intelligence. Nine months later, they left with a baby named newLISP. The women of the ivory towers wept and wailed. "Abomination!" they cried.
TedWalther
 
Posts: 602
Joined: Mon Feb 05, 2007 1:04 am
Location: Abbotsford, BC

Re: about (directory)

Postby qinying » Wed Jul 19, 2017 12:11 am

Code: Select all
> (set-locale)
("Chinese (Simplified)_People's Republic of China.936" ".")
qinying
 
Posts: 7
Joined: Sun Jul 16, 2017 11:12 am

Re: about (directory)

Postby ralph.ronnquist » Wed Jul 19, 2017 11:04 am

Hmm, I get the following
Code: Select all
> (length "新建文本文档")
18
> (unpack (dup "b" 18) "新建文本文档")
(230 150 176 229 187 186 230 150 135 230 156 172 230 150 135 230 161 163)
I.e., my byte sequence for the first string (copy-and-paste from this forum) is the same as your byte sequence for the second string.

Code: Select all
> (length "鏂板缓鏂囨湰鏂囨。")
27
> (unpack (dup "b" 27) "鏂板缓鏂囨湰鏂囨。")
(233 143 130 230 157 191 231 188 147 233 143 130 229 155 168 230 185 176 233 143 130 229 155 168 227 128 130)

Though I have
Code: Select all
> (set-locale)
("en_AU.UTF-8" ".")


Not much help I'm afraid.
ralph.ronnquist
 
Posts: 178
Joined: Mon Jun 02, 2014 1:40 am
Location: Melbourne, Australia

Re: about (directory)

Postby TedWalther » Thu Jul 20, 2017 10:05 pm

Might have something to do with locale; perhaps the output of (directory) is coming in the current locale, which may NOT be UTF-8? So, how do you convert from his locale encoding into UTF8? What text encoding is he using if not UTF8? Some sort of non-UTF multi-byte encoding.
Cavemen in bearskins invaded the ivory towers of Artificial Intelligence. Nine months later, they left with a baby named newLISP. The women of the ivory towers wept and wailed. "Abomination!" they cried.
TedWalther
 
Posts: 602
Joined: Mon Feb 05, 2007 1:04 am
Location: Abbotsford, BC

Re: about (directory)

Postby TedWalther » Thu Jul 20, 2017 10:11 pm

So, the problem is his system is using code page 936 (Simplified Chinese), but newlisp is using UTF8. To convert between encodings perhaps I need to write some bindings for libiconv, libiconv can convert between different character encodings. But I don't see myself having time to write the bindings soon. If he is comfortable with the C interface for newlisp modules, he can install the iconv DLL and write the bindings and that will fix the problem.
Cavemen in bearskins invaded the ivory towers of Artificial Intelligence. Nine months later, they left with a baby named newLISP. The women of the ivory towers wept and wailed. "Abomination!" they cried.
TedWalther
 
Posts: 602
Joined: Mon Feb 05, 2007 1:04 am
Location: Abbotsford, BC

Re: about (directory)

Postby qinying » Sat Jul 22, 2017 12:22 am

I'm new to programming and I can't solve this problem by myself, so I can only look forward to the new version
qinying
 
Posts: 7
Joined: Sun Jul 16, 2017 11:12 am

Re: about (directory)

Postby qinying » Sat Jul 22, 2017 1:04 am

try the non utf-8 version
Code: Select all
newLISP v.10.7.2 64-bit on Windows IPv4/6 libffi, options: newlisp -h

> (set-locale)
("C" ".")
> (directory)
("." ".." "newlisp.exe" "\208\194\189\168\206\196\177\190\206\196\181\181.txt")
> (println (last (directory)))
新建文本文档.txt
"\208\194\189\168\206\196\177\190\206\196\181\181.txt"
> (append-file (last (directory)) "中文可以吗?")
12
> (read-file (last (directory)))
"\214\208\206\196\191\201\210\212\194\240\163\191"
> (println (read-file (last (directory))))
中文可以吗?
"\214\208\206\196\191\201\210\212\194\240\163\191"


then I change the locale

Code: Select all
> (set-locale "Chinese (Simplified)_People's Republic of China.936" ".")
("Chinese (Simplified)_People's Republic of China.936" ".")
> (directory)
("." ".." "newlisp.exe" "新建文本文档.txt")
> (println (read-file (last (directory))))
中文可以吗?
"中文可以吗?"
qinying
 
Posts: 7
Joined: Sun Jul 16, 2017 11:12 am

Re: about (directory)

Postby TedWalther » Sat Jul 22, 2017 6:38 am

Does that mean the non-UTF8 version of newlisp is working for you?
Cavemen in bearskins invaded the ivory towers of Artificial Intelligence. Nine months later, they left with a baby named newLISP. The women of the ivory towers wept and wailed. "Abomination!" they cried.
TedWalther
 
Posts: 602
Joined: Mon Feb 05, 2007 1:04 am
Location: Abbotsford, BC

Re: about (directory)

Postby qinying » Sat Jul 22, 2017 4:38 pm

TedWalther wrote:Does that mean the non-UTF8 version of newlisp is working for you?


Can only handle Chinese directory and file names,
can not parsing strings correctly
qinying
 
Posts: 7
Joined: Sun Jul 16, 2017 11:12 am


Return to Anything else we might add?

Who is online

Users browsing this forum: No registered users and 3 guests

cron