Page 1 of 1

about (directory)

Posted: Sun Jul 16, 2017 1:11 pm
by qinying
The Chinese file name cannot be displayed correctly.on my win7-64-bit computer.
it's a bug?

Re: about (directory)

Posted: Sun Jul 16, 2017 8:48 pm
by TedWalther
To help with debugging, can you run this newlisp code and paste the results?

Code: Select all

(define str->bytes (lambda (s) (unpack (dup "b" (length s)) s)))
(str->bytes "the real filename")
(str->bytes "the filename directory returns")
Don't cut and paste the code; when you run str->bytes function, then put in the different strings, the one you see in Windows Explorer, and the one that (directory) returns. Then we can compare the byte strings.

Re: about (directory)

Posted: Tue Jul 18, 2017 11:51 am
by qinying
so glad to receive the reply,thanks!

Code: Select all

> (define str->bytes (lambda (s) (unpack (dup "b" (length s)) s)))
(lambda (s) (unpack (dup "b" (length s)) s))
> (directory)
("." ".." "guiserver" "index.html" "newlisp.exe" "鏂板缓鏂囨湰鏂囨。.txt")

> (str->bytes "新建文本文档")
(208 194 189 168 206 196 177 190 206 196 181 181)
> (str->bytes "鏂板缓鏂囨湰鏂囨。")
(230 150 176 229 187 186 230 150 135 230 156 172 230 150 135 230 161 163)

Re: about (directory)

Posted: Tue Jul 18, 2017 5:32 pm
by TedWalther
Can you run

Code: Select all

(set-locale)
and tell us the results?

Re: about (directory)

Posted: Wed Jul 19, 2017 12:11 am
by qinying

Code: Select all

> (set-locale)
("Chinese (Simplified)_People's Republic of China.936" ".")

Re: about (directory)

Posted: Wed Jul 19, 2017 11:04 am
by ralph.ronnquist
Hmm, I get the following

Code: Select all

> (length "新建文本文档")
18
> (unpack (dup "b" 18) "新建文本文档")
(230 150 176 229 187 186 230 150 135 230 156 172 230 150 135 230 161 163)
I.e., my byte sequence for the first string (copy-and-paste from this forum) is the same as your byte sequence for the second string.

Code: Select all

> (length "鏂板缓鏂囨湰鏂囨。")
27
> (unpack (dup "b" 27) "鏂板缓鏂囨湰鏂囨。")
(233 143 130 230 157 191 231 188 147 233 143 130 229 155 168 230 185 176 233 143 130 229 155 168 227 128 130)
Though I have

Code: Select all

> (set-locale)
("en_AU.UTF-8" ".")
Not much help I'm afraid.

Re: about (directory)

Posted: Thu Jul 20, 2017 10:05 pm
by TedWalther
Might have something to do with locale; perhaps the output of (directory) is coming in the current locale, which may NOT be UTF-8? So, how do you convert from his locale encoding into UTF8? What text encoding is he using if not UTF8? Some sort of non-UTF multi-byte encoding.

Re: about (directory)

Posted: Thu Jul 20, 2017 10:11 pm
by TedWalther
So, the problem is his system is using code page 936 (Simplified Chinese), but newlisp is using UTF8. To convert between encodings perhaps I need to write some bindings for libiconv, libiconv can convert between different character encodings. But I don't see myself having time to write the bindings soon. If he is comfortable with the C interface for newlisp modules, he can install the iconv DLL and write the bindings and that will fix the problem.

Re: about (directory)

Posted: Sat Jul 22, 2017 12:22 am
by qinying
I'm new to programming and I can't solve this problem by myself, so I can only look forward to the new version

Re: about (directory)

Posted: Sat Jul 22, 2017 1:04 am
by qinying
try the non utf-8 version

Code: Select all

newLISP v.10.7.2 64-bit on Windows IPv4/6 libffi, options: newlisp -h

> (set-locale)
("C" ".")
> (directory)
("." ".." "newlisp.exe" "\208\194\189\168\206\196\177\190\206\196\181\181.txt")
> (println (last (directory)))
新建文本文档.txt
"\208\194\189\168\206\196\177\190\206\196\181\181.txt"
> (append-file (last (directory)) "中文可以吗?")
12
> (read-file (last (directory)))
"\214\208\206\196\191\201\210\212\194\240\163\191"
> (println (read-file (last (directory))))
中文可以吗?
"\214\208\206\196\191\201\210\212\194\240\163\191"
then I change the locale

Code: Select all

> (set-locale "Chinese (Simplified)_People's Republic of China.936" ".")
("Chinese (Simplified)_People's Republic of China.936" ".")
> (directory)
("." ".." "newlisp.exe" "新建文本文档.txt")
> (println (read-file (last (directory))))
中文可以吗?
"中文可以吗?"

Re: about (directory)

Posted: Sat Jul 22, 2017 6:38 am
by TedWalther
Does that mean the non-UTF8 version of newlisp is working for you?

Re: about (directory)

Posted: Sat Jul 22, 2017 4:38 pm
by qinying
TedWalther wrote:Does that mean the non-UTF8 version of newlisp is working for you?
Can only handle Chinese directory and file names,
can not parsing strings correctly