about (directory)

[qinying]

The Chinese file name cannot be displayed correctly.on my win7-64-bit computer.
it's a bug?

[TedWalther]

To help with debugging, can you run this newlisp code and paste the results?

Code: Select all

(define str->bytes (lambda (s) (unpack (dup "b" (length s)) s)))
(str->bytes "the real filename")
(str->bytes "the filename directory returns")

Don't cut and paste the code; when you run str->bytes function, then put in the different strings, the one you see in Windows Explorer, and the one that (directory) returns. Then we can compare the byte strings.

[qinying]

so glad to receive the reply,thanks!

Code: Select all

> (define str->bytes (lambda (s) (unpack (dup "b" (length s)) s)))
(lambda (s) (unpack (dup "b" (length s)) s))
> (directory)
("." ".." "guiserver" "index.html" "newlisp.exe" "鏂板缓鏂囨湰鏂囨。.txt")

> (str->bytes "新建文本文档")
(208 194 189 168 206 196 177 190 206 196 181 181)
> (str->bytes "鏂板缓鏂囨湰鏂囨。")
(230 150 176 229 187 186 230 150 135 230 156 172 230 150 135 230 161 163)

[TedWalther]

Can you run

Code: Select all

(set-locale)

and tell us the results?

[qinying]

Code: Select all

> (set-locale)
("Chinese (Simplified)_People's Republic of China.936" ".")

[ralph.ronnquist]

Hmm, I get the following

Code: Select all

> (length "新建文本文档")
18
> (unpack (dup "b" 18) "新建文本文档")
(230 150 176 229 187 186 230 150 135 230 156 172 230 150 135 230 161 163)

I.e., my byte sequence for the first string (copy-and-paste from this forum) is the same as your byte sequence for the second string.

Code: Select all

> (length "鏂板缓鏂囨湰鏂囨。")
27
> (unpack (dup "b" 27) "鏂板缓鏂囨湰鏂囨。")
(233 143 130 230 157 191 231 188 147 233 143 130 229 155 168 230 185 176 233 143 130 229 155 168 227 128 130)

Though I have

Code: Select all

> (set-locale)
("en_AU.UTF-8" ".")

Not much help I'm afraid.

[TedWalther]

Might have something to do with locale; perhaps the output of (directory) is coming in the current locale, which may NOT be UTF-8? So, how do you convert from his locale encoding into UTF8? What text encoding is he using if not UTF8? Some sort of non-UTF multi-byte encoding.

[TedWalther]

So, the problem is his system is using code page 936 (Simplified Chinese), but newlisp is using UTF8. To convert between encodings perhaps I need to write some bindings for libiconv, libiconv can convert between different character encodings. But I don't see myself having time to write the bindings soon. If he is comfortable with the C interface for newlisp modules, he can install the iconv DLL and write the bindings and that will fix the problem.

[qinying]

I'm new to programming and I can't solve this problem by myself, so I can only look forward to the new version

[qinying]

try the non utf-8 version

Code: Select all

newLISP v.10.7.2 64-bit on Windows IPv4/6 libffi, options: newlisp -h

> (set-locale)
("C" ".")
> (directory)
("." ".." "newlisp.exe" "\208\194\189\168\206\196\177\190\206\196\181\181.txt")
> (println (last (directory)))
新建文本文档.txt
"\208\194\189\168\206\196\177\190\206\196\181\181.txt"
> (append-file (last (directory)) "中文可以吗？")
12
> (read-file (last (directory)))
"\214\208\206\196\191\201\210\212\194\240\163\191"
> (println (read-file (last (directory))))
中文可以吗？
"\214\208\206\196\191\201\210\212\194\240\163\191"

then I change the locale

Code: Select all

> (set-locale "Chinese (Simplified)_People's Republic of China.936" ".")
("Chinese (Simplified)_People's Republic of China.936" ".")
> (directory)
("." ".." "newlisp.exe" "新建文本文档.txt")
> (println (read-file (last (directory))))
中文可以吗？
"中文可以吗？"

[TedWalther]

Does that mean the non-UTF8 version of newlisp is working for you?

[qinying]

Does that mean the non-UTF8 version of newlisp is working for you?

Can only handle Chinese directory and file names,
can not parsing strings correctly