Page 1 of 1
regex doesn't match utf-8 character using \w pattern
Posted: Fri Jan 27, 2017 8:37 pm
by vetelko
Hello newLISP-ers :)
the following code returns nil for utf-8 character
(regex {\w} "č")
and so the following
(regex {\w} "č" "u")
while this works
(regex {\w} "m")
Is this not implemented or am I doing something wrong?
VT
Re: regex doesn't match utf-8 character using \w pattern
Posted: Fri Jan 27, 2017 9:26 pm
by ralph.ronnquist
I believe the magic of regex is explained in
http://www.newlisp.org/downloads/pcrepattern.html, where you find the two paragraphs:
------
A "word" character is an underscore or any character less than 256 that is a letter or digit. The definition of letters and digits is controlled by PCRE's low-valued character tables, and may vary if locale-specific matching is taking place (see "Locale support" in the pcreapi page). For example, in the "fr_FR" (French) locale, some character codes greater than 128 are used for accented letters, and these are matched by \w.
In UTF-8 mode, characters with values greater than 128 never match \d, \s, or \w, and always match \D, \S, and \W. This is true even when Unicode character property support is available. The use of locales with Unicode is discouraged.
------
Re: regex doesn't match utf-8 character using \w pattern
Posted: Sat Jan 28, 2017 11:02 am
by vetelko
So there is no way how to match this character as a word character using pattern?