Characters in C source
`long année = 1990L; // Année de l'écriture de l'œuvre` uses non-ASCII characters in an identifier and a comment. Such code can be appropriate when the project's language suits its participants; difficulty accepting it may instead point to an environment or implementation limitation. section 9.3
Unicode, aligned here with ISO 10646, assigns code points to character concepts rather than to particular printed shapes. The same letter can have several glyphs, while characters from different scripts can look alike. Unicode also classifies characters and records relationships such as uppercase and lowercase forms. section 14.3
In C, distinguish the character from its representation:
- A multibyte string stores encoded bytes. Neither its array size nor `strlen` generally gives its number of characters. Parsing may require an `mbstate_t` conversion state and facilities from `<wchar.h>`. section 14.3
- A source file can contain a character directly, or use a universal character escape such as `\u2502` for a vertical bar. Whether direct multibyte text works as intended depends on the environment. section 14.3
- UTF-8, UTF-16 and UTF-32 are Unicode encodings. C provides prefixed literals such as `u8"text"`, `u"text"` and `U"text"` when an encoding must be specified rather than left to the locale-dependent environment. section 14.4
Table 14.4 Common correspondence between UTF and basic character types
| Literal | typedef |
Type | Encoding | Correspondence |
u8’a’ |
uchar8_t |
unsigned char |
UTF-8 | ASCII in char |
u’π’ |
uchar16_t |
uint_least16_t |
UTF-16 | |
U’𝒜’ |
uchar32_t |
uint_least32_t |
UTF-32 | " wc" in wchar_t |
u8"𝒜" |
uchar8_t[5] |
unsigned char[5] |
UTF-8 | " mbs" in char[] |
u"𝒜" |
uchar16_t[3] |
uint_least16_t[3] |
UTF-16 | |
U"𝒜" |
uchar32_t[2] |
uint_least32_t[2] |
UTF-32 | " wcs" in wchar_t[] |
The table compares UTF literal prefixes, element types and encodings. Its `𝒜` strings also show why one character does not always occupy one string element: UTF-16 uses two words for that code point, plus a terminating null element. section 14.4
TAKEAWAY: Do not equate a Unicode code point, a glyph, a byte or a C string element. Check the encoding used by source files, text files and terminals before moving text between environments; they are not guaranteed to agree. section 14.3 section 14.4
