Two usernames can look identical yet compare unequal. One may contain a single accented code point; the other may contain a base letter followed by a combining mark. Unicode gives symbols code points, but visual similarity does not guarantee identical string contents. section 2.1.2
Unicode supports code points through U+10FFFF. Its UTF encodings turn those values into storage or transmission units: UTF-32 uses four bytes per code point; UTF-16 uses two or four bytes; UTF-8 uses one to four bytes. The compact forms avoid UTF-32’s four-byte cost for ordinary Latin text. section 2.1.2 section 2.3
One code point, two UTF-16 units
U+10437 becomes the UTF-16 surrogate pair 0xD801 0xDC37. Subtract the surrogate-range bases, combine the high part (0x400) and low part (0x37), then add 0x10000 to recover 0x10437. A JavaScript string is a sequence of 16-bit values, so its length need not count Unicode code points; those values need not even form valid UTF-16. section 2.1.2 section 2.3
Table 2.2 The UTF-8 Encoding
|
Codepoint Range
|
Byte 1
|
Byte 2
|
Byte 3
|
Byte 4
|
|
|
U+0000
|
U+007F
|
0yyyzzzz
|
|
|
|
|
U+0080
|
U+07FF
|
110xxxyy
|
10yyzzzz
|
|
|
|
U+0800
|
U+FFFF
|
1110wwww
|
10xxxxyy
|
10yyzzzz
|
|
|
U+010000
|
U+10FFFF
|
11110uvv
|
10vvwwww
|
10xxxxyy
|
10yyzzzz
|
The UTF-8 byte-pattern table shows why encoding a code point can take one, two, three, or four bytes. section 2.1.2
Normalize before comparing usernames
A duplicate-name check based on direct string equality can miss a visually identical username written with different combining characters. Normalize both the candidate and stored names before comparing them; normalizing only one side leaves the mismatch possible. section 2.1.2
Table 2.4 Unicode normalization forms
|
Form
|
Name
|
Description
|
Example Input
|
Result
|
Code Units
|
|
NFC
|
Canonical Composition
|
Decomposes then recombines
characters
|
café (e + ́)
|
café
|
U+0063 U+0061 U+0066 U+00E9
|
|
NFD
|
Canonical Decomposition
|
Separates base characters
from combining marks (like accents)
|
café
|
café (e + ́)
|
U+0063 U+0061 U+0066 U+0065
U+0301
|
|
NFKC
|
Compatibility Composition
|
Like NFC but also
normalizes variants
|
fi (ligature, one symbol)
|
fi (two symbols)
|
U+0066 U+0069
|
|
NFKD
|
Compatibility
Decomposition
|
Like NFD but also
normalizes variants
|
½ (one symbol)
|
1⁄2 (three symbols)
|
U+0031 U+2044 U+0032
|
The normalization table distinguishes canonical composition and decomposition from compatibility forms, using accented text, a ligature, and a fraction. section 2.1.2
For regular expressions involving surrogate pairs or Unicode properties, use the `u` flag. Without it, a match such as `/^./` treats an emoji’s surrogate pair as separate units. Keep encoding separate from string semantics: filenames can also fail when a runtime assumes UTF-8 for bytes produced in another encoding. section 2.1.2 section 2.3








