Why can a symbol take different amounts of storage? A Unicode code point is the number assigned to a symbol; an encoding determines how that number is represented in bytes. For example, lowercase `a` has code point 97 (`0x61`), which UTF-32 stores as `0x00000061`. section 2.1.2
Table 2.1 Comparing the ASCII and UTF-32 encodings of the word hello
|
Method
|
Encoding
|
|
ASCII
|
0x68656c6c6f
|
|
UTF-32
|
0x00000068000000650000006c0000006c0000006f
|
The same word, `hello`, occupies five bytes in ASCII but twenty in UTF-32: each UTF-32 code point takes four bytes. section 2.1.2
UTF-16 and UTF-8 encode code points differently. UTF-16 can combine a high and low surrogate to recover a code point; UTF-8 uses one to four bytes according to the code point’s range. Do not treat a byte count as a count of code points. section 2.1.2
Recovering a code point
For a UTF-16 high surrogate of `0xD801` and low surrogate of `0xDC37`, calculate `(0xD801 − 0xD800) × 0x400 + (0xDC37 − 0xDC00) + 0x10000 = 0x10437`. The pair of code units represents one code point. section 2.1.2






