liveBook · Manning

Code point

As explained in JavaScript in Depth

An integer value assigned to a specific abstract character or symbol in a character encoding standard.

Why can a symbol take different amounts of storage? A Unicode code point is the number assigned to a symbol; an encoding determines how that number is represented in bytes. For example, lowercase `a` has code point 97 (`0x61`), which UTF-32 stores as `0x00000061`. section 2.1.2

Table 2.1 Comparing the ASCII and UTF-32 encodings of the word hello
Method
Encoding
ASCII
0x68656c6c6f
UTF-32
0x00000068000000650000006c0000006c0000006f

The same word, `hello`, occupies five bytes in ASCII but twenty in UTF-32: each UTF-32 code point takes four bytes. section 2.1.2

UTF-16 and UTF-8 encode code points differently. UTF-16 can combine a high and low surrogate to recover a code point; UTF-8 uses one to four bytes according to the code point’s range. Do not treat a byte count as a count of code points. section 2.1.2

Recovering a code point

For a UTF-16 high surrogate of `0xD801` and low surrogate of `0xDC37`, calculate `(0xD801 − 0xD800) × 0x400 + (0xDC37 − 0xDC00) + 0x10000 = 0x10437`. The pair of code units represents one code point. section 2.1.2

How JavaScript in Depth covers it

How deep it goes
Defines code points and works through UTF-32 storage, UTF-16 surrogate arithmetic, and UTF-8 byte ranges.
Examples
Compares ASCII and UTF-32 encodings of `hello` and calculates `0x10437` from a UTF-16 surrogate pair.
What it assumes
Builds from ASCII mappings and hexadecimal notation before introducing Unicode encodings.
What you can do after
Evaluate/understand why a symbol’s assigned number and its encoded byte count are different quantities.
How it connects
Connects code points to ASCII, UTF-32, UTF-16 surrogate pairs, UTF-8, and Unicode normalization.

Also explained in

Compare how 7 books explain code point →

Related concepts

This explainer was generated by Manning's knowledge graph from the text of JavaScript in Depth. The section links point to the parts of the book it draws on.