liveBook · Manning

Code unit

As explained in JavaScript in Depth

A 16-bit unit of storage used by UTF-16 to encode Unicode text.

Five letters, five code units

In UTF-16, `hello` is `0x00680065006c006c006f`: each letter occupies one 16-bit code unit. A code unit is storage for an encoding, not necessarily a whole Unicode codepoint. section 2.1.2

UTF-16 uses one code unit for codepoints through U+FFFF and two for codepoints above that range. The two-unit sequence is a surrogate pair; its units must be combined to recover the codepoint. For example, `0xD801` and `0xDC37` combine to give `0x10437`. section 2.1.2

When one symbol becomes two matches

Without the `u` flag, a JavaScript regular expression treats the surrogate pair in `😀ABC` as two characters; with `u`, matching is Unicode-aware. Do not assume a code-unit boundary is a codepoint boundary. section 2.1.2

How JavaScript in Depth covers it

How deep it goes
Defines the 16-bit unit and works through one- versus two-unit encoding and surrogate-pair calculation.
Examples
Uses the UTF-16 encoding of `hello`, a surrogate-pair calculation, and regex matches against `😀ABC`.
What it assumes
Assumes comfort reading hexadecimal values while introducing code units through concrete encodings.
What you can do after
Diagnose/debug with it when surrogate pairs make default and Unicode-aware regex matches differ.
How it connects
Connects code units to Unicode codepoints, UTF-16, surrogate pairs, and Unicode-aware regular expressions.

Also explained in

Compare how 2 books explain code unit →

Related concepts

This explainer was generated by Manning's knowledge graph from the text of JavaScript in Depth. The section links point to the parts of the book it draws on.