liveBook · Manning

String

As explained in JavaScript in Depth

The JavaScript primitive text type whose immutable values are sequences of UTF-16 code units.

A directory listing can return a filename that cannot be opened. One cause is an encoding mismatch: older filesystems may contain Shift-JIS filename bytes, while Node.js decodes them as UTF-8 before exposing JavaScript strings. If the bytes are not valid UTF-8, the resulting name no longer matches the bytes on disk. The failure is at the filesystem boundary, not in string equality. section 2.1.4

A JavaScript string is an immutable sequence of zero or more 16-bit unsigned integer values. Those values are code units, not necessarily characters: a string need not contain valid UTF-16 text. The language specifies the sequence and its behavior; it does not specify how an engine stores it or how a host decodes bytes into it. section 2.1.1 section 2.3

Keep three distinctions in mind when handling text:

  • Bytes are not code units. UTF-8 encodes Unicode text in one to four bytes; UTF-16 uses one or two 16-bit code units for a Unicode character. A surrogate pair therefore occupies two positions in a JavaScript string. section 2.1.2 section 2.3
  • Appearance is not identity. Visually identical text can have different code-unit sequences, such as an accented letter represented directly or as a base letter plus combining mark. Normalize strings before comparisons that require this equivalence. section 2.1.2
  • Regular expressions need Unicode-aware handling when surrogate pairs matter. Without the `u` flag, a match such as `/^./` can consume only one half of a supplementary character; Unicode property escapes also require that flag. section 2.1.2
Listing 2.5 Iterating over a files and reading the contents in using Buffer instead
import { readdirSync, readFileSync } from 'node:fs';
 
const entries = readdirSync('./tmp', {
  encoding: 'buffer',  #A
  withFileTypes: true,
});
for (const entry of entries) {
  if (entry.isFile()) {
    readFileSync(entry.name);
  }
}

For filenames whose encoding must not be guessed, request Buffer names with `encoding: 'buffer'` and pass those uninterpreted bytes back to the filesystem API. section 2.1.4

The same separation applies inside the runtime. Although the language defines strings as 16-bit sequences, V8 chooses among representations according to content and use. Its ConsString can retain references to pieces during concatenation rather than immediately copying them. That optimization relies on strings being immutable; it is an implementation strategy, not a guarantee that every engine stores strings the same way. section 2.2 section 2.3

How JavaScript in Depth covers it

How deep it goes
Defines strings as 16-bit sequences, works through encoding pitfalls, and surveys V8 representations without prescribing an engine-wide storage format.
Examples
Traces a Shift-JIS filename failure and uses Buffer filenames, visually equivalent usernames, and Unicode regular-expression matches.
What it assumes
Builds character-encoding intuition from symbol-to-number mappings while assuming familiarity with basic JavaScript strings and Node.js examples.
What you can do after
Diagnose/debug with it when decoded filenames fail, visually identical strings compare differently, or regular expressions split surrogate pairs.
How it connects
Connects strings to Unicode normalization, UTF-8, UTF-16, surrogate pairs, regular expressions, Buffer, and V8 string representations.

Also explained in

Compare how 28 books explain string →

Related concepts

This explainer was generated by Manning's knowledge graph from the text of JavaScript in Depth. The section links point to the parts of the book it draws on.