liveBook · Manning

Code point

As explained in 100 Go Mistakes and How to Avoid Them

A Unicode code point is a numeric value that identifies an abstract character or symbol in the Unicode standard.

A Unicode code point identifies an abstract character or symbol with one numeric value. In Go, a rune is a Unicode code point, while UTF-8 encodes that value into one to four bytes. Confusing these levels causes incorrect assumptions about string length and indexing. section 5.1

A character can occupy several bytes

The character 汉 has code point U+6C49 and occupies three UTF-8 bytes: 0xE6, 0xB1, and 0x89. Consequently, len("汉") returns 3, because Go measures the string's bytes rather than its number of code points. section 5.1

Choose the operation according to whether the task concerns bytes or code points:

  • Use range to iterate over runes; each index identifies the starting byte position of the rune's UTF-8 sequence. section 5.7
  • Convert a string to []rune when accessing a specific rune position, such as the third code point. section 5.7
  • Do not assume every string is UTF-8: string literals are UTF-8 encoded, but strings read from external sources are arbitrary bytes. section 5.1
s := "汉"
fmt.Println(len(s)) // 3

Inspect the contrasting byte length of a string containing 汉. section 5.1

How 100 Go Mistakes and How to Avoid Them covers it

How deep it goes
Defines code points, contrasts them with encodings, and traces UTF-8 byte length before stopping at broader string operations.
Examples
Uses 汉, U+6C49, its three UTF-8 bytes, and len("汉") to expose byte-versus-rune counting.
What it assumes
Assumes basic Go strings and bytes, while defining charset, encoding, Unicode code point, and rune relationships.
What you can do after
Apply rune-aware iteration and indexing while diagnosing incorrect assumptions about Go string length or encoding.
How it connects
Connects code points with Unicode, UTF-8, runes, strings, bytes, and range iteration.

Also explained in

Compare how 6 books explain code point →

Related concepts

This explainer was generated by Manning's knowledge graph from the text of 100 Go Mistakes and How to Avoid Them. The section links point to the parts of the book it draws on.