UTF-8
UTF-8 is a variable-width Unicode encoding that represents each Unicode scalar value as one to four bytes and preserves ASCII byte representations.
JavaScript in Depth
A Unicode transformation format that encodes code points into a variable-length sequence of one to four bytes.
- How it is used
- Explains encoded byte counts and why filesystem text can fail when its encoding is assumed incorrectly.
- How deep it goes
- Works through code-point ranges, byte patterns, and Buffer output; stops short of a full decoder implementation.
- Examples
- Compares UTF-16LE and UTF-8 byte lengths for Japanese text and encodes ⇇ with Node.js Buffer.
- What you can do after
- Diagnose/debug with it: distinguish a JavaScript string's character count from its encoded bytes and investigate encoding mismatches.
Modern C
UTF-8 is a Unicode Transformation Format that encodes code points as sequences of 8-bit code units.
- How it is used
- Uses UTF-8 to address encoding consistency across C environments and network communication.
- How deep it goes
- Introduces UTF-8 literals and their type but does not explain the encoding algorithm.
- Examples
- Illustrates UTF-8 with the C string literal u8"text".
- What you can do after
- Evaluate encoding consistency and use a C UTF-8 literal when an explicit encoding is needed.
PostgreSQL Mistakes and How to Avoid Them
UTF-8 is a variable-length character encoding that represents Unicode code points as sequences of one to four bytes.
- How it is used
- Prevents irreversible mixed-encoding text storage and supports multilingual PostgreSQL data.
- How deep it goes
- Explains server and database encodings, conversion failures, legacy-dump cleanup, and UTF8 database creation without detailing Unicode internals.
- Examples
- Uses multilingual ticket text, UTF-8 decode errors, UTF8 database creation, and a latin1 dump import containing NUL bytes.
- What you can do after
- Operate/configure UTF8 PostgreSQL databases and diagnose encoding failures during reads and legacy imports.
Read how PostgreSQL Mistakes and How to Avoid Them explains UTF-8 →
Python Workout: 50 ten-minute exercises
UTF-8 is a variable-width Unicode character encoding that represents characters using one to four eight-bit code units.
- How it is used
- Handles Python text strings and distinguishes Unicode text-file reading from binary-file reading.
- How deep it goes
- Defines code points, byte widths, and ASCII compatibility, then applies the bytes-versus-characters distinction to file handling.
- Examples
- Uses multilingual scripts, emoji, PDFs, JPEGs, binary mode, and fixed-size byte chunks as concrete cases.
- What you can do after
- Operate/configure Python file reads by selecting text or binary mode and consuming binary data in fixed-size chunks.
Read how Python Workout: 50 ten-minute exercises explains UTF-8 →
Learn Go with Pocket-Sized Projects
UTF-8 is a variable-width Unicode encoding that represents characters as sequences of one to four bytes.
- How it is used
- Supports multilingual terminal input and character-count validation in a Go word game.
- How deep it goes
- Explains byte-versus-rune lengths, conversions, iteration choices, and input validation without descending into encoding internals.
- Examples
- Uses "Hello, 世界", multilingual guesses, []byte, and []rune to contrast encoded bytes with characters.
- What you can do after
- Apply it via Go's []rune conversion to count characters and validate multilingual input correctly.
Read how Learn Go with Pocket-Sized Projects explains UTF-8 →
100 Go Mistakes and How to Avoid Them
UTF-8 is a variable-width Unicode character encoding in which characters are represented by sequences of one to four bytes.
- How it is used
- Diagnoses Go string bugs caused by confusing UTF-8 byte offsets, rune values, and character counts.
- How deep it goes
- Explains variable-width encoding, code points, byte lengths, rune counts, and corrected iteration, but defers invalid-encoding handling.
- Examples
- Uses 汉 and hêllo to expose multibyte encoding, incorrect s[i] output, byte lengths, and corrected rune iteration.
- What you can do after
- Diagnose string-length and iteration bugs, then select byte, range, rune-counting, or rune-slice operations appropriately.
Read how 100 Go Mistakes and How to Avoid Them explains UTF-8 →
Julia as a Second Language
UTF-8 is a variable-width Unicode encoding that represents code points using one to four bytes and is ASCII-compatible for one-byte characters.
- How it is used
- Explains why Julia string indices can differ from character positions when text contains multibyte characters.
- How deep it goes
- Explains code points, code units, and traversal through REPL examples and a `nextind` implementation excerpt.
- Examples
- Uses Norwegian words, ASCII letters, Æ, and an emoji to compare character counts, code units, and indices.
- What you can do after
- Apply it in Julia to inspect character encodings and traverse strings without assuming consecutive character indices.
Modern C, Third Edition
UTF-8 is a variable-length Unicode encoding that represents each code point as a sequence of one to four bytes.
- How it is used
- Uses UTF-8 for portable text literals, network communication, and incremental multibyte input processing.
- How deep it goes
- Explains literal types and incremental conversion, then stops short of a complete encoding algorithm.
- Examples
- Uses u8 literals, a UTF-8 correspondence table, and a C scanner built around mbrtoc8 and mbstate_t.
- What you can do after
- Apply UTF-8 through C11 character literals and diagnose incomplete or invalid multibyte input with mbrtoc8.
Data Analysis with LLMs
UTF-8 is a variable-length character encoding standard that represents Unicode text using one to four bytes per character and preserves ASCII encoding for the first 128 characters.
- How it is used
- Converts base64-encoded image and video data into UTF-8 strings suitable for GPT-4o prompts.
- How deep it goes
- Explains variable-length representation, ASCII compatibility, and Python conversion steps, but defers encoding internals.
- Examples
- Demonstrates UTF-8 conversion for locally stored images and OpenCV-extracted JPEG video frames.
- What you can do after
- Apply it via Python to convert base64-encoded media into text strings for GPT-4o prompts.
Go in Practice, Second Edition
UTF-8 is a variable-width Unicode character encoding that represents Unicode text as sequences of bytes.
- How it is used
- Handles Go's internal text representation and supports translation when exchanged data uses other encodings.
- How deep it goes
- Places UTF-8 in Go's data-encoding workflow but stops short of byte-level mechanics or conversion code.
- Examples
- Uses networked data exchange, JSON/XML conversion, and alternate-encoding translation as examples.
- What you can do after
- Apply UTF-8 awareness when exchanging data and select Go packages or interfaces for translation.
The Quick Python Book, Fourth Edition
UTF-8 is a variable-length character encoding that represents Unicode text while preserving the standard ASCII byte representations.
- How it is used
- Handles character encoding for ordinary text-file reading and writing, especially when files may contain more than ASCII.
- How deep it goes
- Explains ASCII compatibility, Unicode coverage, decoding ambiguity, and error policies, but does not detail UTF-8 byte-formation rules.
- Examples
- Uses raw invalid bytes, a UnicodeDecodeError, and four open() error policies with contrasting returned strings.
- What you can do after
- Operate Python file reads with UTF-8 assumptions and select an errors policy for malformed input.
Read how The Quick Python Book, Fourth Edition explains UTF-8 →
Secrets of the JavaScript Ninja, Third Edition
A variable-width Unicode encoding that represents each Unicode scalar value using one to four bytes.
- How it is used
- Protects streamed text decoding from characters split across network chunks.
- How deep it goes
- Explains multibyte chunk boundaries and decoder state, stopping short of byte-level encoding details.
- Examples
- Uses a streamed HTTP response whose body arrives in chunks and is decoded incrementally.
- What you can do after
- Apply TextDecoder in streaming mode to decode response chunks containing split multibyte characters.
Read how Secrets of the JavaScript Ninja, Third Edition explains UTF-8 →
Grokking Algorithms, Second Edition
UTF-8 is a variable-length character encoding for representing Unicode text using one to four bytes per character.
- How it is used
- Recommends UTF-8 as a default when choosing an encoding for a project.
- How deep it goes
- Defines UTF-8 as a one-to-four-byte variable-length encoding but does not explain its byte patterns or decoding mechanics.
- Examples
- Contrasts UTF-8 with ASCII and ISO-8859-1 while discussing multilingual text and project encoding choices.
- What you can do after
- Apply it by choosing UTF-8 as a default encoding for a project.
Read how Grokking Algorithms, Second Edition explains UTF-8 →
Also covered in
- Classic Computer Science Problems in Swift: Essential techniques for practicing programmers
- API Design Patterns
- The Well-Grounded Rubyist, Fourth Edition
- Systems Programming with ZigIn early access (MEAP)
- Idiomatic Rust
- Python Workout, Second Edition
- Go in Action, Second Edition
- ScyllaDB in Action
- Rust in Action: Systems programming concepts and techniques
- Build a Website with ChatGPT
What to know first
- ASCIIASCII is a 7-bit character-encoding standard that assigns numeric codes to 128 basic Latin characters, symbols, punctuation marks, digits, and control characters.
- UnicodeUnicode is a standard that assigns code points and properties to characters and defines encoding forms for representing text digitally.
- stringA string is a programming-language value representing textual data as an ordered sequence of characters or encoded units.
- Base64Base64 is a binary-to-text encoding scheme that represents arbitrary bytes using a fixed 64-character alphabet.
Related concepts
- UTF-16alternative to · UTF-16 is a variable-width Unicode encoding form that represents code points with one or two 16-bit code units, using surrogate pairs when necessary.
- encodingkind of · Encoding is the process or scheme of mapping information into a different representation according to defined rules for storage, processing, transmission, or interpretation.
- stringused by · A string is a programming-language value representing textual data as an ordered sequence of characters or encoded units.
- ASCIIcontrasts with · ASCII is a 7-bit character-encoding standard that assigns numeric codes to 128 basic Latin characters, symbols, punctuation marks, digits, and control characters.
- bytesuses · Bytes are an ordered sequence of byte values, typically eight-bit units, used to represent binary data.
Questions about UTF-8
What is UTF-8?
UTF-8 is a variable-width Unicode encoding that represents each Unicode scalar value as one to four bytes and preserves ASCII byte representations.
Which Manning books explain UTF-8?
JavaScript in Depth, Modern C, PostgreSQL Mistakes and How to Avoid Them, Python Workout: 50 ten-minute exercises, Learn Go with Pocket-Sized Projects, 100 Go Mistakes and How to Avoid Them, Julia as a Second Language, Modern C, Third Edition and 5 more.
What should I know before learning UTF-8?
Manning books that teach UTF-8 usually introduce ASCII, Unicode, string and Base64 first.
What is the difference between UTF-8 and UTF-16?
UTF-8 is a variable-width Unicode encoding that represents each Unicode scalar value as one to four bytes and preserves ASCII byte representations. UTF-16 is a variable-width Unicode encoding form that represents code points with one or two 16-bit code units, using surrogate pairs when necessary.
What is UTF-8 a kind of?
UTF-8 is a kind of encoding. Encoding is the process or scheme of mapping information into a different representation according to defined rules for storage, processing, transmission, or interpretation.
These guides are generated from the text of Manning books by Manning's knowledge graph and link to the sections they draw on. The full books are available on liveBook and at manning.com.












