UTF-8
UTF-8 is a variable-width Unicode encoding that represents each Unicode scalar value using one to four bytes and preserves ASCII byte representations.
Modern C
UTF-8 is a Unicode Transformation Format that encodes code points as sequences of 8-bit code units.
- How it is used
- Identifies an encoding for text exchanged between environments and for C strings whose encoding must not depend on the locale.
- How deep it goes
- Defines UTF-8 among Unicode formats and shows C11 literal syntax, but does not detail its byte encoding.
- Examples
- Contrasts u8"text", u"text", and U"text", with network communication as a reason to ensure an encoding.
- What you can do after
- Operate/configure it by choosing a C11 u8 literal when UTF-8 must be ensured.
PostgreSQL Mistakes and How to Avoid Them
UTF-8 is a variable-length character encoding that represents Unicode code points as sequences of one to four bytes.
- How it is used
- Prevents irreversible mixed-encoding text storage and supports multilingual PostgreSQL data.
- How deep it goes
- Explains server and database encodings, conversion failures, legacy-dump cleanup, and UTF8 database creation without detailing Unicode internals.
- Examples
- Uses multilingual ticket text, UTF-8 decode errors, UTF8 database creation, and a latin1 dump import containing NUL bytes.
- What you can do after
- Operate/configure UTF8 PostgreSQL databases and diagnose encoding failures during reads and legacy imports.
Read how PostgreSQL Mistakes and How to Avoid Them explains UTF-8 →
Python Workout: 50 ten-minute exercises
UTF-8 is a variable-width Unicode character encoding that represents characters using one to four eight-bit code units.
- How it is used
- Handles Python text strings and distinguishes Unicode text-file reading from binary-file reading.
- How deep it goes
- Defines code points, byte widths, and ASCII compatibility, then applies the bytes-versus-characters distinction to file handling.
- Examples
- Uses multilingual scripts, emoji, PDFs, JPEGs, binary mode, and fixed-size byte chunks as concrete cases.
- What you can do after
- Operate/configure Python file reads by selecting text or binary mode and consuming binary data in fixed-size chunks.
Read how Python Workout: 50 ten-minute exercises explains UTF-8 →
Learn Go with Pocket-Sized Projects
UTF-8 is a variable-width Unicode encoding that represents characters as sequences of one to four bytes.
- How it is used
- Supports multilingual terminal input and character-count validation in a Go word game.
- How deep it goes
- Explains byte-versus-rune lengths, conversions, iteration choices, and input validation without descending into encoding internals.
- Examples
- Uses "Hello, 世界", multilingual guesses, []byte, and []rune to contrast encoded bytes with characters.
- What you can do after
- Apply it via Go's []rune conversion to count characters and validate multilingual input correctly.
Read how Learn Go with Pocket-Sized Projects explains UTF-8 →
100 Go Mistakes and How to Avoid Them
UTF-8 is a variable-width Unicode character encoding in which characters are represented by sequences of one to four bytes.
- How it is used
- Diagnoses Go string bugs caused by confusing UTF-8 byte offsets, rune values, and character counts.
- How deep it goes
- Explains variable-width encoding, code points, byte lengths, rune counts, and corrected iteration, but defers invalid-encoding handling.
- Examples
- Uses 汉 and hêllo to expose multibyte encoding, incorrect s[i] output, byte lengths, and corrected rune iteration.
- What you can do after
- Diagnose string-length and iteration bugs, then select byte, range, rune-counting, or rune-slice operations appropriately.
Read how 100 Go Mistakes and How to Avoid Them explains UTF-8 →
Julia as a Second Language
UTF-8 is a variable-width Unicode encoding that represents code points using one to four bytes and is ASCII-compatible for one-byte characters.
- How it is used
- Uses UTF-8 to explain why Julia string lengths and valid character indices need not follow byte counts.
- How deep it goes
- Defines code points and variable-width code units, then works through indexing and iteration without detailing the full UTF-8 byte layout.
- Examples
- Compares length and byte size for Norwegian words, then inspects characters with codepoint, ncodeunits and REPL traversal loops.
- What you can do after
- Diagnose/debug with it when multibyte characters cause surprising Julia string indices, and traverse strings using valid indices.
Modern C, Third Edition
UTF-8 is a variable-length Unicode encoding that represents each code point as a sequence of one to four bytes.
- How it is used
- Uses UTF-8 for portable text literals, network communication, and incremental multibyte input processing.
- How deep it goes
- Explains literal types and incremental conversion, then stops short of a complete encoding algorithm.
- Examples
- Uses u8 literals, a UTF-8 correspondence table, and a C scanner built around mbrtoc8 and mbstate_t.
- What you can do after
- Apply UTF-8 through C11 character literals and diagnose incomplete or invalid multibyte input with mbrtoc8.
Data Analysis with LLMs
UTF-8 is a variable-length character encoding standard that represents Unicode text using one to four bytes per character and preserves ASCII encoding for the first 128 characters.
- How it is used
- Converts base64-encoded image and video data into UTF-8 strings suitable for GPT-4o prompts.
- How deep it goes
- Explains variable-length representation, ASCII compatibility, and Python conversion steps, but defers encoding internals.
- Examples
- Demonstrates UTF-8 conversion for locally stored images and OpenCV-extracted JPEG video frames.
- What you can do after
- Apply it via Python to convert base64-encoded media into text strings for GPT-4o prompts.
Go in Practice, Second Edition
UTF-8 is a variable-width Unicode character encoding that represents Unicode text as sequences of bytes.
- How it is used
- Handles Go's internal text representation and supports translation when exchanged data uses other encodings.
- How deep it goes
- Places UTF-8 in Go's data-encoding workflow but stops short of byte-level mechanics or conversion code.
- Examples
- Uses networked data exchange, JSON/XML conversion, and alternate-encoding translation as examples.
- What you can do after
- Apply UTF-8 awareness when exchanging data and select Go packages or interfaces for translation.
The Quick Python Book, Fourth Edition
UTF-8 is a variable-length character encoding that represents Unicode text while preserving the standard ASCII byte representations.
- How it is used
- Handles character encoding for ordinary text-file reading and writing, especially when files may contain more than ASCII.
- How deep it goes
- Explains ASCII compatibility, Unicode coverage, decoding ambiguity, and error policies, but does not detail UTF-8 byte-formation rules.
- Examples
- Uses raw invalid bytes, a UnicodeDecodeError, and four open() error policies with contrasting returned strings.
- What you can do after
- Operate Python file reads with UTF-8 assumptions and select an errors policy for malformed input.
Read how The Quick Python Book, Fourth Edition explains UTF-8 →
Grokking Algorithms, Second Edition
UTF-8 is a variable-length character encoding for representing Unicode text using one to four bytes per character.
- How it is used
- Recommends UTF-8 as a default when choosing an encoding for a project.
- How deep it goes
- Defines UTF-8 as a one-to-four-byte variable-length encoding but does not explain its byte patterns or decoding mechanics.
- Examples
- Contrasts UTF-8 with ASCII and ISO-8859-1 while discussing multilingual text and project encoding choices.
- What you can do after
- Apply it by choosing UTF-8 as a default encoding for a project.
Read how Grokking Algorithms, Second Edition explains UTF-8 →
Related concepts
- stringused by · A string is a programming-language value representing an ordered sequence of characters or character units as text.
- encodingkind of · Encoding is the transformation of information into a representation defined by a scheme for storage, processing, transmission, or interpretation.
- ASCIIcontrasts with · ASCII is a 7-bit character-encoding standard that assigns numeric codes to 128 basic Latin characters, symbols, punctuation marks, digits, and control characters.
- Unicodeuses · Unicode is a standard that assigns code points and properties to characters and defines rules for representing and processing text across writing systems.
These guides are generated from the text of Manning books by Manning's knowledge graph and link to the sections they draw on. The full books are available on liveBook and at manning.com.