Encoding
Encoding is the process or scheme of mapping information into a different representation according to defined rules for storage, processing, transmission, or interpretation.
Modern C
- How it is used
- Addresses character representation and portability in C text IO, string processing, and extended character sets.
- How deep it goes
- Explains text conversion and numeric escape boundaries, then gives examples using conversion state and multibyte-to-wide-character functions.
- Examples
- Uses newline output, hexadecimal escape boundaries, and a multibyte string length helper as concrete examples.
- What you can do after
- Apply it via C literals and library interfaces to avoid encoding-dependent assumptions in text processing.
Grokking Algorithms, Second Edition
A character encoding is a system that represents text characters as binary code values.
- How it is used
- Explains how text becomes binary data and why different character sets affect file representation and language coverage.
- How deep it goes
- Defines bits and bytes, walks through ISO-8859-1, and contrasts ASCII, Unicode, and UTF-8 without developing decoding internals.
- Examples
- Uses the letter a, the word “tilt,” file-size output, and binary dumps to make encoding concrete.
- What you can do after
- Evaluate how ASCII, ISO-8859-1, Unicode, and UTF-8 represent characters and affect storage size.
Read how Grokking Algorithms, Second Edition explains encoding →
Deep Learning with PyTorch, Second Edition
A tokenizer function that converts text into numeric token representations.
- How it is used
- Prepares character, word, or token sequences for neural-network input and next-token training.
- How deep it goes
- Works through character and word one-hot tensors, integer sequence shifting, padding, and tokenizer pipelines; defers embeddings to the next treatment.
- Examples
- Uses English characters, an “Impossible, Mr. Bennet” sentence, the name "$ada$", and The Odyssey tokenizer dataset.
- What you can do after
- Builds numeric text inputs, shifted targets, padded batches, and a WordLevel tokenizer with encode and decode functions.
Read how Deep Learning with PyTorch, Second Edition explains encoding →
Modern C, Third Edition
An encoding is a mapping or representation that associates data values with character or textual forms.
- How it is used
- Supports portable C text input, output, character handling, and conversion across differing internal and external representations.
- How deep it goes
- Explains platform conversion, escape syntax, locale-sensitive classification, and character-to-number logic, but not a complete encoding specification.
- Examples
- Uses '\n', hexadecimal escapes, <ctype.h> classifiers, and a hexatridecimal character-to-number conversion.
- What you can do after
- Apply it via C text and character interfaces to write portable handling for newlines, character classes, escapes, and numeric representations.
The Quick Python Book, Fourth Edition
encode is a string method that converts text into a bytes object using a character encoding.
- How it is used
- Converts Unicode strings to bytes for binary data, international text, and data exchanged through files or internet formats.
- How deep it goes
- Explains string-to-bytes conversion, reverse decoding, type incompatibility, and ASCII-versus-UTF-8 pitfalls without detailing codec internals.
- Examples
- Uses an accented character, displayed encoded bytes, failed mixed-type concatenation, and text-file decoding errors.
- What you can do after
- Apply it via string and bytes methods to convert text, reverse the conversion, and recognize incompatible types or encoding errors.
Read how The Quick Python Book, Fourth Edition explains encoding →
Grokking AI Algorithms, Second Edition
Text encoding is the conversion of written language into numerical representations that a computer model can process.
- How it is used
- Prepares text for mathematical processing in language-model training and represents candidate solutions within genetic algorithms.
- How deep it goes
- Explains vocabularies, token IDs, and workflow placement, but stops short of tokenization mechanics or encoding mathematics.
- Examples
- Uses books, papers, forums, code, token vocabularies, numerical IDs, and Knapsack-problem chromosomes as examples.
- What you can do after
- Understand how text becomes model input and why representation choices influence algorithmic performance.
Read how Grokking AI Algorithms, Second Edition explains encoding →
Go in Practice, Second Edition
- How it is used
- Handles data translation when systems exchange differently formatted or encoded information.
- How deep it goes
- Names UTF-8, Base64, JSON, and XML cases and points to packages and interfaces without demonstrating implementation.
- Examples
- Uses Base64 data and JSON or XML translated into local objects or structs as examples.
- What you can do after
- Understand how Go packages and interfaces can translate JSON into structs, switch encodings, or add external support.
PostgreSQL Mistakes and How to Avoid Them
A character encoding maps textual characters to coded representations such as bytes for storage and processing.
- How it is used
- Diagnoses character-data failures during MySQL or MariaDB migrations into PostgreSQL.
- How deep it goes
- Explains byte-level failures, legacy MySQL encodings, conversion, null-byte cleanup, and locale configuration.
- Examples
- Uses international names, the character é, invalid UTF8 errors, iconv conversion, and null-byte removal commands.
- What you can do after
- Diagnose import failures, convert latin1 dumps to UTF-8, remove null bytes, and verify PostgreSQL encoding settings.
Read how PostgreSQL Mistakes and How to Avoid Them explains encoding →
Rust in Action: Systems programming concepts and techniques
An encoding is a representation that maps information such as instructions or data to a defined sequence of symbols or bits.
- How it is used
- Explains how types, files, and CPU instructions assign meaning to shared sequences of bits.
- How deep it goes
- Demonstrates bit reinterpretation in Rust and extends the idea to a hexadecimal CHIP-8 CPU implementation.
- Examples
- Uses signed and unsigned integers, float-to-integer reinterpretation, and a hexadecimal CHIP-8 implementation.
- What you can do after
- Evaluate how types and instructions interpret bit patterns, and understand the foundations of a software CPU emulator.
Read how Rust in Action: Systems programming concepts and techniques explains encoding →
Local Image Generation
The transformation of an image into a compact latent representation.
- How it is used
- Compresses images into latents so Stable Diffusion can perform most processing in a smaller representation.
- How deep it goes
- Defines encoding, quantifies its reduction, and connects it to decoding without developing VAE internals.
- Examples
- Uses a 512×512 image from Metropolis and visualizes its transformation into latents.
- What you can do after
- Evaluate/understand why image-generation systems process compact latents instead of ordinary images.
Also covered in
- Natural Language Processing in Action, Second Edition
- Spring Security in Action, Second Edition
- API Design Patterns
- Graph Neural Networks in Action
- Learn Go with Pocket-Sized Projects
- Building User-Friendly DSLs
- 100 Go Mistakes and How to Avoid Them
- The Well-Grounded Rubyist, Fourth Edition
- Optimization Algorithms: AI techniques for design, planning, and control problems
- Quantum Programming in Depth
- Tiny Python Projects: Learn coding and testing with puzzles and games
- Learn AI-Assisted Python Programming, Second Edition
What to know first
- PythonPython is a high-level, general-purpose, dynamically typed programming language with readable syntax and support for multiple programming paradigms.
- ASCIIASCII is a 7-bit character-encoding standard that assigns numeric codes to 128 basic Latin characters, symbols, punctuation marks, digits, and control characters.
- bitA bit is a binary digit and unit of digital information that represents one of two distinguishable values, conventionally 0 or 1.
- characterA character is a unit of text treated as a single symbol or element in a character sequence, independent of its underlying encoding.
Related concepts
- base64url encodingincludes · Base64url encoding is a URL-safe variant of Base64 that replaces plus and slash with hyphen and underscore and commonly omits padding.
Questions about encoding
What is encoding?
Encoding is the process or scheme of mapping information into a different representation according to defined rules for storage, processing, transmission, or interpretation.
Which Manning books explain encoding?
Modern C, Grokking Algorithms, Second Edition, Deep Learning with PyTorch, Second Edition, Modern C, Third Edition, The Quick Python Book, Fourth Edition, Grokking AI Algorithms, Second Edition, Go in Practice, Second Edition, PostgreSQL Mistakes and How to Avoid Them and 2 more.
What should I know before learning encoding?
Manning books that teach encoding usually introduce Python, ASCII, bit and character first.
What is the difference between encoding and charset?
Encoding is the process or scheme of mapping information into a different representation according to defined rules for storage, processing, transmission, or interpretation. A charset is a defined set of characters.
What kinds of encoding are there?
Kinds of encoding include Base32 encoding, base64 url encoding, base64url encoding, bounded encoding and columnar encoding.
What is encoding also called?
Encoding is also called encode.
These guides are generated from the text of Manning books by Manning's knowledge graph and link to the sections they draw on. The full books are available on liveBook and at manning.com.









