liveBook · Manning

Compression

Also called compress, compressing

Compression is the encoding of data or information in a more compact representation to reduce its size while retaining content needed for recovery or use.

How 15 Manning books explain it

LLMs in Production

Compression is the reduction of a model's size and computational requirements while retaining acceptable behavior and accuracy.

How it is used
Uses compression to make large language models less costly and more practical to deploy.
How deep it goes
Explains quantization and illustrates low-rank approximation, while leaving floating-point mathematics unexplored.
Examples
Illustrates post-training quantization in PyTorch, SVD on a matrix, and low-bit Llama 3 70B implementations.
What you can do after
Evaluate compression choices against memory, latency, cost, and model quality.

Read how LLMs in Production explains compression →

Sutskever's List

Compression is the representation of information using fewer bits or a shorter description while preserving the information needed for reconstruction or inference.

How it is used
Uses compression to expose regularity and select models that describe data without spending complexity on noise.
How deep it goes
Works through compressed-size proxies and MDL trade-offs, while noting that exact Kolmogorov complexity cannot be computed.
Examples
Contrasts repeated, random, and biased bit strings; compresses coarse-grained coffee arrays; and compares polynomial fits.
What you can do after
Evaluate/understand it: compare descriptions by length and distinguish compression of a network’s behavior from shrinking its parameters.

Read how Sutskever's List explains compression →

Fast Python: High performance techniques for large datasets

How it is used
Uses compression to reduce storage and potentially speed data movement and processing.
How deep it goes
Benchmarks array and CSV compression, compares algorithms and bit shuffling, and explains cache effects without detailing codec internals.
Examples
Compares zero-filled, tiled, and random arrays, plus compressed and uncompressed taxi CSV files.
What you can do after
Apply it via Blosc or pandas, then measure storage and processing trade-offs on the target data and hardware.

Read how Fast Python: High performance techniques for large datasets explains compression →

100 SQL Server Mistakes and How to Avoid Them

Data compression encodes stored database data in a more compact representation to reduce storage and I/O requirements.

How it is used
Uses compression as a performance enhancement for large tables when I/O limits performance and processor capacity remains available.
How deep it goes
Explains row-compression storage internals, demonstrates page reductions, and names page compression without detailing its internal structure.
Examples
Uses ImpressionArchive, page counts, ALTER TABLE rebuilds, metadata queries, and compression-savings estimation.
What you can do after
Operate/configure SQL Server compression, estimate savings, implement row or page compression, and verify resulting page usage.

Read how 100 SQL Server Mistakes and How to Avoid Them explains compression →

Kafka for Architects

Data compression is the transformation of data into a more compact representation to reduce storage or transmission size.

How it is used
Reduces Kafka network traffic and retained storage while informing message-size, batching, and cloud-cost decisions.
How deep it goes
Explains producer and broker placement, batch behavior, codecs, consumer decompression, and recompression overhead.
Examples
Uses gzip, snappy, lz4, zstd, uncompressed data, producer batches, and repetitive JSON or Avro payloads.
What you can do after
Operate/configure Kafka compression by selecting codecs, placement, batching, and limits against throughput, CPU, and storage requirements.

Read how Kafka for Architects explains compression →

Classic Computer Science Problems in Swift: Essential techniques for practicing programmers

Compression is the process of representing data with fewer bits by eliminating or encoding redundancy.

How it is used
Frames compression as a practical way to reduce storage and bandwidth when space matters more than processing speed.
How deep it goes
Develops type narrowing and two-bit encoding with compression and decompression, but stops short of advanced pattern-based algorithms.
Examples
Uses oversized unsigned integers, a two-bit gene encoding, bit vectors, and internet file transmission as examples.
What you can do after
Apply compression by choosing compact numeric types and encoding fixed-alphabet data into bit vectors.

Read how Classic Computer Science Problems in Swift: Essential techniques for practicing programmers explains compression →

Build AI Drug Discovery Pipelines

Compression is the encoding of data in a smaller representation that can support later reconstruction or use.

How it is used
Supports efficient molecular representation, search, optimization, and generation within machine-learning drug-discovery workflows.
How deep it goes
Defines lower-dimensional encoding and reconstruction, then explains latent-space implementation while flagging invalid decoding and weak neighborhood structure.
Examples
Uses molecular blueprints, large molecule libraries, ZIP-file analogies, and sampled latent points decoded into structures.
What you can do after
Apply it via a generative model to reconstruct molecules or explore candidate structures through latent-space sampling.

Read how Build AI Drug Discovery Pipelines explains compression →

AI Agents in Action

Memory compression condenses stored memory information while retaining content useful for later retrieval.

How it is used
Condenses stored memories or knowledge to support retrieval in agent memory and knowledge systems.
How deep it goes
Explains clustering and summarization, names k-means, and stops short of detailing the compression function.
Examples
Uses a knowledge-store exercise and contrasts literary prose with code as compression candidates.
What you can do after
Apply compression by clustering and summarizing a knowledge store, then observing effects on query performance.

Read how AI Agents in Action explains compression →

How Computers Make Books

Compression is the reduction of the amount of data needed to represent information while preserving either all or an acceptably useful part of its content.

How it is used
Reduces storage needs and transmission delays for text, images, and video.
How deep it goes
Explains reversible representations, pattern-based codes, and basic limits, while stopping short of specialized algorithms.
Examples
Uses a coffee-order abbreviation and a common-word code applied to a passage from Metamorphosis.
What you can do after
Evaluate simple compression schemes by comparing representations, repeated patterns, reversibility, and measured size.

Read how How Computers Make Books explains compression →

AI Agents in Action, Second Edition

Compression is the reduction or summarization of stored information while preserving information needed for later retrieval or use.

How it is used
Uses compression to control token costs, reduce retrieval noise, and keep memory and knowledge stores scalable.
How deep it goes
Explains clustering, summarization, storage, triggers, and timing, but does not specify a compression-function implementation.
Examples
Uses clustered memories, verbose documents, repetitive code, and updated vector databases as concrete cases.
What you can do after
Apply it by clustering similar items, summarizing clusters, and scheduling compression according to store behavior.

Read how AI Agents in Action, Second Edition explains compression →

Local Image Generation

A technique for representing an image with fewer data while retaining enough information for reconstruction.

How it is used
Reduces image size and Stable Diffusion’s calculation load while encouraging generalization.
How deep it goes
Explains encoding into latent space and decoding back into images, with practical motivation rather than implementation details.
Examples
Uses JPEG and ZIP comparisons plus a child’s imperfect house drawing to illustrate compression and generalization.
What you can do after
Understand why Stable Diffusion compresses images into latents before performing most of its work.

Read how Local Image Generation explains compression →

Logs and Telemetry

Compression is the encoding of data into a representation that requires less storage or transmission capacity.

How it is used
Reduces HTTP payload size when Fluent Bit sends substantial JSON data across networks.
How deep it goes
Explains enabling compression through a property and identifies gzip as the available algorithm, requiring support at both endpoints.
Examples
Demonstrates compression for Fluent Bit HTTP output carrying JSON data through data-center or proxy-connected networks.
What you can do after
Apply compression via Fluent Bit's HTTP output configuration using gzip.

Read how Logs and Telemetry explains compression →

Context Engineering

The process of representing information with fewer bits or tokens while retaining relevant information.

How it is used
Reduces agent context pressure while retaining information needed to continue a task.
How deep it goes
Details techniques, triggering rules, preservation priorities, and functional evaluation probes.
Examples
Uses token-heavy search results and bulky tool observations to illustrate summarization and masking.
What you can do after
Evaluate compression by probing recall, artifact state, plans, decision rationale, and instruction adherence rather than relying on textual similarity.

Read how Context Engineering explains compression →

WordPress Power Toolkit: Harness AI to build next-level websites

Compression is the reduction of digital data size while preserving the information needed to reconstruct or use it.

How it is used
Improves website delivery by shrinking video and code files, reducing bandwidth demands and helping pages load or play media more smoothly.
How deep it goes
Explains practical benefits and code minification, but does not detail compression algorithms or optimization settings.
Examples
Uses YouTube video embedding, WP Rocket minification, removed whitespace, and the compact notation “5bl” for repeated blank lines.
What you can do after
Apply it via YouTube and WP Rocket to reduce media or code size and improve website loading and playback.

Read how WordPress Power Toolkit: Harness AI to build next-level websites explains compression →

Timeless Algorithms, The Foundational Ideas

Encoding information using fewer bits or symbols while preserving what is needed for its recovery or use.

How it is used
Uses source entropy to estimate removable redundancy, transmission requirements, and channel capacity in communication-system design.
How deep it goes
Explains predictable versus unpredictable sequences and distinguishes removable redundancy from redundancy added to detect or correct noise.
Examples
Contrasts AAAAAA, encoded as A × 6, with W7dX8$ and compares entropy values for coins, dice, and cards.
What you can do after
Evaluate compression opportunities by relating source entropy and predictable redundancy to the bits required for transmission.

Read how Timeless Algorithms, The Foundational Ideas explains compression →

Also covered in

What to know first

Related concepts

Questions about compression

What is compression?

Compression is the encoding of data or information in a more compact representation to reduce its size while retaining content needed for recovery or use.

Which Manning books explain compression?

LLMs in Production, Sutskever's List, Fast Python: High performance techniques for large datasets, 100 SQL Server Mistakes and How to Avoid Them, Kafka for Architects, Classic Computer Science Problems in Swift: Essential techniques for practicing programmers, Build AI Drug Discovery Pipelines, AI Agents in Action and 7 more.

What should I know before learning compression?

Manning books that teach compression usually introduce digital image, similarity search, HTTP and language model first.

What kinds of compression are there?

Kinds of compression include batch compression and columnar compression.

What is compression also called?

Compression is also called compress or compressing.

These guides are generated from the text of Manning books by Manning's knowledge graph and link to the sections they draw on. The full books are available on liveBook and at manning.com.