liveBook · Manning

Memory bound

A workload is memory-bound when its performance is limited primarily by memory access or data-transfer capacity rather than computational throughput.

How 18 Manning books explain it

"Looks Good to Me"

A workflow constraint that limits throughput or delays progress through a process.

How it is used
Diagnoses workflow constraints that delay code review, approval, deployment, or the delivery of customer value.
How deep it goes
Explains bottlenecks through approval chains, reviewer concentration, deployment architecture, and targeted process changes.
Examples
Uses approval chains, a single senior reviewer, oversized monolithic deployments, and domain-based projects as concrete bottleneck examples.
What you can do after
Diagnose workflow waiting points and choose distributed approval, safeguards, or smaller deployment units to increase delivery cadence.

Read how "Looks Good to Me" explains memory bound →

GPU Programming with Triton

A memory-bound regime is a performance regime in which data-transfer capacity limits execution more than arithmetic throughput does.

How it is used
Classifies GPU kernels whose execution is limited more by data delivery than arithmetic throughput.
How deep it goes
Defines the regime through arithmetic intensity and roofline ceilings, using a first-order model rather than a detailed performance analysis.
Examples
Uses vector addition, reductions, softmax, normalization, and fused SwiGLU to illustrate memory-bound workloads.
What you can do after
Evaluate a kernel’s likely bottleneck using arithmetic intensity and roofline ceilings, then identify data transfers fusion can remove.

Read how GPU Programming with Triton explains memory bound →

Grokking GPU Programming

A memory-bound kernel is a GPU kernel whose performance is limited primarily by memory data transfer.

How it is used
Guides GPU kernel optimization by identifying when data transfer, rather than arithmetic, limits performance.
How deep it goes
Works through arithmetic intensity and the Roofline ridge point, then qualifies occupancy and requested-memory-traffic estimates.
Examples
Calculates intensity for an increment and naive matrix multiplication, then compares repeated requests with tiling.
What you can do after
Evaluate whether a kernel is likely memory-bound using arithmetic intensity and the GPU’s ridge point.

Read how Grokking GPU Programming explains memory bound →

Architecture Modernization: Socio-technical alignment of software, strategy, and structure

A bottleneck is a constraint in a process or system that limits the rate or efficiency of overall work.

How it is used
Frames bottlenecks as organizational constraints affecting platform reuse, process flow, team capacity, and operational efficiency.
How deep it goes
Explains organizational causes through examples and diagnostic questions, without detailing technical performance measurement.
Examples
Uses an overloaded integration platform team, slow internal workflows, and shared platforms as examples.
What you can do after
Diagnose organizational bottlenecks by examining process durations, team dependencies, coordination load, and tool friction.

Read how Architecture Modernization: Socio-technical alignment of software, strategy, and structure explains memory bound →

Vibe Engineering

A bottleneck is a limiting resource or constraint that restricts the progress or throughput of a system.

How it is used
Uses bottleneck measurement to locate performance-critical code under expected traffic, without establishing a memory constraint.
How deep it goes
Traces method timings, file reads, and scans but does not explain memory-bound behavior.
Examples
Measures a word-lookup service under a Gatling simulation and compares the results with an AI-generated diagnosis.
What you can do after
Evaluate bottlenecks by comparing runtime metrics with static hypotheses, without diagnosing a memory limit.

Read how Vibe Engineering explains memory bound →

Refactoring to Rust

A bottleneck is a component or operation that limits the throughput or execution speed of a larger system.

How it is used
Frames bottlenecks as triggers for rewriting code or scaling a system when performance reaches a limit.
How deep it goes
Defines bottlenecks and contrasts scaling approaches, but does not explain how to diagnose memory as the limiting resource.
Examples
Uses a Mandelbrot workload to illustrate Python-to-Rust refactoring and shows multiple Python processes as a horizontal-scaling example.
What you can do after
Evaluate whether to rewrite code or scale horizontally or vertically after identifying a system performance limit.

Read how Refactoring to Rust explains memory bound →

CUDA for LLMs

A memory-bound operation is an operation whose performance is limited primarily by memory access rather than computation speed.

How it is used
Frames kernel optimization around data movement when memory bandwidth, rather than arithmetic, limits execution.
How deep it goes
Quantifies GEMV intensity and uses profiling to distinguish bandwidth ceilings from occupancy limits.
Examples
Uses matrix-vector traffic in GEMV and the attention score matrix as examples of costly data movement.
What you can do after
Diagnose bandwidth-limited kernels by comparing DRAM throughput, compute throughput, and occupancy in Nsight Compute.

Read how CUDA for LLMs explains memory bound →

Fast Python: High performance techniques for large datasets

How it is used
Locates memory-related performance limits in Python data processing through profiling before optimization.
How deep it goes
Connects memory costs to Python data structures and large-data processing, and data-transfer costs to distributed task graphs.
Examples
Uses climate-data ingestion and Dask maximum and sorting operations to illustrate bottlenecks and data movement.
What you can do after
Diagnose likely memory or transfer bottlenecks by profiling code and inspecting Dask task graphs.

Read how Fast Python: High performance techniques for large datasets explains memory bound →

Modern C, Third Edition

A performance bottleneck is a program component or operation that substantially limits overall execution efficiency.

How it is used
Assess whether performance improvements matter and measure performance bottlenecks.
How deep it goes
Introduces relevance and measurement, leaving detailed bottleneck assessment to a later discussion of performance measurement.
Examples
Contrasts reducing an interactive program’s runtime from 1 ms to 0.9 ms with worthwhile performance work.
What you can do after
Evaluate whether a performance improvement is relevant before investing effort in it.

Read how Modern C, Third Edition explains memory bound →

Modern C

A performance bottleneck is a program component or operation that limits overall execution performance.

How it is used
Frames bottleneck assessment as part of deciding whether performance improvements are worthwhile.
How deep it goes
Introduces relevance and measurement but does not explain memory-bound behavior or its causes.
Examples
Compares reducing an interactive program’s runtime from 1 ms to 0.9 ms.
What you can do after
Evaluate whether a proposed runtime improvement is relevant enough to justify the effort.

Read how Modern C explains memory bound →

Domain-Specific Small Language Models

A performance regime in which memory movement, rather than compute, limits inference speed.

How it is used
Diagnoses whether inference speed is constrained by data transfer rather than available compute, guiding hardware-specific optimization.
How deep it goes
Defines the ops-per-byte threshold and compares it with model arithmetic intensity, including a GPT-Neo calculation.
Examples
Compares GPT-Neo’s 30.29 ops/byte arithmetic intensity with a T4 GPU’s 216.6 ops/byte threshold.
What you can do after
Diagnose an inference workload by comparing its arithmetic intensity with the hardware’s ops-per-byte ratio.

Read how Domain-Specific Small Language Models explains memory bound →

Elixir in Action, Third Edition

A bottleneck is a component or operation whose limited processing capacity constrains the throughput of a larger system.

How it is used
Diagnoses serialized processes that limit throughput in concurrent and production-oriented systems.
How deep it goes
Explains sequential processing, demonstrates measured impact, and recommends optimization before parallelization.
Examples
Uses a slow echo server and a GenServer key-value store benchmark with one and 10,000 clients.
What you can do after
Diagnose bottlenecks with concurrent behavior and benchmarks, then optimize handling or assess process decomposition.

Read how Elixir in Action, Third Edition explains memory bound →

Enterprise RAG

A bottleneck is a component or operation that limits system throughput or causes excessive delay under load.

How it is used
Uses load testing to find constrained components before real users encounter slow responses, failures, or exhausted API quotas.
How deep it goes
Explains symptoms, log-based diagnosis, and targeted capacity changes, but stops short of detailed implementation code.
Examples
Uses 429 errors, ten-second answers, missing API responses, excessive search chunks, and overloaded Web App instances as concrete cases.
What you can do after
Diagnose/debug with it by using Locust and logs to isolate constrained services, then apply targeted scaling or demand reduction.

Read how Enterprise RAG explains memory bound →

Machine Learning System Design

A bottleneck is a system component or operation that limits overall performance because it constrains throughput or increases processing time.

How it is used
Uses profiling to find promising optimization targets in serving and inference systems.
How deep it goes
Explains system-wide profiling and target selection through examples, without detailing memory-specific diagnostics.
Examples
Contrasts model and preprocessing runtimes, then recounts a sequential-model pipeline sped up by optimizing preprocessing.
What you can do after
Evaluate optimization targets by profiling the full pipeline and choosing components with room for improvement.

Read how Machine Learning System Design explains memory bound →

Advanced Algorithms and Data Structures

A bottleneck is a component or operation that limits the overall throughput or performance of a system.

How it is used
Uses coin counting to show how splitting aggregation work across levels can reduce a computational bottleneck.
How deep it goes
Works through a concrete redistribution of summation tasks, without memory-specific diagnosis or quantitative performance analysis.
Examples
Uses Donald Duck’s coin-counting task to contrast one overloaded summation stage with an intermediate level of workers.
What you can do after
Understand how an intermediate aggregation level can divide result-summing work among several workers.

Read how Advanced Algorithms and Data Structures explains memory bound →

Apache Kafka in Action

A bottleneck is a component or resource that limits the throughput or capacity of a system.

How it is used
Uses bottleneck diagnosis to size topic partitions for parallel consumer throughput.
How deep it goes
Works through consumer-rate sizing and partition trade-offs, including client RAM costs, but not a general memory-bound analysis.
Examples
Uses a 100 ms processing example to estimate consumer capacity and partitions for a 100-message-per-second peak.
What you can do after
Diagnose whether Kafka, consumers, or external dependencies cap throughput, then estimate consumer and partition needs from processing rate.

Read how Apache Kafka in Action explains memory bound →

Starting Data Analytics with Generative AI and Python

A bottleneck is a component or operation that limits the overall performance or throughput of a system.

How it is used
Uses resource measurements alongside execution time to identify potential performance limits or contention.
How deep it goes
Distinguishes CPU-bound and I/O-bound work and demonstrates basic measurements, while deferring detailed resource monitoring.
Examples
Uses numerical simulations, sorting, large-file reads, and function-level CPU, I/O, and memory measurements as examples.
What you can do after
Diagnose likely limits by comparing execution time with CPU, disk I/O, and memory measurements.

Read how Starting Data Analytics with Generative AI and Python explains memory bound →

Rearchitecting LLMs

Memory-bound describes a workload whose performance is limited primarily by memory transfers rather than arithmetic processing.

How it is used
Frames memory-bound data movement as the inference bottleneck addressed by depth pruning.
How deep it goes
Explains memory transfers, GPU underuse, and KV-cache effects, without detailing hardware internals.
Examples
Uses Qwen3-06B block sizes and a 25% layer-removal example to illustrate memory reductions.
What you can do after
Apply depth pruning to reduce memory transfers and KV-cache growth during inference.

Read how Rearchitecting LLMs explains memory bound →

Also covered in

What to know first

Related concepts

Questions about memory bound

What is memory bound?

A workload is memory-bound when its performance is limited primarily by memory access or data-transfer capacity rather than computational throughput.

Which Manning books explain memory bound?

"Looks Good to Me", GPU Programming with Triton, Grokking GPU Programming, Architecture Modernization: Socio-technical alignment of software, strategy, and structure, Vibe Engineering, Refactoring to Rust, CUDA for LLMs, Fast Python: High performance techniques for large datasets and 10 more.

What should I know before learning memory bound?

Manning books that teach memory bound usually introduce CPU, throughput, caching and GPU memory first.

What is the difference between memory bound and compute-bound?

A workload is memory-bound when its performance is limited primarily by memory access or data-transfer capacity rather than computational throughput. A computation is compute-bound when its performance is limited primarily by processing capacity rather than memory transfer, input/output, or other resources.

What kinds of memory bound are there?

Kinds of memory bound include matrix-vector multiplication.

These guides are generated from the text of Manning books by Manning's knowledge graph and link to the sections they draw on. The full books are available on liveBook and at manning.com.