Memory bound
A workload is memory-bound when its performance is limited primarily by memory access or data-transfer capacity rather than computational throughput.
"Looks Good to Me"
A workflow constraint that limits throughput or delays progress through a process.
- How it is used
- Diagnoses workflow constraints that delay code review, approval, deployment, or the delivery of customer value.
- How deep it goes
- Explains bottlenecks through approval chains, reviewer concentration, deployment architecture, and targeted process changes.
- Examples
- Uses approval chains, a single senior reviewer, oversized monolithic deployments, and domain-based projects as concrete bottleneck examples.
- What you can do after
- Diagnose workflow waiting points and choose distributed approval, safeguards, or smaller deployment units to increase delivery cadence.
GPU Programming with Triton
A memory-bound regime is a performance regime in which data-transfer capacity limits execution more than arithmetic throughput does.
- How it is used
- Classifies GPU kernels whose execution is limited more by data delivery than arithmetic throughput.
- How deep it goes
- Defines the regime through arithmetic intensity and roofline ceilings, using a first-order model rather than a detailed performance analysis.
- Examples
- Uses vector addition, reductions, softmax, normalization, and fused SwiGLU to illustrate memory-bound workloads.
- What you can do after
- Evaluate a kernel’s likely bottleneck using arithmetic intensity and roofline ceilings, then identify data transfers fusion can remove.
Read how GPU Programming with Triton explains memory bound →
Grokking GPU Programming
A memory-bound kernel is a GPU kernel whose performance is limited primarily by memory data transfer.
- How it is used
- Guides GPU kernel optimization by identifying when data transfer, rather than arithmetic, limits performance.
- How deep it goes
- Works through arithmetic intensity and the Roofline ridge point, then qualifies occupancy and requested-memory-traffic estimates.
- Examples
- Calculates intensity for an increment and naive matrix multiplication, then compares repeated requests with tiling.
- What you can do after
- Evaluate whether a kernel is likely memory-bound using arithmetic intensity and the GPU’s ridge point.
Architecture Modernization: Socio-technical alignment of software, strategy, and structure
A bottleneck is a constraint in a process or system that limits the rate or efficiency of overall work.
- How it is used
- Frames bottlenecks as organizational constraints affecting platform reuse, process flow, team capacity, and operational efficiency.
- How deep it goes
- Explains organizational causes through examples and diagnostic questions, without detailing technical performance measurement.
- Examples
- Uses an overloaded integration platform team, slow internal workflows, and shared platforms as examples.
- What you can do after
- Diagnose organizational bottlenecks by examining process durations, team dependencies, coordination load, and tool friction.
Vibe Engineering
A bottleneck is a limiting resource or constraint that restricts the progress or throughput of a system.
- How it is used
- Uses bottleneck measurement to locate performance-critical code under expected traffic, without establishing a memory constraint.
- How deep it goes
- Traces method timings, file reads, and scans but does not explain memory-bound behavior.
- Examples
- Measures a word-lookup service under a Gatling simulation and compares the results with an AI-generated diagnosis.
- What you can do after
- Evaluate bottlenecks by comparing runtime metrics with static hypotheses, without diagnosing a memory limit.
Refactoring to Rust
A bottleneck is a component or operation that limits the throughput or execution speed of a larger system.
- How it is used
- Frames bottlenecks as triggers for rewriting code or scaling a system when performance reaches a limit.
- How deep it goes
- Defines bottlenecks and contrasts scaling approaches, but does not explain how to diagnose memory as the limiting resource.
- Examples
- Uses a Mandelbrot workload to illustrate Python-to-Rust refactoring and shows multiple Python processes as a horizontal-scaling example.
- What you can do after
- Evaluate whether to rewrite code or scale horizontally or vertically after identifying a system performance limit.
CUDA for LLMs
A memory-bound operation is an operation whose performance is limited primarily by memory access rather than computation speed.
- How it is used
- Frames kernel optimization around data movement when memory bandwidth, rather than arithmetic, limits execution.
- How deep it goes
- Quantifies GEMV intensity and uses profiling to distinguish bandwidth ceilings from occupancy limits.
- Examples
- Uses matrix-vector traffic in GEMV and the attention score matrix as examples of costly data movement.
- What you can do after
- Diagnose bandwidth-limited kernels by comparing DRAM throughput, compute throughput, and occupancy in Nsight Compute.
Fast Python: High performance techniques for large datasets
- How it is used
- Locates memory-related performance limits in Python data processing through profiling before optimization.
- How deep it goes
- Connects memory costs to Python data structures and large-data processing, and data-transfer costs to distributed task graphs.
- Examples
- Uses climate-data ingestion and Dask maximum and sorting operations to illustrate bottlenecks and data movement.
- What you can do after
- Diagnose likely memory or transfer bottlenecks by profiling code and inspecting Dask task graphs.
Read how Fast Python: High performance techniques for large datasets explains memory bound →
Modern C, Third Edition
A performance bottleneck is a program component or operation that substantially limits overall execution efficiency.
- How it is used
- Assess whether performance improvements matter and measure performance bottlenecks.
- How deep it goes
- Introduces relevance and measurement, leaving detailed bottleneck assessment to a later discussion of performance measurement.
- Examples
- Contrasts reducing an interactive program’s runtime from 1 ms to 0.9 ms with worthwhile performance work.
- What you can do after
- Evaluate whether a performance improvement is relevant before investing effort in it.
Modern C
A performance bottleneck is a program component or operation that limits overall execution performance.
- How it is used
- Frames bottleneck assessment as part of deciding whether performance improvements are worthwhile.
- How deep it goes
- Introduces relevance and measurement but does not explain memory-bound behavior or its causes.
- Examples
- Compares reducing an interactive program’s runtime from 1 ms to 0.9 ms.
- What you can do after
- Evaluate whether a proposed runtime improvement is relevant enough to justify the effort.
Domain-Specific Small Language Models
A performance regime in which memory movement, rather than compute, limits inference speed.
- How it is used
- Diagnoses whether inference speed is constrained by data transfer rather than available compute, guiding hardware-specific optimization.
- How deep it goes
- Defines the ops-per-byte threshold and compares it with model arithmetic intensity, including a GPT-Neo calculation.
- Examples
- Compares GPT-Neo’s 30.29 ops/byte arithmetic intensity with a T4 GPU’s 216.6 ops/byte threshold.
- What you can do after
- Diagnose an inference workload by comparing its arithmetic intensity with the hardware’s ops-per-byte ratio.
Read how Domain-Specific Small Language Models explains memory bound →
Elixir in Action, Third Edition
A bottleneck is a component or operation whose limited processing capacity constrains the throughput of a larger system.
- How it is used
- Diagnoses serialized processes that limit throughput in concurrent and production-oriented systems.
- How deep it goes
- Explains sequential processing, demonstrates measured impact, and recommends optimization before parallelization.
- Examples
- Uses a slow echo server and a GenServer key-value store benchmark with one and 10,000 clients.
- What you can do after
- Diagnose bottlenecks with concurrent behavior and benchmarks, then optimize handling or assess process decomposition.
Read how Elixir in Action, Third Edition explains memory bound →
Enterprise RAG
A bottleneck is a component or operation that limits system throughput or causes excessive delay under load.
- How it is used
- Uses load testing to find constrained components before real users encounter slow responses, failures, or exhausted API quotas.
- How deep it goes
- Explains symptoms, log-based diagnosis, and targeted capacity changes, but stops short of detailed implementation code.
- Examples
- Uses 429 errors, ten-second answers, missing API responses, excessive search chunks, and overloaded Web App instances as concrete cases.
- What you can do after
- Diagnose/debug with it by using Locust and logs to isolate constrained services, then apply targeted scaling or demand reduction.
Machine Learning System Design
A bottleneck is a system component or operation that limits overall performance because it constrains throughput or increases processing time.
- How it is used
- Uses profiling to find promising optimization targets in serving and inference systems.
- How deep it goes
- Explains system-wide profiling and target selection through examples, without detailing memory-specific diagnostics.
- Examples
- Contrasts model and preprocessing runtimes, then recounts a sequential-model pipeline sped up by optimizing preprocessing.
- What you can do after
- Evaluate optimization targets by profiling the full pipeline and choosing components with room for improvement.
Read how Machine Learning System Design explains memory bound →
Advanced Algorithms and Data Structures
A bottleneck is a component or operation that limits the overall throughput or performance of a system.
- How it is used
- Uses coin counting to show how splitting aggregation work across levels can reduce a computational bottleneck.
- How deep it goes
- Works through a concrete redistribution of summation tasks, without memory-specific diagnosis or quantitative performance analysis.
- Examples
- Uses Donald Duck’s coin-counting task to contrast one overloaded summation stage with an intermediate level of workers.
- What you can do after
- Understand how an intermediate aggregation level can divide result-summing work among several workers.
Read how Advanced Algorithms and Data Structures explains memory bound →
Apache Kafka in Action
A bottleneck is a component or resource that limits the throughput or capacity of a system.
- How it is used
- Uses bottleneck diagnosis to size topic partitions for parallel consumer throughput.
- How deep it goes
- Works through consumer-rate sizing and partition trade-offs, including client RAM costs, but not a general memory-bound analysis.
- Examples
- Uses a 100 ms processing example to estimate consumer capacity and partitions for a 100-message-per-second peak.
- What you can do after
- Diagnose whether Kafka, consumers, or external dependencies cap throughput, then estimate consumer and partition needs from processing rate.
Starting Data Analytics with Generative AI and Python
A bottleneck is a component or operation that limits the overall performance or throughput of a system.
- How it is used
- Uses resource measurements alongside execution time to identify potential performance limits or contention.
- How deep it goes
- Distinguishes CPU-bound and I/O-bound work and demonstrates basic measurements, while deferring detailed resource monitoring.
- Examples
- Uses numerical simulations, sorting, large-file reads, and function-level CPU, I/O, and memory measurements as examples.
- What you can do after
- Diagnose likely limits by comparing execution time with CPU, disk I/O, and memory measurements.
Read how Starting Data Analytics with Generative AI and Python explains memory bound →
Rearchitecting LLMs
Memory-bound describes a workload whose performance is limited primarily by memory transfers rather than arithmetic processing.
- How it is used
- Frames memory-bound data movement as the inference bottleneck addressed by depth pruning.
- How deep it goes
- Explains memory transfers, GPU underuse, and KV-cache effects, without detailing hardware internals.
- Examples
- Uses Qwen3-06B block sizes and a 25% layer-removal example to illustrate memory reductions.
- What you can do after
- Apply depth pruning to reduce memory transfers and KV-cache growth during inference.
Also covered in
- Distributed Machine Learning Patterns
- Learn Concurrent Programming with Go
- Lead Developer Career Guide
- Fast by DefaultIn early access (MEAP)
- Grokking Concurrency
- Retrieval Augmented Generation, The Foundational IdeasIn early access (MEAP)
- Practices of the Python Pro
- The Art of AI Product Development
- Go by Example
- Build a DeepSeek Model (From Scratch)In early access (MEAP)
- Coding with AI
- Troubleshooting Java, Second Edition
What to know first
- CPUA central processing unit (CPU) is a computer processor that executes program instructions and performs general-purpose computational operations.
- throughputThroughput is the rate at which a system processes or completes work, data, or requests over a specified period.
- cachingCaching is the temporary storage of data or computed results for reuse, reducing the work or delay of retrieving or computing them again.
- GPU memoryGPU memory is memory available to a graphics processing unit for storing data and state used in graphics or computation.
Related concepts
- FlashAttentionaddressed by · FlashAttention is an exact, I/O-aware attention algorithm that tiles computation to reduce memory traffic without materializing the full attention matrix in high-bandwidth memory.
- matrix-vector multiplicationincludes · Matrix-vector multiplication is the linear algebra operation of multiplying a matrix by a compatible vector to produce a vector of row-wise dot products.
- compute-boundcontrasts with · A computation is compute-bound when its performance is limited primarily by processing capacity rather than memory transfer, input/output, or other resources.
- profilingmeasured by · Program profiling is the measurement and analysis of software execution behavior and resource use to characterize performance and identify bottlenecks.
- bidirectional DualPipe scheduleaddressed by · A bidirectional DualPipe schedule is a pipeline schedule that runs two data streams concurrently in opposite directions through the staged model.
Questions about memory bound
What is memory bound?
A workload is memory-bound when its performance is limited primarily by memory access or data-transfer capacity rather than computational throughput.
Which Manning books explain memory bound?
"Looks Good to Me", GPU Programming with Triton, Grokking GPU Programming, Architecture Modernization: Socio-technical alignment of software, strategy, and structure, Vibe Engineering, Refactoring to Rust, CUDA for LLMs, Fast Python: High performance techniques for large datasets and 10 more.
What should I know before learning memory bound?
Manning books that teach memory bound usually introduce CPU, throughput, caching and GPU memory first.
What is the difference between memory bound and compute-bound?
A workload is memory-bound when its performance is limited primarily by memory access or data-transfer capacity rather than computational throughput. A computation is compute-bound when its performance is limited primarily by processing capacity rather than memory transfer, input/output, or other resources.
What kinds of memory bound are there?
Kinds of memory bound include matrix-vector multiplication.
These guides are generated from the text of Manning books by Manning's knowledge graph and link to the sections they draw on. The full books are available on liveBook and at manning.com.

















