Bottleneck
A bottleneck is a constraint, component, or stage that limits the throughput or performance of a larger system or workflow.
Build High Value Software Systems
A stage whose capacity persistently falls short of demand and limits the output of a value stream.
- How it is used
- Identifies the stage limiting delivery or request throughput so improvements target end-to-end flow rather than local speed.
- How deep it goes
- Defines the demand-versus-capacity test, gives observable symptoms, and works through five constraint-improvement steps.
- Examples
- Uses an accumulating pull-request queue, a deployment approval gate, and peak-load latency across services sharing a database.
- What you can do after
- Diagnose/debug with it by comparing demand with capacity, improving the constraint, and checking where the limit moves next.
Read how Build High Value Software Systems explains bottleneck →
CUDA for LLMs
A performance bottleneck is a resource or component whose limited capacity constrains a system's execution speed or throughput.
- How it is used
- Uses bottleneck analysis to guide GPU optimization, from deciding when to write custom CUDA kernels to selecting attention optimizations.
- How deep it goes
- Traces transfer costs, timed training stages, and attention HBM traffic through naive and fused kernels, while noting production optimizations omitted.
- Examples
- Uses a timed PyTorch training breakdown and naive attention’s repeated HBM transfers as concrete cases.
- What you can do after
- Diagnose likely performance limits by measuring training stages and tracing memory movement across attention kernels.
Also covered in
- Spec-Driven DevelopmentIn early access (MEAP)
- Prompt Engineering in Practice
What to know first
- context switchingContext switching is the transition from one active task or context to another, involving a change in execution state or attention.
- throughputThroughput is the rate at which a system processes or completes work, data, or requests over a specified period.
Related concepts
- FlashAttentionaddressed by · FlashAttention is an exact, I/O-aware attention algorithm that tiles computation to reduce memory traffic without materializing the full attention matrix in high-bandwidth memory.
- event processingaddressed by · Event processing is the computation that captures, interprets, filters, transforms, routes, or responds to discrete events as they arrive or occur.
- theory of constraintsaddressed by · The theory of constraints is a management approach that improves system performance by identifying and addressing the constraint that most limits it.
- throughputconfigures · Throughput is the rate at which a system processes or completes work, data, or requests over a specified period.
Questions about bottleneck
What is bottleneck?
A bottleneck is a constraint, component, or stage that limits the throughput or performance of a larger system or workflow.
Which Manning books explain bottleneck?
Build High Value Software Systems and CUDA for LLMs.
What should I know before learning bottleneck?
Manning books that teach bottleneck usually introduce context switching and throughput first.
These guides are generated from the text of Manning books by Manning's knowledge graph and link to the sections they draw on. The full books are available on liveBook and at manning.com.

