liveBook · Manning

Memory bound

As explained in Machine Learning System Design

A bottleneck is a system component or operation that limits overall performance because it constrains throughput or increases processing time.

A bottleneck is a system component or operation that limits overall performance. Profiling measures system performance to locate bottlenecks across the full pipeline and identify promising optimization targets. section 15.4.1

The slowest component is not automatically the best target: a highly optimized model may have little room to improve, while preprocessing may offer more. In one sequential-model pipeline, preprocessing dominated inference time; optimizing it improved system speed by 40%. section 15.4.1

How Machine Learning System Design covers it

How deep it goes
Explains system-wide profiling and target selection through examples, without detailing memory-specific diagnostics.
Examples
Contrasts model and preprocessing runtimes, then recounts a sequential-model pipeline sped up by optimizing preprocessing.
What you can do after
Evaluate optimization targets by profiling the full pipeline and choosing components with room for improvement.
How it connects
Connects profiling with latency, data preprocessing, model inference, and batch inference.
Cover of Machine Learning System Design
liveBookmanning.com
Read Machine Learning System Design on liveBook Get the book at manning.com

Also explained in

Compare how 11 books explain memory bound →

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Machine Learning System Design. The section links point to the parts of the book it draws on.