Model optimization
Model optimization is modifying a trained model or its execution to improve computational efficiency, resource use, or deployment compatibility while preserving acceptable behavior or output quality.
Deep Learning with R, Third Edition
Model optimization is the modification of a trained model to reduce computational, memory, or latency costs while preserving acceptable predictive performance.
- How it is used
- Prepares trained models for inference on constrained devices and low-latency applications.
- How deep it goes
- Defines pruning and quantization with tradeoffs and an API call, but does not detail pruning algorithms or evaluation procedures.
- Examples
- Uses significant-weight pruning and Keras int8 quantization as concrete optimization examples.
- What you can do after
- Apply it via Keras to compress weights before deploying an inference model.
Read how Deep Learning with R, Third Edition explains model optimization →
Machine Learning System Design
Model optimization is the improvement of a trained model's execution efficiency or resource requirements while preserving acceptable predictive behavior.
- How it is used
- Guides framework selection and inference preparation for efficient model serving across heterogeneous hardware.
- How deep it goes
- Explains hardware placement, graph trimming, operation fusion, and LLM constraints, but stops short of implementation tutorials.
- Examples
- Uses CoreML device splitting, ONNX Runtime graph optimization, and VLLM, TGI, and GGML as inference-tool examples.
- What you can do after
- Operate/configure inference tools by selecting frameworks and preparing graphs for target hardware.
Read how Machine Learning System Design explains model optimization →
Also covered in
- Machine Learning Platform Engineering
- Fight Fraud with Machine LearningIn early access (MEAP)
- AI Model EvaluationIn early access (MEAP)
What to know first
- language modelA language model is a computational model of linguistic sequences that estimates their probabilities or predicts tokens from context, enabling it to generate or evaluate language.
- parallelismParallelism is the concurrent execution or progress of multiple independent tasks or computational operations rather than their strictly sequential execution.
- ONNXONNX (Open Neural Network Exchange) is an open standard and format for representing and exchanging machine-learning models across frameworks, tools, and runtimes.
Related concepts
- optimizationkind of · Optimization is the systematic selection or modification of values, configurations, or implementations to improve a defined objective, often subject to constraints.
- pruningincludes · Pruning is the selective removal or restriction of components, branches, or candidates to reduce complexity or computation while retaining useful structure or results.
- deploymentconfigures · Deployment is the process of placing or releasing software, models, or configuration into a target environment so they can operate there.
- quantizationincludes · Model quantization represents model parameters or computations with reduced-precision numerical values, often to lower storage or computational requirements.
- distillationincludes · Distillation is a training technique in which a student model learns to approximate the outputs or behavior of a teacher model.
- structural modificationuses · A structural modification is a modification of a model's architecture that physically removes or bypasses components.
Questions about model optimization
What is model optimization?
Model optimization is modifying a trained model or its execution to improve computational efficiency, resource use, or deployment compatibility while preserving acceptable behavior or output quality.
Which Manning books explain model optimization?
Deep Learning with R, Third Edition and Machine Learning System Design.
What should I know before learning model optimization?
Manning books that teach model optimization usually introduce language model, parallelism and ONNX first.
What kinds of model optimization are there?
Kinds of model optimization include pruning, quantization and distillation.
What is model optimization a kind of?
Model optimization is a kind of optimization. Optimization is the systematic selection or modification of values, configurations, or implementations to improve a defined objective, often subject to constraints.
These guides are generated from the text of Manning books by Manning's knowledge graph and link to the sections they draw on. The full books are available on liveBook and at manning.com.

