Model optimization
Model optimization is modifying a trained model or its execution to improve computational efficiency, resource use, or deployment compatibility while preserving acceptable behavior or output quality.
Deep Learning with R, Third Edition
Model optimization is the modification of a trained model to reduce computational, memory, or latency costs while preserving acceptable predictive performance.
- How it is used
- Prepares trained models for inference on constrained devices and low-latency applications.
- How deep it goes
- Defines pruning and quantization with tradeoffs and an API call, but does not detail pruning algorithms or evaluation procedures.
- Examples
- Uses significant-weight pruning and Keras int8 quantization as concrete optimization examples.
- What you can do after
- Apply it via Keras to compress weights before deploying an inference model.
Read how Deep Learning with R, Third Edition explains model optimization →
Machine Learning System Design
Model optimization is the improvement of a trained model's execution efficiency or resource requirements while preserving acceptable predictive behavior.
- How it is used
- Guides framework selection and inference preparation for efficient model serving across heterogeneous hardware.
- How deep it goes
- Explains hardware placement, graph trimming, operation fusion, and LLM constraints, but stops short of implementation tutorials.
- Examples
- Uses CoreML device splitting, ONNX Runtime graph optimization, and VLLM, TGI, and GGML as inference-tool examples.
- What you can do after
- Operate/configure inference tools by selecting frameworks and preparing graphs for target hardware.
Read how Machine Learning System Design explains model optimization →
Related concepts
- pruningincludes · Pruning is the selective removal or restriction of parts of a model, structure, or candidate set, typically to reduce complexity or computation while retaining useful results.
- deploymentconfigures · Deployment is the process of making software, models, or other computational systems available to operate in a target environment.
- quantizationincludes · Quantization maps numerical values to a finite set of discrete representations, often using reduced precision to lower model storage or computation requirements.
- distillationincludes · Knowledge distillation is a training technique in which a student model learns to reproduce or approximate the outputs or behavior of a teacher model.
- ONNXaddressed by · ONNX is an open standard for representing machine-learning models in a portable format that supports exchange and execution across frameworks and runtimes.
These guides are generated from the text of Manning books by Manning's knowledge graph and link to the sections they draw on. The full books are available on liveBook and at manning.com.