Ensemble
An ensemble is a technique that combines or considers outputs from multiple models or component methods collectively to produce or assess a joint result.
Outlier Detection in Python
An ensemble is a combination of multiple models or detectors whose outputs are combined to obtain more robust results than an individual model or detector.
- How it is used
- Combines complementary outlier detectors to broaden anomaly coverage and stabilize per-record scores.
- How deep it goes
- Works through candidate selection, score scaling, combination rules, and evaluation, including trade-offs rather than a single prescribed formula.
- Examples
- Uses six detectors on baseball records, row-sampled One-Class SVMs, and tables comparing alternative score combinations.
- What you can do after
- Build it from scratch by selecting complementary detectors, making scores comparable, combining them, and testing the resulting ranking.
Machine Learning for Tabular Data: XGBoost, Deep Learning, and AI
A technique that combines multiple models or their predictions to improve predictive results.
- How it is used
- Improves tabular-data predictions by combining tree models or blending gradient boosting with deep learning.
- How deep it goes
- Explains averaging and boosting, sampling variants, bias-variance trade-offs, hyperparameters, and out-of-fold blend evaluation without deriving their mathematics.
- Examples
- Uses animal classification trees, bagged classifiers, and Tokyo Airbnb price predictions blended from XGBoost and fastai.
- What you can do after
- Apply it via scikit-learn bagging or weighted XGBoost-fastai predictions, evaluating candidate blends with out-of-fold metrics.
Read how Machine Learning for Tabular Data: XGBoost, Deep Learning, and AI explains ensemble →
Ensemble Methods for Machine Learning
An ensemble is a predictive model formed by combining the outputs of multiple component models.
- How it is used
- Motivates robust predictive modeling by combining multiple learners for structured-data tasks and practical applications.
- How deep it goes
- Defines diversity and aggregation, implements averaged regression predictions, and classifies parallel versus sequential ensembles without detailing every method.
- Examples
- Uses a diagnostic voting team, Netflix Prize blending, six-regressor Friedman-1 averaging, bike rentals, and demographic income prediction.
- What you can do after
- Apply it via scikit-learn by training diverse regressors, averaging predictions, tuning parallel ensembles, and comparing held-out performance.
Read how Ensemble Methods for Machine Learning explains ensemble →
Statistics Every Programmer Needs
An ensemble model combines predictions from multiple constituent models to produce a collective prediction.
- How it is used
- Improves classification and regression predictions by aggregating diverse decision-tree outputs.
- How deep it goes
- Explains bagging, feature randomness, voting, averaging, feature importance, and evaluation, but does not develop advanced boosting methods.
- Examples
- Uses loan-approval trees, majority voting, accuracy comparisons, confusion matrices, and feature-importance analysis.
- What you can do after
- Evaluate ensemble predictions with accuracy and confusion matrices while interpreting feature contributions and robustness.
Read how Statistics Every Programmer Needs explains ensemble →
Experimentation for Engineers
A model ensemble is a collection of predictive models whose predictions or decisions are combined, compared, or otherwise used collectively.
- How it is used
- Supports Thompson sampling by turning parameter uncertainty into randomized action selection in a recommender.
- How deep it goes
- Defines ensembles, explains bootstrap construction, and traces online selection, but does not develop general ensemble averaging.
- Examples
- Uses resampled visit data, a bootstrap regression recommender, and ten models choosing between two posts.
- What you can do after
- Apply Thompson sampling by fitting bootstrap models and randomly selecting one to choose each highest-reward action.
Deep Learning with Python, Third Edition
A technique that combines multiple models or views to improve performance by exploiting complementary errors or representations.
- How it is used
- Improves predictive performance by combining independently trained models, particularly when moving toward real-world or competition-level results.
- How deep it goes
- Explains prediction averaging, weighted averaging, model diversity, and the danger of including a substantially weaker classifier.
- Examples
- Uses four classifier predictions, equal averaging, weighted averaging, the blind-men-and-elephant analogy, and Kaggle competitions.
- What you can do after
- Apply it by averaging or weighting classifier predictions at inference time and practice model ensembling through Kaggle competitions.
Read how Deep Learning with Python, Third Edition explains ensemble →
Machine Learning Algorithms in Depth
An ensemble method combines predictions or decisions from multiple models to obtain a joint predictive result.
- How it is used
- Deploys multiple models to reduce variance, reduce bias, or improve predictive performance in supervised learning.
- How deep it goes
- Explains sequential and parallel organization, bagging mechanics, and stacking architecture, but only summarizes boosting.
- Examples
- Uses iris decision boundaries, decision trees, KNN, random forests, naive Bayes, logistic regression, voting, and learning curves.
- What you can do after
- Apply it via bagging or stacking implementations and compare ensemble accuracy, decision boundaries, learning curves, and computational cost.
Read how Machine Learning Algorithms in Depth explains ensemble →
Deep Learning with R, Third Edition
A model ensemble is a collection of predictive models whose outputs are combined to produce a joint prediction.
- How it is used
- Improves predictive performance by pooling independently trained models, especially in demanding real-world machine-learning workflows.
- How deep it goes
- Explains model diversity, equal averaging, and validation-weighted averaging, but does not derive ensemble theory mathematically.
- Examples
- Uses the blind-men-and-elephant analogy, four classifier predictions, equal averaging, and validation-weighted averaging.
- What you can do after
- Apply prediction averaging or validation-weighted averaging to combine several trained models into a stronger prediction.
Read how Deep Learning with R, Third Edition explains ensemble →
Inside Deep Learning: Math, Algorithms, Models
A collection of multiple models whose predictions are combined, typically to improve accuracy or robustness.
- How it is used
- Uses multiple predictions to improve accuracy while reducing the cost of independently training many neural networks.
- How deep it goes
- Explains snapshot timing, the Tmax formula, and parameter averaging, but does not provide ensemble-prediction code.
- Examples
- Uses 20 snapshots across T epochs and contrasts them with independently training 20 neural networks.
- What you can do after
- Apply it via cosine annealing or stochastic weight averaging to obtain ensemble-like accuracy with lower training or storage costs.
Read how Inside Deep Learning: Math, Algorithms, Models explains ensemble →
Generative AI in Action
A model ensemble is an architecture that combines multiple machine-learning models to improve overall capability, performance, or reliability.
- How it is used
- Deploys complementary models to route requests, improve response quality, and increase reliability in generative-AI applications.
- How deep it goes
- Explains selection, routing, integration, scaling, redundancy, and workload handling, without deriving ensemble mathematics or model internals.
- Examples
- Uses Phi-2 as a local dog-question classifier and GPT-4 as the answering model for irrelevant questions.
- What you can do after
- Build it from scratch by selecting models, implementing routing, integrating endpoints, and planning resilience for demand spikes.
Building LLM Applications with DSPy
A modeling technique that combines outputs from multiple models or programs.
- How it is used
- Combines outputs from multiple programs to seek higher-quality results in language-model applications.
- How deep it goes
- Specifies program compilation, inference-time execution, and reduction, but does not detail how to build the constituent programs.
- Examples
- Uses classifier labels, a flight-meal query, majority voting, and an agreement-based “Uncertain” result.
- What you can do after
- Apply DSPy's Ensemble optimizer to combine programs with a majority-vote or custom reduce function.
Read how Building LLM Applications with DSPy explains ensemble →
Related concepts
- random forestincludes · A random forest is an ensemble model that combines predictions from multiple decision trees built with randomized data samples, feature selections, or both.
- gradient boostingincludes · Gradient boosting is an ensemble-learning technique that sequentially adds predictive models, typically weak learners, to reduce a loss by fitting its gradient.
- baggingincludes · Bagging, or bootstrap aggregation, is an ensemble technique that trains multiple models on bootstrap samples of data and combines their predictions.
- language modeluses · A language model is a computational model that represents patterns in linguistic tokens to estimate sequence probabilities and predict or generate language from context.
- decision treeuses · A decision tree is a branching structure that represents sequential tests or choices, with paths leading to outcomes, predictions, or actions.
- diversityuses · Diversity is the degree of variation among members of a set or population in relevant characteristics, perspectives, behaviors, or outcomes.
These guides are generated from the text of Manning books by Manning's knowledge graph and link to the sections they draw on. The full books are available on liveBook and at manning.com.