Ensemble
An ensemble is a technique or model that combines or considers outputs from multiple models, methods, or components to produce or assess a joint result.
Outlier Detection in Python
An ensemble is a combination of multiple models or detectors whose outputs are combined to obtain more robust results than an individual model or detector.
- How it is used
- Combines complementary outlier detectors to broaden anomaly coverage and stabilize per-record scores.
- How deep it goes
- Works through candidate selection, score scaling, combination rules, and evaluation, including trade-offs rather than a single prescribed formula.
- Examples
- Uses six detectors on baseball records, row-sampled One-Class SVMs, and tables comparing alternative score combinations.
- What you can do after
- Build it from scratch by selecting complementary detectors, making scores comparable, combining them, and testing the resulting ranking.
Machine Learning for Tabular Data: XGBoost, Deep Learning, and AI
A technique that combines multiple models or their predictions to improve predictive results.
- How it is used
- Improves tabular-data predictions by combining tree models or blending gradient boosting with deep learning.
- How deep it goes
- Explains averaging and boosting, sampling variants, bias-variance trade-offs, hyperparameters, and out-of-fold blend evaluation without deriving their mathematics.
- Examples
- Uses animal classification trees, bagged classifiers, and Tokyo Airbnb price predictions blended from XGBoost and fastai.
- What you can do after
- Apply it via scikit-learn bagging or weighted XGBoost-fastai predictions, evaluating candidate blends with out-of-fold metrics.
Read how Machine Learning for Tabular Data: XGBoost, Deep Learning, and AI explains ensemble →
Ensemble Methods for Machine Learning
An ensemble is a predictive model formed by combining the outputs of multiple component models.
- How it is used
- Motivates robust predictive modeling by combining multiple learners for structured-data tasks and practical applications.
- How deep it goes
- Defines diversity and aggregation, implements averaged regression predictions, and classifies parallel versus sequential ensembles without detailing every method.
- Examples
- Uses a diagnostic voting team, Netflix Prize blending, six-regressor Friedman-1 averaging, bike rentals, and demographic income prediction.
- What you can do after
- Apply it via scikit-learn by training diverse regressors, averaging predictions, tuning parallel ensembles, and comparing held-out performance.
Read how Ensemble Methods for Machine Learning explains ensemble →
Statistics Every Programmer Needs
An ensemble model combines predictions from multiple constituent models to produce a collective prediction.
- How it is used
- Improves classification and regression predictions by aggregating diverse decision-tree outputs.
- How deep it goes
- Explains bagging, feature randomness, voting, averaging, feature importance, and evaluation, but does not develop advanced boosting methods.
- Examples
- Uses loan-approval trees, majority voting, accuracy comparisons, confusion matrices, and feature-importance analysis.
- What you can do after
- Evaluate ensemble predictions with accuracy and confusion matrices while interpreting feature contributions and robustness.
Read how Statistics Every Programmer Needs explains ensemble →
Experimentation for Engineers
A model ensemble is a collection of predictive models whose predictions or decisions are combined, compared, or otherwise used collectively.
- How it is used
- Supports Thompson sampling by turning parameter uncertainty into randomized action selection in a recommender.
- How deep it goes
- Defines ensembles, explains bootstrap construction, and traces online selection, but does not develop general ensemble averaging.
- Examples
- Uses resampled visit data, a bootstrap regression recommender, and ten models choosing between two posts.
- What you can do after
- Apply Thompson sampling by fitting bootstrap models and randomly selecting one to choose each highest-reward action.
Build AI Drug Discovery Pipelines
Ensemble learning combines multiple individual predictive models to form a composite model with improved robustness or accuracy.
- How it is used
- Deploys collections of predictive models to improve robustness and accuracy in drug-discovery classification and QSAR workflows.
- How deep it goes
- Explains bootstrap sampling, voting, averaging, out-of-bag evaluation, interpretation, and conformer aggregation while deferring multiple-instance implementation.
- Examples
- Demonstrates CYP3A4 inhibition prediction with molecular fingerprints, out-of-bag metrics, feature importance, similarity maps, and conformer-level consensus.
- What you can do after
- Build random-forest ensembles, evaluate them with out-of-bag estimates, and inspect their feature importance and applicability-domain behavior.
Read how Build AI Drug Discovery Pipelines explains ensemble →
AI Model Evaluation
An evaluation setup that combines judgments from multiple evaluators, models, runs, or prompts for a single assessment.
- How it is used
- Strengthens LLM-as-a-judge evaluation when a single opinion is insufficient for high-stakes, subjective, or unvalidated decisions.
- How deep it goes
- Details construction methods, aggregation, disagreement routing, and cost trade-offs, while deferring broader validation and escalation patterns.
- Examples
- Uses repeated samples, distinct models, varied prompts, launch gates, safety judgments, production checks, and periodic audits.
- What you can do after
- Operate a judge panel that aggregates verdicts, measures agreement, and routes uncertain evaluations for deeper review.
Deep Learning with Python, Third Edition
A technique that combines multiple models or views to improve performance by exploiting complementary errors or representations.
- How it is used
- Improves predictive performance by combining independently trained models, particularly when moving toward real-world or competition-level results.
- How deep it goes
- Explains prediction averaging, weighted averaging, model diversity, and the danger of including a substantially weaker classifier.
- Examples
- Uses four classifier predictions, equal averaging, weighted averaging, the blind-men-and-elephant analogy, and Kaggle competitions.
- What you can do after
- Apply it by averaging or weighting classifier predictions at inference time and practice model ensembling through Kaggle competitions.
Read how Deep Learning with Python, Third Edition explains ensemble →
LLM Evaluation and Alignment, The Foundational Ideas
An ensemble is a collection of separately trained models whose predictions are combined or compared to improve information or reliability.
- How it is used
- Uses reward-model disagreement to select informative human preference queries.
- How deep it goes
- Explains ensemble construction and query selection, but stops short of implementation code or uncertainty formulas.
- Examples
- Uses simulated MuJoCo backflips and Atari games to illustrate labeling efficiency.
- What you can do after
- Apply ensemble disagreement to prioritize preference comparisons under a limited labeling budget.
Read how LLM Evaluation and Alignment, The Foundational Ideas explains ensemble →
Machine Learning Algorithms in Depth
An ensemble method combines predictions or decisions from multiple models to obtain a joint predictive result.
- How it is used
- Deploys multiple models to reduce variance, reduce bias, or improve predictive performance in supervised learning.
- How deep it goes
- Explains sequential and parallel organization, bagging mechanics, and stacking architecture, but only summarizes boosting.
- Examples
- Uses iris decision boundaries, decision trees, KNN, random forests, naive Bayes, logistic regression, voting, and learning curves.
- What you can do after
- Apply it via bagging or stacking implementations and compare ensemble accuracy, decision boundaries, learning curves, and computational cost.
Read how Machine Learning Algorithms in Depth explains ensemble →
Deep Learning with R, Third Edition
A model ensemble is a collection of predictive models whose outputs are combined to produce a joint prediction.
- How it is used
- Improves predictive performance by pooling independently trained models, especially in demanding real-world machine-learning workflows.
- How deep it goes
- Explains model diversity, equal averaging, and validation-weighted averaging, but does not derive ensemble theory mathematically.
- Examples
- Uses the blind-men-and-elephant analogy, four classifier predictions, equal averaging, and validation-weighted averaging.
- What you can do after
- Apply prediction averaging or validation-weighted averaging to combine several trained models into a stronger prediction.
Read how Deep Learning with R, Third Edition explains ensemble →
Human-in-the-Loop Machine Learning: Active learning and annotation for human-centered AI
A method that combines or compares predictions from multiple models to support a prediction or decision.
- How it is used
- Uses disagreement among model predictions to estimate uncertainty and select unlabeled items for human review.
- How deep it goes
- Details rank aggregation, confidence statistics, model-diversity strategies, and IoU handling for missing localization predictions.
- Examples
- Demonstrates random forests, neural-model committees, object-detection localization, and uncertainty estimation for language generation.
- What you can do after
- Apply model disagreement or aggregated confidence to rank unlabeled items for human review.
Inside Deep Learning: Math, Algorithms, Models
A collection of multiple models whose predictions are combined, typically to improve accuracy or robustness.
- How it is used
- Improves predictive accuracy by averaging predictions from separately trained models, while motivating cosine-annealing snapshots as a cheaper alternative.
- How deep it goes
- Defines prediction averaging, then describes saving weights at cosine-schedule minima and averaging parameters with SWA.
- Examples
- Contrasts training 20 networks with saving 20 weight snapshots from one cosine-annealed run, and mentions averaging those parameters with SWA.
- What you can do after
- Apply cosine annealing by saving weights at learning-rate dips and combining their predictions, or average the parameters with SWA.
Read how Inside Deep Learning: Math, Algorithms, Models explains ensemble →
Generative AI in Action
A model ensemble is an architecture that combines multiple machine-learning models to improve overall capability, performance, or reliability.
- How it is used
- Deploys complementary models to route requests, improve response quality, and increase reliability in generative-AI applications.
- How deep it goes
- Explains selection, routing, integration, scaling, redundancy, and workload handling, without deriving ensemble mathematics or model internals.
- Examples
- Uses Phi-2 as a local dog-question classifier and GPT-4 as the answering model for irrelevant questions.
- What you can do after
- Build it from scratch by selecting models, implementing routing, integrating endpoints, and planning resilience for demand spikes.
Timeless Algorithms, The Foundational Ideas
Ensemble learning combines multiple predictive models to improve predictions.
- How it is used
- Combines multiple models to stabilize predictions and improve generalization.
- How deep it goes
- Explains aggregation, variance reduction, model diversity, and how correlated errors limit ensemble gains.
- Examples
- Uses bagged trees, random forests, AdaBoost, gradient boosting, and XGBoost as examples of ensemble methods.
- What you can do after
- Evaluate ensemble behavior by distinguishing variance reduction from the limits imposed by shared errors.
Read how Timeless Algorithms, The Foundational Ideas explains ensemble →
Building LLM Applications with DSPy
A modeling technique that combines outputs from multiple models or programs.
- How it is used
- Combines outputs from multiple programs to seek higher-quality results in language-model applications.
- How deep it goes
- Specifies program compilation, inference-time execution, and reduction, but does not detail how to build the constituent programs.
- Examples
- Uses classifier labels, a flight-meal query, majority voting, and an agreement-based “Uncertain” result.
- What you can do after
- Apply DSPy's Ensemble optimizer to combine programs with a majority-vote or custom reduce function.
Read how Building LLM Applications with DSPy explains ensemble →
Also covered in
- Advanced Algorithms and Data Structures
- Grokking Machine Learning
- Distributed Machine Learning Patterns
- AI Agents and Applications
- Financial AI in Practice
- Sutskever's List
What to know first
- language modelA language model is a computational model that represents patterns in linguistic sequences to assign probabilities, predict, generate, or evaluate language.
- accuracyAccuracy is the degree to which a result agrees with a specified truth, reference, or criterion, commonly measured as the proportion of evaluated predictions that are correct.
- decision treeA decision tree is a tree-structured model or representation in which successive tests or choices lead through branches to outcomes, predictions, or actions.
- random forestA random forest is an ensemble learning model that aggregates predictions from multiple decision trees trained with randomized data samples, feature subsets, or both.
Related concepts
- random forestincludes · A random forest is an ensemble learning model that aggregates predictions from multiple decision trees trained with randomized data samples, feature subsets, or both.
- baggingincludes · Bagging, or bootstrap aggregation, is an ensemble technique that trains multiple models on bootstrap samples of data and combines their predictions.
- gradient boostingincludes · Gradient boosting is an ensemble-learning technique that sequentially adds predictive models fitted to reduce a loss by following its gradient.
- decision treeuses · A decision tree is a tree-structured model or representation in which successive tests or choices lead through branches to outcomes, predictions, or actions.
- diversityuses · Diversity is the degree of variation among members of a set or population in characteristics relevant to that set or population.
- language modeluses · A language model is a computational model that represents patterns in linguistic sequences to assign probabilities, predict, generate, or evaluate language.
Questions about ensemble
What is ensemble?
An ensemble is a technique or model that combines or considers outputs from multiple models, methods, or components to produce or assess a joint result.
Which Manning books explain ensemble?
Outlier Detection in Python, Machine Learning for Tabular Data: XGBoost, Deep Learning, and AI, Ensemble Methods for Machine Learning, Statistics Every Programmer Needs, Experimentation for Engineers, Build AI Drug Discovery Pipelines, AI Model Evaluation, Deep Learning with Python, Third Edition and 8 more.
What should I know before learning ensemble?
Manning books that teach ensemble usually introduce language model, accuracy, decision tree and random forest first.
What kinds of ensemble are there?
Kinds of ensemble include random forest, bagging and gradient boosting.
What is ensemble also called?
Ensemble is also called ensembling.
These guides are generated from the text of Manning books by Manning's knowledge graph and link to the sections they draw on. The full books are available on liveBook and at manning.com.















