liveBook · Manning

Ensemble

Also called ensembling

An ensemble is a technique or model that combines or considers outputs from multiple models, methods, or components to produce or assess a joint result.

How 16 Manning books explain it

Outlier Detection in Python

An ensemble is a combination of multiple models or detectors whose outputs are combined to obtain more robust results than an individual model or detector.

How it is used
Combines complementary outlier detectors to broaden anomaly coverage and stabilize per-record scores.
How deep it goes
Works through candidate selection, score scaling, combination rules, and evaluation, including trade-offs rather than a single prescribed formula.
Examples
Uses six detectors on baseball records, row-sampled One-Class SVMs, and tables comparing alternative score combinations.
What you can do after
Build it from scratch by selecting complementary detectors, making scores comparable, combining them, and testing the resulting ranking.

Read how Outlier Detection in Python explains ensemble →

Machine Learning for Tabular Data: XGBoost, Deep Learning, and AI

A technique that combines multiple models or their predictions to improve predictive results.

How it is used
Improves tabular-data predictions by combining tree models or blending gradient boosting with deep learning.
How deep it goes
Explains averaging and boosting, sampling variants, bias-variance trade-offs, hyperparameters, and out-of-fold blend evaluation without deriving their mathematics.
Examples
Uses animal classification trees, bagged classifiers, and Tokyo Airbnb price predictions blended from XGBoost and fastai.
What you can do after
Apply it via scikit-learn bagging or weighted XGBoost-fastai predictions, evaluating candidate blends with out-of-fold metrics.

Read how Machine Learning for Tabular Data: XGBoost, Deep Learning, and AI explains ensemble →

Ensemble Methods for Machine Learning

An ensemble is a predictive model formed by combining the outputs of multiple component models.

How it is used
Motivates robust predictive modeling by combining multiple learners for structured-data tasks and practical applications.
How deep it goes
Defines diversity and aggregation, implements averaged regression predictions, and classifies parallel versus sequential ensembles without detailing every method.
Examples
Uses a diagnostic voting team, Netflix Prize blending, six-regressor Friedman-1 averaging, bike rentals, and demographic income prediction.
What you can do after
Apply it via scikit-learn by training diverse regressors, averaging predictions, tuning parallel ensembles, and comparing held-out performance.

Read how Ensemble Methods for Machine Learning explains ensemble →

Statistics Every Programmer Needs

An ensemble model combines predictions from multiple constituent models to produce a collective prediction.

How it is used
Improves classification and regression predictions by aggregating diverse decision-tree outputs.
How deep it goes
Explains bagging, feature randomness, voting, averaging, feature importance, and evaluation, but does not develop advanced boosting methods.
Examples
Uses loan-approval trees, majority voting, accuracy comparisons, confusion matrices, and feature-importance analysis.
What you can do after
Evaluate ensemble predictions with accuracy and confusion matrices while interpreting feature contributions and robustness.

Read how Statistics Every Programmer Needs explains ensemble →

Experimentation for Engineers

A model ensemble is a collection of predictive models whose predictions or decisions are combined, compared, or otherwise used collectively.

How it is used
Supports Thompson sampling by turning parameter uncertainty into randomized action selection in a recommender.
How deep it goes
Defines ensembles, explains bootstrap construction, and traces online selection, but does not develop general ensemble averaging.
Examples
Uses resampled visit data, a bootstrap regression recommender, and ten models choosing between two posts.
What you can do after
Apply Thompson sampling by fitting bootstrap models and randomly selecting one to choose each highest-reward action.

Read how Experimentation for Engineers explains ensemble →

Build AI Drug Discovery Pipelines

Ensemble learning combines multiple individual predictive models to form a composite model with improved robustness or accuracy.

How it is used
Deploys collections of predictive models to improve robustness and accuracy in drug-discovery classification and QSAR workflows.
How deep it goes
Explains bootstrap sampling, voting, averaging, out-of-bag evaluation, interpretation, and conformer aggregation while deferring multiple-instance implementation.
Examples
Demonstrates CYP3A4 inhibition prediction with molecular fingerprints, out-of-bag metrics, feature importance, similarity maps, and conformer-level consensus.
What you can do after
Build random-forest ensembles, evaluate them with out-of-bag estimates, and inspect their feature importance and applicability-domain behavior.

Read how Build AI Drug Discovery Pipelines explains ensemble →

AI Model Evaluation

An evaluation setup that combines judgments from multiple evaluators, models, runs, or prompts for a single assessment.

How it is used
Strengthens LLM-as-a-judge evaluation when a single opinion is insufficient for high-stakes, subjective, or unvalidated decisions.
How deep it goes
Details construction methods, aggregation, disagreement routing, and cost trade-offs, while deferring broader validation and escalation patterns.
Examples
Uses repeated samples, distinct models, varied prompts, launch gates, safety judgments, production checks, and periodic audits.
What you can do after
Operate a judge panel that aggregates verdicts, measures agreement, and routes uncertain evaluations for deeper review.

Read how AI Model Evaluation explains ensemble →

Deep Learning with Python, Third Edition

A technique that combines multiple models or views to improve performance by exploiting complementary errors or representations.

How it is used
Improves predictive performance by combining independently trained models, particularly when moving toward real-world or competition-level results.
How deep it goes
Explains prediction averaging, weighted averaging, model diversity, and the danger of including a substantially weaker classifier.
Examples
Uses four classifier predictions, equal averaging, weighted averaging, the blind-men-and-elephant analogy, and Kaggle competitions.
What you can do after
Apply it by averaging or weighting classifier predictions at inference time and practice model ensembling through Kaggle competitions.

Read how Deep Learning with Python, Third Edition explains ensemble →

LLM Evaluation and Alignment, The Foundational Ideas

An ensemble is a collection of separately trained models whose predictions are combined or compared to improve information or reliability.

How it is used
Uses reward-model disagreement to select informative human preference queries.
How deep it goes
Explains ensemble construction and query selection, but stops short of implementation code or uncertainty formulas.
Examples
Uses simulated MuJoCo backflips and Atari games to illustrate labeling efficiency.
What you can do after
Apply ensemble disagreement to prioritize preference comparisons under a limited labeling budget.

Read how LLM Evaluation and Alignment, The Foundational Ideas explains ensemble →

Machine Learning Algorithms in Depth

An ensemble method combines predictions or decisions from multiple models to obtain a joint predictive result.

How it is used
Deploys multiple models to reduce variance, reduce bias, or improve predictive performance in supervised learning.
How deep it goes
Explains sequential and parallel organization, bagging mechanics, and stacking architecture, but only summarizes boosting.
Examples
Uses iris decision boundaries, decision trees, KNN, random forests, naive Bayes, logistic regression, voting, and learning curves.
What you can do after
Apply it via bagging or stacking implementations and compare ensemble accuracy, decision boundaries, learning curves, and computational cost.

Read how Machine Learning Algorithms in Depth explains ensemble →

Deep Learning with R, Third Edition

A model ensemble is a collection of predictive models whose outputs are combined to produce a joint prediction.

How it is used
Improves predictive performance by pooling independently trained models, especially in demanding real-world machine-learning workflows.
How deep it goes
Explains model diversity, equal averaging, and validation-weighted averaging, but does not derive ensemble theory mathematically.
Examples
Uses the blind-men-and-elephant analogy, four classifier predictions, equal averaging, and validation-weighted averaging.
What you can do after
Apply prediction averaging or validation-weighted averaging to combine several trained models into a stronger prediction.

Read how Deep Learning with R, Third Edition explains ensemble →

Human-in-the-Loop Machine Learning: Active learning and annotation for human-centered AI

A method that combines or compares predictions from multiple models to support a prediction or decision.

How it is used
Uses disagreement among model predictions to estimate uncertainty and select unlabeled items for human review.
How deep it goes
Details rank aggregation, confidence statistics, model-diversity strategies, and IoU handling for missing localization predictions.
Examples
Demonstrates random forests, neural-model committees, object-detection localization, and uncertainty estimation for language generation.
What you can do after
Apply model disagreement or aggregated confidence to rank unlabeled items for human review.

Read how Human-in-the-Loop Machine Learning: Active learning and annotation for human-centered AI explains ensemble →

Inside Deep Learning: Math, Algorithms, Models

A collection of multiple models whose predictions are combined, typically to improve accuracy or robustness.

How it is used
Improves predictive accuracy by averaging predictions from separately trained models, while motivating cosine-annealing snapshots as a cheaper alternative.
How deep it goes
Defines prediction averaging, then describes saving weights at cosine-schedule minima and averaging parameters with SWA.
Examples
Contrasts training 20 networks with saving 20 weight snapshots from one cosine-annealed run, and mentions averaging those parameters with SWA.
What you can do after
Apply cosine annealing by saving weights at learning-rate dips and combining their predictions, or average the parameters with SWA.

Read how Inside Deep Learning: Math, Algorithms, Models explains ensemble →

Generative AI in Action

A model ensemble is an architecture that combines multiple machine-learning models to improve overall capability, performance, or reliability.

How it is used
Deploys complementary models to route requests, improve response quality, and increase reliability in generative-AI applications.
How deep it goes
Explains selection, routing, integration, scaling, redundancy, and workload handling, without deriving ensemble mathematics or model internals.
Examples
Uses Phi-2 as a local dog-question classifier and GPT-4 as the answering model for irrelevant questions.
What you can do after
Build it from scratch by selecting models, implementing routing, integrating endpoints, and planning resilience for demand spikes.

Read how Generative AI in Action explains ensemble →

Timeless Algorithms, The Foundational Ideas

Ensemble learning combines multiple predictive models to improve predictions.

How it is used
Combines multiple models to stabilize predictions and improve generalization.
How deep it goes
Explains aggregation, variance reduction, model diversity, and how correlated errors limit ensemble gains.
Examples
Uses bagged trees, random forests, AdaBoost, gradient boosting, and XGBoost as examples of ensemble methods.
What you can do after
Evaluate ensemble behavior by distinguishing variance reduction from the limits imposed by shared errors.

Read how Timeless Algorithms, The Foundational Ideas explains ensemble →

Building LLM Applications with DSPy

A modeling technique that combines outputs from multiple models or programs.

How it is used
Combines outputs from multiple programs to seek higher-quality results in language-model applications.
How deep it goes
Specifies program compilation, inference-time execution, and reduction, but does not detail how to build the constituent programs.
Examples
Uses classifier labels, a flight-meal query, majority voting, and an agreement-based “Uncertain” result.
What you can do after
Apply DSPy's Ensemble optimizer to combine programs with a majority-vote or custom reduce function.

Read how Building LLM Applications with DSPy explains ensemble →

Also covered in

What to know first

Related concepts

Questions about ensemble

What is ensemble?

An ensemble is a technique or model that combines or considers outputs from multiple models, methods, or components to produce or assess a joint result.

Which Manning books explain ensemble?

Outlier Detection in Python, Machine Learning for Tabular Data: XGBoost, Deep Learning, and AI, Ensemble Methods for Machine Learning, Statistics Every Programmer Needs, Experimentation for Engineers, Build AI Drug Discovery Pipelines, AI Model Evaluation, Deep Learning with Python, Third Edition and 8 more.

What should I know before learning ensemble?

Manning books that teach ensemble usually introduce language model, accuracy, decision tree and random forest first.

What kinds of ensemble are there?

Kinds of ensemble include random forest, bagging and gradient boosting.

What is ensemble also called?

Ensemble is also called ensembling.

These guides are generated from the text of Manning books by Manning's knowledge graph and link to the sections they draw on. The full books are available on liveBook and at manning.com.