Ensemble

As explained in Experimentation for Engineers

A model ensemble is a collection of predictive models whose predictions or decisions are combined, compared, or otherwise used collectively.

An ensemble is a collection of similarly purposed predictive models. Models may be averaged, compared, or used collectively; here, one model is selected randomly for each recommender decision rather than averaging predictions. section 5.3.1 section 5.3

Figure 5.8 Thompson sampling alters both the fit and the policy. You fit several models—called an ensemble—instead of just one. The policy uses a different, randomly selected model for each decision it makes.

Shows how ensemble construction changes both offline fitting and online policy selection: several models are fitted, then one is randomly selected per decision. section 5.3

Build and operate an ensemble through this workflow:

• Draw a bootstrap sample with replacement from logged samples; duplicates are allowed, and the resampled set may differ in size. section 5.3.1
• Fit a model, such as a linear regression, on each bootstrap sample and repeat until the ensemble has the desired number of models. section 5.3.1
• At decision time, select a model uniformly at random, then choose the action that model predicts will produce the highest reward. section 5.3
• Treat disagreement across similarly fitted models as uncertainty: actions selected by more models receive higher probability, while increasing certainty reduces exploration. section 5.3.2

Ten-model post recommendation

Suppose ten equally believable models evaluate two posts. If seven select post #1 as best and three select post #2, randomized probability matching displays post #1 with probability 7/10 and post #2 with probability 3/10. The policy therefore explores in proportion to model disagreement instead of assigning a fixed fraction of decisions to arbitrary actions. section 5.3.2

How Experimentation for Engineers covers it

How deep it goes

Defines ensembles, explains bootstrap construction, and traces online selection, but does not develop general ensemble averaging.

Examples

Uses resampled visit data, a bootstrap regression recommender, and ten models choosing between two posts.

What it assumes

Assumes logged samples, predictive modeling, and linear regression while explaining bootstrap resampling from basic mechanics.

What you can do after

Apply Thompson sampling by fitting bootstrap models and randomly selecting one to choose each highest-reward action.

How it connects

Connects bootstrap sampling, linear regression, Thompson sampling, contextual bandits, and randomized probability matching.

liveBookmanning.com

Read Experimentation for Engineers on liveBook Get the book at manning.com

Also explained in

• 

  liveBookmanning.com
  Outlier Detection in PythonAn ensemble is a combination of multiple models or detectors whose outputs are combined to obtain more robust results than an individual model or detector.
• 

  liveBookmanning.com
  Machine Learning for Tabular Data: XGBoost, Deep Learning, and AIA technique that combines multiple models or their predictions to improve predictive results.
• 

  liveBookmanning.com
  Ensemble Methods for Machine LearningAn ensemble is a predictive model formed by combining the outputs of multiple component models.
• 

  liveBookmanning.com
  Statistics Every Programmer NeedsAn ensemble model combines predictions from multiple constituent models to produce a collective prediction.
• 

  liveBookmanning.com
  Deep Learning with Python, Third EditionA technique that combines multiple models or views to improve performance by exploiting complementary errors or representations.
• 

  liveBookmanning.com
  Machine Learning Algorithms in DepthAn ensemble method combines predictions or decisions from multiple models to obtain a joint predictive result.
• 

  liveBookmanning.com
  Deep Learning with R, Third EditionA model ensemble is a collection of predictive models whose outputs are combined to produce a joint prediction.
• 

  liveBookmanning.com
  Inside Deep Learning: Math, Algorithms, ModelsA collection of multiple models whose predictions are combined, typically to improve accuracy or robustness.
• 

  liveBookmanning.com
  Generative AI in ActionA model ensemble is an architecture that combines multiple machine-learning models to improve overall capability, performance, or reliability.
• 

  liveBookmanning.com
  Building LLM Applications with DSPyA modeling technique that combines outputs from multiple models or programs.

Compare how 11 books explain ensemble →

Related concepts

• random forestincludes · A random forest is an ensemble model that combines predictions from multiple decision trees built with randomized data samples, feature selections, or both.
• gradient boostingincludes · Gradient boosting is an ensemble-learning technique that sequentially adds predictive models, typically weak learners, to reduce a loss by fitting its gradient.
• baggingincludes · Bagging, or bootstrap aggregation, is an ensemble technique that trains multiple models on bootstrap samples of data and combines their predictions.
• language modeluses · A language model is a computational model that represents patterns in linguistic tokens to estimate sequence probabilities and predict or generate language from context.
• decision treeuses · A decision tree is a branching structure that represents sequential tests or choices, with paths leading to outcomes, predictions, or actions.
• diversityuses · Diversity is the degree of variation among members of a set or population in relevant characteristics, perspectives, behaviors, or outcomes.

This explainer was generated by Manning's knowledge graph from the text of Experimentation for Engineers. The section links point to the parts of the book it draws on.