liveBook · Manning

Wrapper method

As explained in Machine Learning System Design

A feature-selection technique that evaluates feature subsets by repeatedly training or assessing a predictive model.

A wrapper method selects feature subsets by repeatedly evaluating a predictive model against a chosen metric. The learning algorithm is “wrapped” inside the search process, so the selection reflects model performance rather than feature scores alone. section 11.3.3

Common search strategies include:

  • Sequential forward selection adds features one by one, starting from an empty set. section 11.3.3
  • Sequential backward elimination removes features one by one; floating variants can reverse earlier inclusion or exclusion decisions. section 11.3.3
  • Evolutionary algorithms sample subsets stochastically, including binary feature masks where 1 means included and 0 means excluded. section 11.3.3

A practical caution

Wrapper methods can provide an accurate estimate of how a subset affects the target metric, but they require nested validation and can be computationally intensive or converge to local optima. section 11.3.3

Figure 11.8 Families of feature selection methods
figure

Use the feature-selection family diagram to place wrapper methods alongside alternative selection approaches. section 11.3.3

How Machine Learning System Design covers it

How deep it goes
Defines the method, explains nested validation, names search variants, and cautions about computation and local optima.
Examples
Uses forward selection, backward elimination, floating variants, and binary feature masks as concrete search examples.
What it assumes
Assumes a predictive model, target metric, and validation schema, while emphasizing nested validation for reliable selection.
What you can do after
Apply it via sequential or evolutionary search to evaluate subsets against a chosen model metric.
How it connects
Connects feature selection with model validation, sequential search, evolutionary algorithms, feature masks, and embedded methods.
Cover of Machine Learning System Design
liveBookmanning.com
Read Machine Learning System Design on liveBook Get the book at manning.com

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Machine Learning System Design. The section links point to the parts of the book it draws on.