Feature selection
Feature selection is the process of choosing a subset of input features for use in a modeling, analysis, or prediction task.
Machine Learning System Design
Feature selection is the process of choosing a subset of available input features for a machine learning model.
- How it is used
- Chooses informative model inputs while balancing prediction quality, interpretability, and training and serving costs.
- How deep it goes
- Explains filter rankings, wrapper subset search, and nested validation; identifies embedded methods and illustrates Lasso without deriving algorithms.
- Examples
- Uses photo-ranking metadata and tags to weigh greedy selection against transformer input costs.
- What you can do after
- Apply it via filter or wrapper methods to choose a subset, validate the choice, and freeze features before training.
Read how Machine Learning System Design explains feature selection →
Grokking Machine Learning
Feature selection is a technique that chooses relevant input features and removes unnecessary or harmful ones before model training.
- How it is used
- Uses preprocessing to remove unnecessary features before training machine-learning models.
- How deep it goes
- Defines manual removal, demonstrates column dropping, and notes L1 regularization while stopping short of its algorithmic mechanics.
- Examples
- Uses Titanic columns and a student-admissions decision tree to illustrate dropping fields and the limits of greedy selection.
- What you can do after
- Apply it via pandas by removing selected columns before fitting a model, while avoiding conclusions from tree usage alone.
Read how Grokking Machine Learning explains feature selection →
Statistics Every Programmer Needs
Feature selection is the process of choosing a subset of input variables for use in a predictive model.
- How it is used
- Guides predictor choice before fitting binary logistic regression.
- How deep it goes
- Uses class-wise histograms and correlation heat maps for screening, but specifies no formal selection algorithm.
- Examples
- Compares raisin predictors, highlighting MajorAxisLength and Perimeter against Eccentricity and Extent.
- What you can do after
- Evaluate candidate predictors by comparing class-wise distributions and inspecting pairwise correlations.
Read how Statistics Every Programmer Needs explains feature selection →
Machine Learning for Tabular Data: XGBoost, Deep Learning, and AI
Feature sampling selects a subset of input features as candidates for modeling or a split.
- How it is used
- Uses feature selection to reduce noisy inputs, training costs, and model complexity while retaining valuable predictors.
- How deep it goes
- Explains domain filtering, univariate tests, permutation-style shuffling, and per-split sampling, but defers detailed selection tools.
- Examples
- Uses correlation, chi-square analysis, feature shuffling, random-forest split candidates, diverse tree partitions, and majority voting.
- What you can do after
- Apply feature selection through domain checks, univariate tests, and performance-based shuffling before training selected models.
Investing for Programmers
Feature selection is the process of choosing input variables that are most relevant or useful for a predictive model.
- How it is used
- Uses financial and nonfinancial variables to search for predictors of rising stock prices with supervised machine learning.
- How deep it goes
- Defines candidate sources and selection risks, then stops short of specifying an automated feature-ranking procedure.
- Examples
- Uses prices, expenses, earnings, macroeconomic indicators, weather data, and labeled stock-price datasets as examples.
- What you can do after
- Apply it by narrowing candidate variables, matching them to relevant companies, and constructing a labeled time-series dataset.
Read how Investing for Programmers explains feature selection →
Deep Learning with R, Third Edition
Feature selection is the process of choosing a subset of input features that are useful for a machine-learning task.
- How it is used
- Reduces irrelevant or distracting inputs before training to limit overfitting and improve generalization.
- How deep it goes
- Explains usefulness scoring and thresholding, then demonstrates noisy-feature effects experimentally without detailing selection algorithms.
- Examples
- Uses noisy MNIST channels, all-zero channels, validation-accuracy curves, and the top 10,000 IMDb words as examples.
- What you can do after
- Apply feature selection by scoring features, filtering below a threshold, and comparing validation performance against a baseline.
Read how Deep Learning with R, Third Edition explains feature selection →
Outlier Detection in Python
Feature selection is the process of choosing a subset of input features that are most useful for a specified analysis or prediction task.
- How it is used
- Guides outlier-detection preprocessing by retaining features whose unusual values or combinations could matter.
- How deep it goes
- Explains relevance questions, dimensionality effects, and staff-record checks, but stops short of a general selection algorithm.
- Examples
- Uses staff IDs, start dates, duplicate checks, range checks, and rank comparisons as concrete selection tests.
- What you can do after
- Apply feature selection by removing irrelevant dimensions and testing whether rare values or combinations would be interesting.
Read how Outlier Detection in Python explains feature selection →
Evolutionary Deep Learning: Genetic algorithms and neural networks
Feature selection is the process of identifying and retaining input variables that contribute to a model's predictions while excluding less-relevant variables.
- How it is used
- Uses symbolic regression to select useful inputs while coevolving feature scalers and predictive equations.
- How deep it goes
- Demonstrates dataset preparation, toolbox construction, fitness evaluation, and evolution, but does not develop selection theory.
- Examples
- Uses the BH dataset, a 13-input equation, selected ARG variables, and executable DEAP-style evolution listings.
- What you can do after
- Apply coevolved genetic programming and genetic algorithms to discover equations that ignore less-relevant features.
Related concepts
- feature engineeringpart of · Feature engineering is the creation, transformation, or selection of input features from data to provide useful representations for analysis or modeling.
- inputuses · An input is a value, information, or signal supplied to a system, computation, or interface for processing or to influence its behavior.
- L1 regularizationused by · L1 regularization is a technique that adds a penalty proportional to the sum of absolute model-parameter values, encouraging sparse parameter estimates.
- curse of dimensionalityaddresses · The curse of dimensionality is the set of statistical, geometric, and computational difficulties that tend to increase as the number of dimensions grows.
- random forestused by · A random forest is an ensemble model that combines predictions from multiple decision trees built with randomized data samples, feature selections, or both.
These guides are generated from the text of Manning books by Manning's knowledge graph and link to the sections they draw on. The full books are available on liveBook and at manning.com.