Feature selection
Feature selection is the process of choosing a subset of input features for use in a modeling, analysis, or prediction task.
Machine Learning System Design
Feature selection is the process of choosing a subset of available input features for a machine learning model.
- How it is used
- Uses feature selection to control model and pipeline complexity while balancing predictive quality, interpretability, and serving speed.
- How deep it goes
- Compares filter, wrapper, and embedded families, outlining ranking, sequential selection, and validation considerations without detailed algorithm tutorials.
- Examples
- Uses photo-ranking metadata and a latency-sensitive personalization scenario to illustrate selection decisions.
- What you can do after
- Evaluate feature subsets against predictive quality, latency, and pipeline costs while accounting for validation bias.
Read how Machine Learning System Design explains feature selection →
Grokking Machine Learning
Feature selection is a technique that chooses relevant input features and removes unnecessary or harmful ones before model training.
- How it is used
- Uses preprocessing to remove unnecessary features before training machine-learning models.
- How deep it goes
- Defines manual removal, demonstrates column dropping, and notes L1 regularization while stopping short of its algorithmic mechanics.
- Examples
- Uses Titanic columns and a student-admissions decision tree to illustrate dropping fields and the limits of greedy selection.
- What you can do after
- Apply it via pandas by removing selected columns before fitting a model, while avoiding conclusions from tree usage alone.
Read how Grokking Machine Learning explains feature selection →
Statistics Every Programmer Needs
Feature selection is the process of choosing a subset of input variables for use in a predictive model.
- How it is used
- Guides predictor choice before fitting binary logistic regression.
- How deep it goes
- Uses class-wise histograms and correlation heat maps for screening, but specifies no formal selection algorithm.
- Examples
- Compares raisin predictors, highlighting MajorAxisLength and Perimeter against Eccentricity and Extent.
- What you can do after
- Evaluate candidate predictors by comparing class-wise distributions and inspecting pairwise correlations.
Read how Statistics Every Programmer Needs explains feature selection →
Machine Learning for Tabular Data: XGBoost, Deep Learning, and AI
Feature sampling selects a subset of input features as candidates for modeling or a split.
- How it is used
- Uses feature selection to reduce noisy inputs, training costs, and model complexity while retaining valuable predictors.
- How deep it goes
- Explains domain filtering, univariate tests, permutation-style shuffling, and per-split sampling, but defers detailed selection tools.
- Examples
- Uses correlation, chi-square analysis, feature shuffling, random-forest split candidates, diverse tree partitions, and majority voting.
- What you can do after
- Apply feature selection through domain checks, univariate tests, and performance-based shuffling before training selected models.
Investing for Programmers
Feature selection is the process of choosing input variables that are most relevant or useful for a predictive model.
- How it is used
- Uses financial and nonfinancial variables to search for predictors of rising stock prices with supervised machine learning.
- How deep it goes
- Defines candidate sources and selection risks, then stops short of specifying an automated feature-ranking procedure.
- Examples
- Uses prices, expenses, earnings, macroeconomic indicators, weather data, and labeled stock-price datasets as examples.
- What you can do after
- Apply it by narrowing candidate variables, matching them to relevant companies, and constructing a labeled time-series dataset.
Read how Investing for Programmers explains feature selection →
Deep Learning with R, Third Edition
Feature selection is the process of choosing a subset of input features that are useful for a machine-learning task.
- How it is used
- Reduces irrelevant or distracting inputs before training to limit overfitting and improve generalization.
- How deep it goes
- Explains usefulness scoring and thresholding, then demonstrates noisy-feature effects experimentally without detailing selection algorithms.
- Examples
- Uses noisy MNIST channels, all-zero channels, validation-accuracy curves, and the top 10,000 IMDb words as examples.
- What you can do after
- Apply feature selection by scoring features, filtering below a threshold, and comparing validation performance against a baseline.
Read how Deep Learning with R, Third Edition explains feature selection →
Outlier Detection in Python
Feature selection is the process of choosing a subset of input features that are most useful for a specified analysis or prediction task.
- How it is used
- Guides outlier-detection preprocessing by retaining features whose unusual values or combinations could matter.
- How deep it goes
- Explains relevance questions, dimensionality effects, and staff-record checks, but stops short of a general selection algorithm.
- Examples
- Uses staff IDs, start dates, duplicate checks, range checks, and rank comparisons as concrete selection tests.
- What you can do after
- Apply feature selection by removing irrelevant dimensions and testing whether rare values or combinations would be interesting.
Read how Outlier Detection in Python explains feature selection →
Evolutionary Deep Learning: Genetic algorithms and neural networks
Feature selection is the process of identifying and retaining input variables that contribute to a model's predictions while excluding less-relevant variables.
- How it is used
- Uses symbolic regression to select useful inputs while coevolving feature scalers and predictive equations.
- How deep it goes
- Demonstrates dataset preparation, toolbox construction, fitness evaluation, and evolution, but does not develop selection theory.
- Examples
- Uses the BH dataset, a 13-input equation, selected ARG variables, and executable DEAP-style evolution listings.
- What you can do after
- Apply coevolved genetic programming and genetic algorithms to discover equations that ignore less-relevant features.
Timeless Algorithms, The Foundational Ideas
Feature selection identifies a subset of predictors that contributes useful information while excluding predictors that add little value.
- How it is used
- Uses information reduction to identify predictors that contribute useful information about a target.
- How deep it goes
- Defines mutual information and illustrates predictor value with a numerical entropy calculation and medical example.
- Examples
- Illustrates predictor value with blood pressure and shoe size in disease prediction, plus a mutual-information calculation.
- What you can do after
- Evaluate predictors by comparing how much uncertainty each reduces about a target.
Read how Timeless Algorithms, The Foundational Ideas explains feature selection →
Build AI Drug Discovery Pipelines
Feature selection is the process of choosing a subset of input features for model construction or prediction.
- How it is used
- Uses feature selection to build interpretable QSAR models from descriptor-rich, small ligand datasets.
- How deep it goes
- Explains L1 and L2 penalties, cross-validated Lasso selection, descriptor-combination search, and performance filtering, but not selection theory beyond these workflows.
- Examples
- Demonstrates molecular descriptors, sparse fingerprint bits, a 48-ligand QSAR dataset, and a scikit-learn Lasso search listing.
- What you can do after
- Apply it via scikit-learn to select nonzero-coefficient features before fitting and evaluating a simpler regression model.
Read how Build AI Drug Discovery Pipelines explains feature selection →
Vibe Engineering
Feature selection is the process of choosing a subset of input variables for use by a predictive model.
- How it is used
- Prepares tabular inputs for price prediction by separating features, removing unusable columns, and controlling categorical feature size.
- How deep it goes
- Explains input-target separation, null filtering, outlier handling, and top-N categorical encoding, while deferring feature embeddings.
- Examples
- Uses Salt_Name, Manufacturer, Dosage_Unit, null columns, and the 99th-percentile price threshold as concrete preparation examples.
- What you can do after
- Apply it via X/y separation, null filtering, outlier filtering, and one-hot encoding before model training.
Also covered in
- Grokking Machine Learning, Second EditionIn early access (MEAP)
- Demand Forecasting Best Practices
- Ensemble Methods for Machine Learning
- Cyber Threat Hunting
- Data Without Labels
What to know first
- language modelA language model is a computational model that represents patterns in linguistic sequences to assign probabilities, predict, generate, or evaluate language.
- overfittingOverfitting is a modeling condition in which a model adapts too closely to particular data, including its noise or idiosyncrasies, and consequently generalizes poorly to new data.
- inputAn input is information, a value, or an action supplied to a system, program, or computation for processing or to influence its behavior.
- linear regressionLinear regression is a statistical method that models a numeric response as a linear function of one or more predictor variables with fitted parameters.
Related concepts
- feature engineeringpart of · Feature engineering is the creation, transformation, or selection of input features from data to provide useful representations for analysis or modeling.
- mutual informationuses · Mutual information is a measure of statistical dependence between random variables, quantifying the information about one variable provided by observing another.
- inputuses · An input is information, a value, or an action supplied to a system, program, or computation for processing or to influence its behavior.
- curse of dimensionalityaddresses · The curse of dimensionality is the set of statistical, geometric, and computational difficulties that arise as the number of dimensions grows, including data sparsity, less informative distances, and rapidly increasing resource requirements.
- L1 regularizationused by · L1 regularization is a technique that adds a penalty proportional to the sum of absolute model-parameter values, encouraging sparse parameter estimates.
- random forestused by · A random forest is an ensemble learning model that aggregates predictions from multiple decision trees trained with randomized data samples, feature subsets, or both.
Questions about feature selection
What is feature selection?
Feature selection is the process of choosing a subset of input features for use in a modeling, analysis, or prediction task.
Which Manning books explain feature selection?
Machine Learning System Design, Grokking Machine Learning, Statistics Every Programmer Needs, Machine Learning for Tabular Data: XGBoost, Deep Learning, and AI, Investing for Programmers, Deep Learning with R, Third Edition, Outlier Detection in Python, Evolutionary Deep Learning: Genetic algorithms and neural networks and 3 more.
What should I know before learning feature selection?
Manning books that teach feature selection usually introduce language model, overfitting, input and linear regression first.
What is feature selection part of?
Feature selection is part of feature engineering. Feature engineering is the creation, transformation, or selection of input features from data to provide useful representations for analysis or modeling.
These guides are generated from the text of Manning books by Manning's knowledge graph and link to the sections they draw on. The full books are available on liveBook and at manning.com.










