Feature selection

As explained in Deep Learning with R, Third Edition

Feature selection is the process of choosing a subset of input features that are useful for a machine-learning task.

Feature selection chooses a subset of input features that are useful for a machine-learning task. Removing irrelevant or noisy features can reduce spurious correlations and overfitting without changing the information originally present in the useful data. section 5.1.1

Filtering noisy MNIST features

Create normalized MNIST images, append either random-noise channels or all-zero channels, and train the same dense classifier on each version. Compare validation accuracy across ten epochs. The noise-channel model performs worse because the added features encourage spurious correlations; the zero channels do not add information. In R/Keras, the workflow uses array_reshape(), cbind(), keras_model_sequential(), fit(), and validation metrics. section 5.1.1

A practical feature-selection workflow is:

• Compute a usefulness score for every available feature, such as mutual information between the feature and labels. section 5.1.1
• Keep features above a chosen threshold, then train the model on the reduced dataset. section 5.1.1
• Use selection alongside feature engineering when building a small model intended to beat a simple baseline. section 6.2.3

Figure 5.6 Effect of noise channels on validation accuracy

Compare validation-accuracy curves to see how adding noisy features affects generalization. section 5.1.1

How Deep Learning with R, Third Edition covers it

How deep it goes

Explains usefulness scoring and thresholding, then demonstrates noisy-feature effects experimentally without detailing selection algorithms.

Examples

Uses noisy MNIST channels, all-zero channels, validation-accuracy curves, and the top 10,000 IMDb words as examples.

What it assumes

Assumes input features, labels, model training, and validation accuracy, while defining why irrelevant features can harm learning.

What you can do after

Apply feature selection by scoring features, filtering below a threshold, and comparing validation performance against a baseline.

How it connects

Connects feature selection with overfitting, noisy features, mutual information, feature engineering, and baseline models.

liveBookmanning.com

Read Deep Learning with R, Third Edition on liveBook Get the book at manning.com

Also explained in

• 

  liveBookmanning.com
  Machine Learning System DesignFeature selection is the process of choosing a subset of available input features for a machine learning model.
• 

  liveBookmanning.com
  Grokking Machine LearningFeature selection is a technique that chooses relevant input features and removes unnecessary or harmful ones before model training.
• 

  liveBookmanning.com
  Statistics Every Programmer NeedsFeature selection is the process of choosing a subset of input variables for use in a predictive model.
• 

  liveBookmanning.com
  Machine Learning for Tabular Data: XGBoost, Deep Learning, and AIFeature sampling selects a subset of input features as candidates for modeling or a split.
• 

  liveBookmanning.com
  Investing for ProgrammersFeature selection is the process of choosing input variables that are most relevant or useful for a predictive model.
• 

  liveBookmanning.com
  Outlier Detection in PythonFeature selection is the process of choosing a subset of input features that are most useful for a specified analysis or prediction task.
• 

  liveBookmanning.com
  Evolutionary Deep Learning: Genetic algorithms and neural networksFeature selection is the process of identifying and retaining input variables that contribute to a model's predictions while excluding less-relevant variables.

Compare how 8 books explain feature selection →

Related concepts

• feature engineeringpart of · Feature engineering is the creation, transformation, or selection of input features from data to provide useful representations for analysis or modeling.
• inputuses · An input is a value, information, or signal supplied to a system, computation, or interface for processing or to influence its behavior.
• curse of dimensionalityaddresses · The curse of dimensionality is the set of statistical, geometric, and computational difficulties that tend to increase as the number of dimensions grows.
• L1 regularizationused by · L1 regularization is a technique that adds a penalty proportional to the sum of absolute model-parameter values, encouraging sparse parameter estimates.
• random forestused by · A random forest is an ensemble model that combines predictions from multiple decision trees built with randomized data samples, feature selections, or both.

This explainer was generated by Manning's knowledge graph from the text of Deep Learning with R, Third Edition. The section links point to the parts of the book it draws on.