liveBook · Manning

Feature importance score

A feature importance score is a numerical estimate of an input feature’s relative contribution or influence on a model’s predictions or performance.

How 10 Manning books explain it

Rearchitecting LLMs

An importance score numerically estimates how important a model component is to its information or performance.

How it is used
Ranks blocks, neuron pairs, and attention modules as candidates for structured pruning.
How deep it goes
Works through hook-based block scoring, static and hybrid neuron metrics, safety checks, and score-based selection.
Examples
Compares WinoGrande and WikiText block selections and calculates neuron scores from projection weight ranges.
What you can do after
Apply it using PyTorch hooks and top-k selection to identify pruning candidates, then evaluate the resulting models.

Read how Rearchitecting LLMs explains feature importance score →

Machine Learning System Design

A quantitative attribution describing how much an input feature contributes to a model's output or predictive performance.

How it is used
Supports model transparency, deployment verification, and feature-selection decisions.
How deep it goes
Compares model-specific and model-agnostic methods and outlines deep-learning techniques without deriving their algorithms.
Examples
Illustrates importance through taxi pricing, retail demand forecasting, and explanations for search-ranking complaints.
What you can do after
Evaluate and understand feature contributions using model-specific and model-agnostic approaches to guide feature selection and verification.

Read how Machine Learning System Design explains feature importance score →

Build AI Drug Discovery Pipelines

Feature importance is a quantitative estimate of how much an input feature contributes to a predictive model's decisions.

How it is used
Uses feature importance to interpret molecular classifiers for CYP 3A4 inhibition and RNA-binding activity.
How deep it goes
Explains weighted impurity reduction and three XGBoost measures, then demonstrates ranking and local sensitivity analysis.
Examples
Ranks Morgan fingerprint bits for CYP 3A4 inhibition and examines descriptors for RNA-binding predictions.
What you can do after
Apply it via scikit-learn and XGBoost to rank features, and use sensitivity analysis to inspect individual predictions.

Read how Build AI Drug Discovery Pipelines explains feature importance score →

Ensemble Methods for Machine Learning

Feature importance is a score representing the contribution or relevance of an input feature to a model's predictions.

How it is used
Supports feature selection and communicates which inputs matter most to a model’s overall decisions.
How deep it goes
Derives weighted entropy-decrease scores for trees, then contrasts ensemble aggregation with hold-out permutation testing.
Examples
Uses two-dimensional data, Iris tree splits, breast cancer diagnosis, and XGBoost ensemble comparisons.
What you can do after
Evaluate/understand feature rankings from tree ensembles and test feature reliance through hold-out permutation scores.

Read how Ensemble Methods for Machine Learning explains feature importance score →

Timeless Algorithms, The Foundational Ideas

Variable importance is a quantitative assessment of how much a predictor contributes to a model's predictions or reduction of uncertainty.

How it is used
Uses importance scores to recover an interpretable view of how decision trees and random forests use predictors.
How deep it goes
Explains permutation and impurity-based importance, including out-of-bag permutation steps and limitations of interpreting scores.
Examples
Illustrates redundant predictors with Celsius and Fahrenheit measurements and tree importance with income and credit history in loan approval.
What you can do after
Evaluate and understand importance scores by distinguishing predictive dependence from split improvement and recognizing correlation and bias.

Read how Timeless Algorithms, The Foundational Ideas explains feature importance score →

Outlier Detection in Python

Feature importance is a quantitative indication of how much an input feature contributes to a model’s prediction, score, or decision.

How it is used
Identifies features relevant to outlier scores when a detector’s scores or labels alone do not explain flagged records.
How deep it goes
Distinguishes global from local importance and demonstrates plotting workflows, while emphasizing that rankings need not fully explain scores.
Examples
Uses abalone outlier scores, SHAP plots, PyOD IForest rankings, and BHAD’s local and global explanations.
What you can do after
Apply it via SHAP or supported detectors to rank relevant features, then assess whether those rankings adequately explain an outlier score.

Read how Outlier Detection in Python explains feature importance score →

Statistics Every Programmer Needs

Feature importance is a measure of how strongly each input feature contributes to a model's predictions, often based on its effect on split quality.

How it is used
Interprets which input variables most affect predictions in decision trees and random forests.
How deep it goes
Defines impurity-based scoring, explains relative values, and demonstrates forest-wide visualization without discussing alternative importance methods.
Examples
Uses fourth-down conversion prediction, TO_GO, PLAY_TYPE, importance scores, and a sorted bar chart.
What you can do after
Apply it via RandomForestClassifier and Matplotlib to rank and visualize influential features.

Read how Statistics Every Programmer Needs explains feature importance score →

Build a DeepSeek Model (From Scratch)

Expert importance is the aggregate selection weight assigned to an expert across the tokens in a batch.

How it is used
Quantifies each expert’s aggregate routing weight to assess balance and supply inputs to auxiliary-loss and router-probability calculations.
How deep it goes
Defines the batch column sum, works through numerical scores, and connects them to coefficient of variation and mean router probability.
Examples
Calculates scores of 1.4, 1.0, and 1.6, then contrasts equal scores from one high-probability token and four low-probability tokens.
What you can do after
Evaluate expert balance by aggregate routing weight while recognizing that equal importance scores do not guarantee equal token workloads.

Read how Build a DeepSeek Model (From Scratch) explains feature importance score →

Machine Learning for Tabular Data: XGBoost, Deep Learning, and AI

Feature importance is a measure of how strongly an input feature contributes to a model's predictions or performance.

How it is used
Uses feature importance to assess model transparency and communicate which inputs most affect predictions.
How deep it goes
Defines importance and demonstrates XGBoost gain scores, but stops short of detailing alternative importance methods or Keras implementation.
Examples
Uses a house-market dataset and an Airbnb NYC price-classification model, including gain values, feature rankings, and a bar chart.
What you can do after
Apply it via XGBoost’s built-in API to rank features and visualize their relative importance.

Read how Machine Learning for Tabular Data: XGBoost, Deep Learning, and AI explains feature importance score →

Financial AI in Practice

Feature importance is a measure of the contribution or influence of an input feature on a model’s predictions.

How it is used
Supports portfolio-level model review for risk managers and regulators, and helps diagnose changes in financial model behavior.
How deep it goes
Defines split-count, gain, permutation, and mean-absolute-SHAP approaches, including key biases, limitations, and costs.
Examples
Uses income and age split counts, shuffled-feature error, WoE-coded credit inputs, and feature importance for market-model diagnosis.
What you can do after
Evaluate which features drive predictions while accounting for the chosen importance method’s limitations.

Read how Financial AI in Practice explains feature importance score →

Also covered in

What to know first

Related concepts

Questions about feature importance score

What is feature importance score?

A feature importance score is a numerical estimate of an input feature’s relative contribution or influence on a model’s predictions or performance.

Which Manning books explain feature importance score?

Rearchitecting LLMs, Machine Learning System Design, Build AI Drug Discovery Pipelines, Ensemble Methods for Machine Learning, Timeless Algorithms, The Foundational Ideas, Outlier Detection in Python, Statistics Every Programmer Needs, Build a DeepSeek Model (From Scratch) and 2 more.

What should I know before learning feature importance score?

Manning books that teach feature importance score usually introduce random forest, gradient boosting, accuracy and overfitting first.

What is the difference between feature importance score and causation?

A feature importance score is a numerical estimate of an input feature’s relative contribution or influence on a model’s predictions or performance. Causation is a relationship in which a change in one event, factor, or condition produces or influences a change in another.

What kinds of feature importance score are there?

Kinds of feature importance score include permutation feature importance.

These guides are generated from the text of Manning books by Manning's knowledge graph and link to the sections they draw on. The full books are available on liveBook and at manning.com.