Importance score

As explained in Machine Learning System Design

A quantitative attribution describing how much an input feature contributes to a model's output or predictive performance.

Suppose a demand forecast uses promotions, past sales, and weather. An importance score helps answer which inputs contribute to its prediction or performance. That answer can guide feature selection, make a prediction easier to explain, and help decide whether a newly trained model is ready to deploy. A low-scoring feature may be a candidate for removal, but the decision should also consider the feature’s cost and the model’s overall performance. section 11.5.1 section 11.2 section 11.1.1

Choose an importance method that fits the model and the question:

• For some classical models, inspect coefficients or decision rules. An embedded feature selector can use those scores: recursive feature elimination repeatedly removes the lowest-ranked features. section 11.2.1 section 11.3.3
• To examine a model without relying only on its internal structure, alter or shuffle an input and observe the effect. SHAP, LIME, and shuffle importance are options for comparing contributions. section 11.2.1 section 11.5.1
• For images or text, first decide what counts as a feature: a pixel, token, or higher-level pattern. SHAP can attribute outcomes to tokens or pixels; a saliency map highlights image regions whose changes affect an output. These methods do not provide a universal explanation of deep learning. section 11.2.3 section 11.2.1

Use scores to make a feature decision

For a store-and-product demand model, candidate inputs include a promotion calendar, recent sales, and predicted weather. Measure their contributions with methods such as SHAP, LIME, shuffle importance, model coefficients, or gradient-boosting split counts. Drop a feature that contributes little; during early exploration, recursive feature elimination can help establish a basic set. Keep range, correlation, contribution, and computation-time tests in the training pipeline rather than treating an importance score as the only check. section 11.5.1

Scores can serve two different decisions. An overall view can show where to invest in feature engineering. An explanation for one prediction can help investigate a complaint—for example, why a search result ranked as it did. If a model relies heavily on one source of information, examine whether that reliance creates overfitting or opportunities to exploit its weaknesses. Importance is evidence for investigation, not a substitute for checking the system’s behavior. section 11.5.2 section 11.1.1

Figure 11.2 An example of a taxi aggregator app’s UI that clarifies why its dynamic pricing algorithm chooses this particular price in this area and during this time of the day

The taxi-pricing interface illustrates the user-facing purpose of an explanation: connecting a particular price to conditions affecting that ride. section 11.2

How Machine Learning System Design covers it

How deep it goes

Surveys model-specific, model-agnostic, and deep-learning techniques with practical selection examples; stops short of a universal deep-learning explanation method.

Examples

Uses taxi-price explanations, retail-demand features, and search-ranking complaints to show distinct reasons for examining contributions.

What it assumes

Assumes familiarity with model inputs and predictions while introducing why contribution analysis matters for feature decisions.

What you can do after

Evaluate/understand which inputs merit retention, further engineering, or investigation when a prediction needs explaining.

How it connects

Connects feature engineering and feature selection with interpretability, explainability, recursive feature elimination, and model verification.

liveBookmanning.com

Read Machine Learning System Design on liveBook Get the book at manning.com

Also explained in

• 

  liveBookmanning.com
  Ensemble Methods for Machine LearningFeature importance is a score representing the contribution or relevance of an input feature to a model's predictions.
• 

  liveBookmanning.com
  Outlier Detection in PythonFeature importance is a quantitative indication of how much an input feature contributes to a model’s prediction, score, or decision.
• 

  liveBookmanning.com
  Statistics Every Programmer NeedsFeature importance is a measure of how strongly each input feature contributes to a model's predictions, often based on its effect on split quality.
• 

  liveBookmanning.com
  Machine Learning for Tabular Data: XGBoost, Deep Learning, and AIFeature importance is a measure of how strongly an input feature contributes to a model's predictions or performance.
• 

  liveBookmanning.com
  Financial AI in PracticeFeature importance is a measure of the contribution or influence of an input feature on a model’s predictions.

Compare how 6 books explain importance score →

Related concepts

• random forestused by · A random forest is an ensemble model that combines predictions from multiple decision trees built with randomized data samples, feature selections, or both.
• permutation feature importanceincludes · Permutation feature importance measures a feature’s importance to a model by randomly permuting its values and assessing the resulting change in predictive performance.
• tree-based methoduses · A tree-based method is a predictive or decision-making technique that represents its rules or decision structure using one or more trees.
• SHAPmeasured by · SHAP is a feature-attribution method that explains model outputs by assigning input features contributions based on Shapley values.

This explainer was generated by Manning's knowledge graph from the text of Machine Learning System Design. The section links point to the parts of the book it draws on.