Occam's razor

As explained in Delivering Machine Learning Projects: From design to deployment

Occam's razor is the principle of preferring the simplest explanation or model when alternatives have otherwise comparable adequacy.

Occam’s razor prefers the simplest model when alternatives offer roughly equivalent performance. In machine learning, that usually means fewer parameters, smaller size, or greater compression relative to the training set. Simplicity is a selection criterion, not an excuse to ignore nonfunctional requirements: a less interpretable model may be justified when its performance advantage matters. section 8.3.4

Use the principle as a practical model-selection checkpoint:

• Compare candidate models with roughly equivalent effectiveness, then prefer the smaller or more interpretable option. section 8.3.4
• Record the tradeoff when a complex model is retained for materially better nonfunctional performance. section 8.3.4
• Use a complex model such as XGBoost as a baseline or denoiser when a simpler explainable model is required. section 7.3.4

Choosing a sensor-failure tree

A decision tree can split sensors by installation location or by age. Although both may divide the data effectively, the age split produces a pure branch when sensors older than 100 days always fail, so it supports a simpler tree with no further test on that branch. section 7.3.4

Figure 7.5 Splitting the data using the age attribute to yield a pure node on the left branch. No further tests are required on this tree.

Inspect the age-based split that yields a pure node and demonstrates how a simpler tree can stop branching earlier. section 7.3.4

How Delivering Machine Learning Projects: From design to deployment covers it

How deep it goes

Explains simplicity through model size, parameter count, compression, and decision-tree splits, but avoids formal mathematical treatment.

Examples

Uses sensor-failure trees, age-based and location-based splits, XGBoost baselines, and simpler explainable models.

What you can do after

Evaluate model candidates by comparing effectiveness, complexity, interpretability, and nonfunctional performance.

How it connects

Connects Occam’s razor to inductive bias, decision trees, model interpretability, XGBoost, boosting algorithms, and random forests.

liveBookmanning.com

Read Delivering Machine Learning Projects: From design to deployment on liveBook Get the book at manning.com

Also explained in

• 

  liveBookmanning.com
  Machine Learning for Tabular Data: XGBoost, Deep Learning, and AIOccam's razor principle favors the hypothesis that explains the evidence with the fewest assumptions among competing hypotheses.
• 

  liveBookmanning.com
  Sutskever's ListOccam's razor is the principle of preferring the simplest explanation that adequately accounts for the observations.

Compare how 3 books explain Occam's razor →

Related concepts

• decision treeconfigures · A decision tree is a branching structure that represents sequential tests or choices, with paths leading to outcomes, predictions, or actions.

This explainer was generated by Manning's knowledge graph from the text of Delivering Machine Learning Projects: From design to deployment. The section links point to the parts of the book it draws on.