liveBook · Manning

Baseline

As explained in Delivering Machine Learning Projects: From design to deployment

A baseline performance measure is an initial quantitative assessment against which later model performance can be compared.

A baseline is an initial performance measure used to judge later models. A quickly implemented baseline model provides a bottom line for comparison and early feedback on whether a modelling problem looks promising. section 6.0 section 6.6.2

Compare performance both with a simple alternative and with the business requirement. A sophisticated model should improve enough to justify its cost, not merely perform slightly better than a simple system. section 6.3

Establish a baseline early:

  • Define and implement a simple model, such as a majority-class predictor, decision tree, or low-dimensional perceptron. A small data sample can support quick iteration. section 6.3
  • Measure its performance and record the assumptions and limitations. Early models may overfit or fail to capture the data’s complexity; they indicate challenges rather than solve the problem. section 6.3 section 6.6.2
  • Compare later models against the baseline and the improvement needed for the project’s required return on investment. section 6.3

Accuracy needs context

A classifier with 99% accuracy may be no better than always predicting the majority class if that class accounts for 99% of the test set. The majority-class prediction provides a simple comparison point. section 6.3

Demand and churn

For an inventory and customer-churn project, a quick regression model provided baseline measures using limited data. Its predictions correlated with observed fluctuations, but substantial assumptions meant the problem was far from solved. section 6.6.2

How Delivering Machine Learning Projects: From design to deployment covers it

How deep it goes
Defines a baseline ticket and simple modelling choices, while deferring detailed performance estimation methods.
Examples
Uses majority-class accuracy and a quick regression for inventory demand and customer churn to illustrate baseline measures.
What you can do after
Evaluate whether later models improve on simple alternatives enough to meet business needs and justify their cost.
How it connects
Connects baselines to exploratory data analysis, classification, regression, overfitting, and return on investment.

Also explained in

Compare how 27 books explain baseline →

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Delivering Machine Learning Projects: From design to deployment. The section links point to the parts of the book it draws on.