liveBook · Manning

Scaled mean squared error

As explained in Machine Learning System Design

A mean squared error whose squared errors are scaled by the squared average actual over the forecast horizon.

Scaled mean squared error (sMSE) evaluates forecast accuracy by averaging squared errors after scaling them by the squared average actual demand over the forecast horizon. section 5.3.1

Use sMSE when forecast errors must be compared across time series with different demand scales:

  • Compare series because scaling normalizes the error measure across different time series. section 5.3.1
  • Expect minimization by the forecast distribution’s expectation. section 5.3.1
  • Treat sMSE as more robust than MSE to dramatic underforecasts, while remaining sensitive to large overforecasts. section 5.3.1
  • Check for an all-zero horizon, where the measure is undefined. section 5.3.1

How Machine Learning System Design covers it

How deep it goes
Defines the scaling, optimization target, comparability, and error sensitivities but does not provide a calculation example.
What it assumes
Assumes familiarity with forecast errors and mean squared error; does not teach those concepts from zero.
What you can do after
Evaluate forecasts across series while checking zero-demand validity and distinguishing underforecast from overforecast sensitivity.
How it connects
Connects mean squared error with intermittent demand, point forecasts, predictive distributions, and loss functions.
Cover of Machine Learning System Design
liveBookmanning.com
Read Machine Learning System Design on liveBook Get the book at manning.com

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Machine Learning System Design. The section links point to the parts of the book it draws on.