liveBook · Manning

Conformity score

As explained in Time Series Forecasting Using Foundation Models

A conformity score is a numerical measure of how atypical an observation is relative to a prediction, here computed as the absolute forecast residual.

A conformity score measures how atypical an observation is relative to a prediction. For time-series forecasting, it is the absolute residual: the absolute difference between the actual and predicted value. section 3.2.2

Historical forecast errors supply conformity scores for each forecast step. Adding and subtracting those scores from a point forecast creates possible-value ranges; selected quantiles then form a prediction interval. section 3.2.2

Forecast range

For a forecast of 6, scores of 2, 3, 1, 1.5, 2.5, and 0.8 are added to and subtracted from the forecast. An 80% interval uses the 10th and 90th percentiles of the resulting values, giving bounds of 3.55 and 8.45. section 3.2.2

How Time Series Forecasting Using Foundation Models covers it

How deep it goes
Defines absolute residuals and works through cross-validation windows, score aggregation, ranges, and quantiles.
Examples
Uses two cross-validation windows, a three-step horizon, forecast 6, six residual scores, and an 80% interval.
What it assumes
Assumes predicted and actual values from cross-validation, while defining conformity scores from their differences.
What you can do after
Build conformal prediction intervals by calculating absolute residuals and applying their quantiles to point forecasts.
How it connects
Connects conformity scores with conformal prediction, cross-validation, prediction intervals, and time-series forecasting.
Cover of Time Series Forecasting Using Foundation Models
liveBookmanning.com
Read Time Series Forecasting Using Foundation Models on liveBook Get the book at manning.com

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Time Series Forecasting Using Foundation Models. The section links point to the parts of the book it draws on.