Mixture model

As explained in Time Series Forecasting Using Foundation Models

A mixture distribution is a probability distribution formed by weighting and combining multiple component distributions.

A mixture model forms one probability distribution by weighting and combining multiple component distributions. Its components can represent different shapes in the same predicted outcome. section 6.1.1

In Moirai, a mixture distribution produces the final predicted distribution of future time-series values. Combining distributions gives the forecaster a more flexible output for heterogeneous series with different domains, scales, and frequencies. section 6.1.1

Combining distribution shapes

A mixture distribution can combine a Student’s t-distribution with a log-normal distribution. Drawing points from both components produces a distribution that reflects both shapes. section 6.1.1

Figure 6.1 Architecture of Moirai. Here, the series is patched before entering the encoder-only transformer. Then, a mixture of distributions generates a flexible output distribution of future values.

Trace how patched time-series inputs pass through Moirai’s encoder-only transformer before a mixture of distributions generates future values. section 6.1.1

How Time Series Forecasting Using Foundation Models covers it

How deep it goes

Defines mixture distributions and places them at Moirai’s forecasting output, with a visual component-combination example.

Examples

Uses a Student’s t-distribution and log-normal distribution to illustrate how component shapes combine.

What you can do after

Evaluate how a forecasting architecture can represent future values with a flexible combined probability distribution.

How it connects

Connects mixture distributions with time-series forecasting, transformer architecture, patching, Student’s t-distribution, and log-normal distribution.

liveBookmanning.com

Read Time Series Forecasting Using Foundation Models on liveBook Get the book at manning.com

Related concepts

• Gaussian mixture modelincludes · A Gaussian mixture model is a probabilistic model representing a distribution as a weighted combination of multiple Gaussian component distributions.
• kernel density estimatealternative to · A kernel density estimate is a nonparametric probability-density estimate formed by combining localized kernel functions centered on observed data points.
• Moiraiused by · Moirai is a foundation model designed for time-series forecasting across varied datasets.
• mixture density networkincludes · A mixture density network is a neural network that predicts the parameters of a conditional mixture probability distribution, optionally at each sequence step.

This explainer was generated by Manning's knowledge graph from the text of Time Series Forecasting Using Foundation Models. The section links point to the parts of the book it draws on.