Patching

As explained in Time Series Forecasting Using Foundation Models

Patching is the technique of grouping adjacent time-series points into fixed-length segments that are processed as tokens.

Patching groups adjacent time-series points into fixed-length, usually nonoverlapping segments; each segment becomes one token for an embedding layer or transformer. Grouping preserves local relationships while reducing the token count, making forecasting models lighter and faster. section 6.1.1 section 7.1.1

Apply patching by making three architectural choices:

• Choose a patch length suited to the series frequency; Moirai uses length 8 for yearly and quarterly data, while minute-level data uses 32, 64, or 128. section 6.1.1
• Replace individual time-step tokens with grouped tokens so each token carries information from multiple neighboring observations. section 6.1.1
• Balance specialization against simplicity: frequency-specific lengths require multiple embedding layers, whereas Moirai-MoE uses one fixed length and one input projection. section 6.2.1

Forecasting 256 future steps with TimesFM

With 32-step input patches and 128-step output patches, TimesFM needs two autoregressive steps to forecast 256 future steps; matching the output to the input length would require eight. section 7.1.1

Figure 6.2 The process of patching. Typically, each data point (circle) is a token that’s sent to the embedding layer. With patching, we group data points, and each patch becomes a token that is sent to the embedding layer.

Compare individual-point tokenization with grouped patches: each patch becomes one token sent to the embedding layer. section 6.1.1

How Time Series Forecasting Using Foundation Models covers it

How deep it goes

Explains token formation, frequency-based lengths, architectural trade-offs, and autoregressive effects, but does not detail patch extraction code.

Examples

Uses two-point patches, frequency-specific Moirai lengths, TimesFM output patches, and Time-LLM patch representations such as “down steady.”

What it assumes

Assumes time-series observations and transformer embeddings, while introducing grouped tokens and their forecasting rationale.

What you can do after

Apply it via a forecasting architecture to reduce token counts, retain local information, or shorten autoregressive prediction sequences.

How it connects

Connects time-series forecasting with tokenization, embeddings, transformers, autoregressive forecasting, and patch reprogramming.

liveBookmanning.com

Read Time Series Forecasting Using Foundation Models on liveBook Get the book at manning.com

Related concepts

• Moiraiused by · Moirai is a foundation model designed for time-series forecasting across varied datasets.
• Moirai-MoEused by · Moirai-MoE is a Moirai-based universal time-series forecasting model that uses fixed-length, nonoverlapping patches across frequencies.
• TimesFMuses · TimesFM is a foundation model designed for deterministic time-series forecasting.
• transformer architectureuses · A transformer architecture is a neural-network architecture that processes sequences or other structured inputs using attention mechanisms and related transformations.

This explainer was generated by Manning's knowledge graph from the text of Time Series Forecasting Using Foundation Models. The section links point to the parts of the book it draws on.