Autoregressive model
An autoregressive model represents or predicts each value in a sequence based on preceding values in that sequence.
Build a Text-to-Image Generator (from Scratch)
A process that produces an ordered output sequence one element at a time, conditioning each element on previously generated elements and available conditioning information.
- How it is used
- Generates translations, image captions, and image tokens by repeatedly predicting the next token from prior output and encoded input.
- How deep it goes
- Traces start tokens, iterative prediction, stopping, and a 256-step decoder loop, including state updates and sampling settings.
- Examples
- Uses German-to-English translation, a skier caption, and intermediate images from a text-to-image prompt.
- What you can do after
- Apply it via a transformer decoder to generate output token by token and inspect saved stages of image generation.
Read how Build a Text-to-Image Generator (from Scratch) explains autoregressive model →
Statistics Every Programmer Needs
An autoregressive component models the current value of a time series as a linear function of specified previous values.
- How it is used
- Supports time-series forecasting by relating current observations to their own historical values.
- How deep it goes
- Explains lagged relationships, stationarity preparation, ACF/PACF selection, and AR(1), but defers full ARIMA estimation details.
- Examples
- Uses Apple daily closing prices, first-order differences, ACF/PACF plots, and an actual-versus-forecast comparison.
- What you can do after
- Builds an ARIMA forecasting workflow by preparing data, selecting an AR order, fitting the model, and forecasting held-out observations.
Read how Statistics Every Programmer Needs explains autoregressive model →
The Art of AI Product Development
An autoregressive language model generates each token by predicting it from the preceding context.
- How it is used
- Explains the basic language-modeling objective behind sequential text generation.
- How deep it goes
- Defines next-token prediction and its directional limitation, but stops short of implementation details.
- Examples
- Uses sentence completion, Mistral, ChatGPT, and context illustrations to contrast raw prediction with conversational behavior.
- What you can do after
- Evaluate how next-token objectives support generation while limiting context and conversational usefulness.
Read how The Art of AI Product Development explains autoregressive model →
Build a Robo-Advisor with Python (From Scratch)
A time-series model in which the current value depends linearly on one or more previous values and a random error term.
- How it is used
- Models persistent, mean-reverting inflation within retirement Monte Carlo simulations.
- How deep it goes
- Estimates one-lag parameters with regression and implements simulated paths, stopping at the AR(1) case.
- Examples
- Uses CPI-U data, lagged-inflation regression, residual volatility, and simulated thirty-year wealth paths.
- What you can do after
- Build an AR(1) inflation component and insert it into a retirement Monte Carlo simulation.
Read how Build a Robo-Advisor with Python (From Scratch) explains autoregressive model →
Introduction to Generative AI, Second Edition
Autoregressive generation produces a sequence by predicting each successive element from previously generated elements and available context.
- How it is used
- Generates coherent sequences by predicting successive tokens from preceding context.
- How deep it goes
- Explains the prediction task, training intuition, and bidirectional contrast without descending into implementation details.
- Examples
- Uses a honeymoon-completion prompt and identifies GPT-family models and PaLM as autoregressive examples.
- What you can do after
- Evaluate/understand why generative models use left-context next-token prediction.
Read how Introduction to Generative AI, Second Edition explains autoregressive model →
Related concepts
- bidirectional language modelcontrasts with · A bidirectional language model represents or predicts tokens using contextual information from both preceding and following tokens in a sequence.
- GPT-4includes · GPT-4 is a generative large language model developed by OpenAI for understanding and generating text and other supported inputs and outputs.
- forward passuses · A forward pass is a calculation that propagates inputs through an ordered computational or dependency structure to produce outputs or derived values.
- decodeincludes · Decoding is the process of transforming an encoded or internal representation into an interpretable or target-form output.
- arimapart of · ARIMA is a statistical time-series model that combines autoregressive terms, differencing, and moving-average terms to represent temporal dependence and generate forecasts.
These guides are generated from the text of Manning books by Manning's knowledge graph and link to the sections they draw on. The full books are available on liveBook and at manning.com.