liveBook · Manning

Autoregressive generation

Autoregressive generation produces a sequence one element at a time, conditioning each new element on the elements already available.

How 3 Manning books explain it

CUDA for LLMs

Autoregressive generation produces a sequence one token at a time, conditioning each new token on preceding tokens.

How it is used
Uses autoregressive generation as the sequential decoding workload for transformer inference optimization.
How deep it goes
Traces sampling, cache reuse, decode tensor shapes, and CUDA-backed inference, including the memory–computation tradeoff.
Examples
Uses a prompt continuation and a 100-token cache accounting example with a 24-layer transformer configuration.
What you can do after
Evaluate and understand redundant decoding work and the memory tradeoff of KV caching.

Read how CUDA for LLMs explains autoregressive generation →

Timeless Algorithms, The Foundational Ideas

Autoregressive generation is the process of producing a sequence by predicting one token at a time, conditioning each prediction on tokens already available.

How it is used
Explains how successive next-token predictions build a multistep response.
How deep it goes
Works through causal masking and a multistep example, using simplified operations rather than reproducing a complete trained transformer.
Examples
Uses an overtaking-trains calculation and a four-token attention mask to illustrate chained predictions and available context.
What you can do after
Evaluate and understand how sequential dependencies affect generation, latency, and reliability.

Read how Timeless Algorithms, The Foundational Ideas explains autoregressive generation →

Rearchitecting LLMs

Autoregressive generation produces a sequence one token at a time, conditioning each new token on previously generated tokens.

How it is used
Explains how depth pruning affects inference memory and why static benchmarks may miss generation degradation.
How deep it goes
Defines token-by-token dependence and explains per-layer key/value storage as the cache grows with sequence length.
Examples
Contrasts one-token benchmark answers with short-prompt coherence and long-form factual inconsistencies after layer removal.
What you can do after
Evaluate whether pruned models retain coherent text generation instead of relying only on static benchmarks.

Read how Rearchitecting LLMs explains autoregressive generation →

Also covered in

What to know first

Related concepts

Questions about autoregressive generation

What is autoregressive generation?

Autoregressive generation produces a sequence one element at a time, conditioning each new element on the elements already available.

Which Manning books explain autoregressive generation?

CUDA for LLMs, Timeless Algorithms, The Foundational Ideas and Rearchitecting LLMs.

What should I know before learning autoregressive generation?

Manning books that teach autoregressive generation usually introduce attention, attention head, inner product and feedforward network first.

What are the parts of autoregressive generation?

Autoregressive generation includes decode phase and prefill.

These guides are generated from the text of Manning books by Manning's knowledge graph and link to the sections they draw on. The full books are available on liveBook and at manning.com.