liveBook · Manning

Decoder-only transformer

As explained in LLM Evaluation and Alignment, The Foundational Ideas

A decoder model is an autoregressive sequence model that predicts each token from preceding tokens and is typically used for text generation.

A decoder-only transformer is an autoregressive sequence model: it reads tokens left to right and predicts each next token from preceding context. This makes it suited to text generation, where future tokens do not yet exist. section 3.2.2

Distinguish decoder-only transformers from encoder-only transformers by their available context:

  • Decoder-only models predict the next token using only preceding tokens. section 3.2.2
  • Encoder-only models read a complete input bidirectionally, using future context to build richer representations for comparison and understanding. section 3.2.2

Context changes by architecture

When processing “attorney” in “The attorney filed the brief,” an encoder can use “filed” and “brief.” A decoder processes “attorney” without knowing what follows, because it is predicting the continuation.

How LLM Evaluation and Alignment, The Foundational Ideas covers it

How deep it goes
Contrasts left-to-right decoding with bidirectional encoding but does not implement decoder inference.
Examples
Uses GPT and Claude as generation-oriented examples and contrasts them with BERT-style encoders.
What you can do after
Evaluate whether bidirectional encoders or autoregressive decoders fit a text-comparison task.
How it connects
Connects decoder-only transformers to encoder-only transformers, autoregressive modeling, and text generation.

Also explained in

Compare how 13 books explain decoder-only transformer →

Related concepts

This explainer was generated by Manning's knowledge graph from the text of LLM Evaluation and Alignment, The Foundational Ideas. The section links point to the parts of the book it draws on.