liveBook · Manning

Decoder-only transformer

As explained in Fight Fraud with Machine Learning

A decoder-only architecture is a generative sequence-modeling design containing decoder blocks without a separate encoder component.

A decoder-only transformer generates text autoregressively: each decoder block attends only to tokens generated so far, then predicts one next token. That token is appended to the running sequence and fed back in to continue generation until an end token appears. section 5.2.1

GPT uses this design for prompt-based text production. Unlike the original encoder-decoder transformer used for translation, it has no separate encoder and no encoder-decoder attention layer. Training on large text data teaches the model to predict the most likely next token from its preceding sequence. section 5.2.1

Tokens need not be whole words. Byte-pair encoding represents text as subwords, balancing the large vocabulary of word tokens against the long sequences created by character-level tokens. section 5.2.1

Figure 5.9 The Generative Pretrained Transformer (GPT) model architecture, and how it auto-regressively performs text generation.

Shows the GPT decoder-only architecture and its autoregressive text-generation flow. section 5.2.1

How Fight Fraud with Machine Learning covers it

How deep it goes
Explains causal attention, iterative next-token prediction, removed encoder-decoder attention, and byte-pair encoding without decoder-block implementation details.
Examples
Illustrates next-word prediction, greedy search, English-to-French translation, and GPT autoregressive generation diagrams.
What it assumes
Introduces prompt-based next-token generation and tokenization from concrete text examples; no prior background is stated.
What you can do after
Evaluate/understand why GPT generates sequentially and differs structurally from an encoder-decoder translation model.
How it connects
Connects decoder-only transformers to autoregressive models, GPT, prompts, encoder-decoder transformers, attention, and byte-pair encoding.

Also explained in

Compare how 13 books explain decoder-only transformer →

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Fight Fraud with Machine Learning. The section links point to the parts of the book it draws on.