liveBook · Manning

Decoder-only transformer

As explained in Building Specialized AI Systems

A decoder-only model generates sequences with decoder layers and causal attention over preceding tokens.

A decoder-only transformer generates text one token at a time. Causal masking restricts each token to attending to preceding tokens, making the architecture suited to generation and causal language modeling. section 3.1.1

Use the architecture as a practical choice when:

  • Generation is the primary task rather than full-context representation or sequence transduction. section 3.1.1
  • You need to weigh long-context cost: full self-attention has quadratic complexity in sequence length. section 3.1.1
  • You plan inexpensive adaptation through parameter-efficient methods such as LoRA or adapters. section 3.1.1
Figure 3.2 Transformer model architecture. It consists of an encoder stack (left) and a decoder stack (right). The encoder processes the input sequence using multi-head self-attention (MSA) and multi-layer perceptron (MLP) blocks with residual connections and layer normalization (Add & LN). The decoder processes the target sequence using masked multi-head self-attention (MMSA) and multi-head cross attention (MCA) to integrate encoder features, then projects the output to probabilities through a linear layer and softmax. Source: [9]

Compare the decoder stack with the encoder stack and inspect masked self-attention, cross-attention, normalization, and output projection. section 3.1.1

How Building Specialized AI Systems covers it

How deep it goes
Explains causal masking, architectural alternatives, pretraining objectives, and quadratic attention costs, then points to mitigation techniques.
Examples
Names GPT-4, Llama, and Mistral as modern large language models using decoder-only architectures.
What it assumes
Starts from encoder and decoder roles before distinguishing representation, generation, and sequence-transduction layouts.
What you can do after
Evaluate whether a generation workload warrants decoder-only architecture and identify its context-length and adaptation trade-offs.
How it connects
Connects decoder-only transformers to causal language modeling, self-attention, encoder-only models, encoder-decoder architectures, LoRA, and adapters.

Also explained in

Compare how 13 books explain decoder-only transformer →

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Building Specialized AI Systems. The section links point to the parts of the book it draws on.