Natural language generation produces human-readable sequences by predicting output tokens from context. Earlier sequence-to-sequence systems encoded an input sentence into a fixed-length representation, then generated the target one word at a time; decoder-only transformers instead reuse their own preceding outputs. section 2.1 section 3.2
Decoder-only generation depends on three linked mechanisms:
- Causal attention prevents a prediction from attending to future tokens, so each output uses only the preceding context. section 3.2
- Autoregressive decoding predicts one token, feeds it back into the input stream, and repeats the process. section 3.2
- Next-token prediction trains the model to continue sequences within its context window. section 3.2
- Decoding and sampling choices shape output quality: greedy and beam search are deterministic, while top-k, top-p, and temperature sampling are probabilistic. section 4.0
From text to generated tokens
An input encoder can convert raw Unicode text into byte-pair-encoded subwords and vocabulary IDs. A decoder-only model processes those IDs through masked self-attention and transformer blocks, predicts the next token, and appends it before making the following prediction.
