Decoder-only transformer

As explained in Effective Conversational AI

A decoder-only architecture is a language-model architecture trained to predict successive tokens and generate text from preceding context.

A decoder-only transformer generates text by predicting the next token from the preceding context. Unlike a classifier limited to predefined intents, it can produce specific, personalized responses and can be instructed to perform tasks such as classification, question answering, and summarization. section 4.1.4

From intent labels to useful responses

A traditional model might map a user message to one predefined intent. A decoder-only model instead continues the input with a likely text completion, allowing a conversational system to address the request directly and with context. section 4.1.4

Figure 4.4 Traditional classification models use supervised learning to predict one of several predefined classifications. They look for the intent, or meaning, of a user input. Generative models use decoding transformers to create a text completion. They predict the next sequence of tokens (loosely, words or characters) that are most likely to occur after the user input.

Compare supervised classification, which selects among predefined categories, with generative decoding, which predicts the next sequence of tokens. section 4.1.4

How Effective Conversational AI covers it

How deep it goes

Defines next-token prediction, training style, and task range, but does not descend into transformer internals.

Examples

Uses conversational responses, classification, question answering, summarization, and next-token text completion as examples.

What it assumes

Requires understanding the user domain and likely problem range, while treating next-token generation as the core model behavior.

What you can do after

Evaluate whether generative modeling fits conversational tasks requiring responses beyond predefined intent categories.

How it connects

Contrasts with encoder-only and encoder–decoder architectures, traditional classification models, and prompt engineering.

liveBookmanning.com

Read Effective Conversational AI on liveBook Get the book at manning.com

Also explained in

• 

  liveBookmanning.com
  Learn Generative AI with PyTorchA Transformer design that uses only decoder stacks for autoregressive generation.
• 

  liveBookmanning.com
  Time Series Forecasting Using Foundation ModelsA decoder-only transformer is a transformer architecture that generates sequence outputs autoregressively from previously generated tokens without an encoder component.
• 

  liveBookmanning.com
  Transformers in ActionA decoder-only architecture is a sequence-model architecture that generates outputs using a decoder without a separate input encoder.
• 

  liveBookmanning.com
  Build a Text-to-Image Generator (from Scratch)A decoder-only transformer consists of decoder layers that autoregressively generate output sequences from preceding tokens.
• 

  liveBookmanning.com
  Sutskever's ListA transformer language-model architecture that generates sequences using only a decoder stack, without a separate encoder.
• 

  liveBookmanning.com
  How Large Language Models WorkA decoder-only transformer is a transformer architecture composed of decoder-style layers that generates outputs autoregressively from preceding context.
• 

  liveBookmanning.com
  Deep Learning with PyTorch, Second EditionA transformer variant that uses only the decoding stack to generate outputs autoregressively from prior context.

Compare how 8 books explain decoder-only transformer →

Related concepts

• GPT modelincludes · A GPT model is a generative pretrained Transformer language model that produces text autoregressively by predicting successive tokens from preceding context.
• TimesFMused by · TimesFM is a foundation model designed for deterministic time-series forecasting.
• Moirai-MoEused by · Moirai-MoE is a Moirai-based universal time-series forecasting model that uses fixed-length, nonoverlapping patches across frequencies.
• transformer architecturekind of · A transformer architecture is a neural-network architecture that processes sequences or other structured inputs using attention mechanisms and related transformations.
• encoder-only transformercontrasts with · An encoder-only Transformer is a Transformer model composed of encoder layers that process an input sequence into contextual representations without a separate decoder.
• masked attentionuses · Masked attention is an attention mechanism in which a mask restricts which positions can influence each other, often preventing sequence positions from accessing future tokens.

This explainer was generated by Manning's knowledge graph from the text of Effective Conversational AI. The section links point to the parts of the book it draws on.