liveBook · Manning

Generation

As explained in LLM Evaluation and Alignment, The Foundational Ideas

Natural language generation is the task of producing human-readable language from structured, sensory, or linguistic inputs.

Generation produces a response by synthesizing a user question with retrieved documents. In Retrieval-Augmented Generation, it supplies reasoning while retrieval supplies domain knowledge. section 6.0 section 6.1

Treat generation as one independently failing part of a larger system:

  • Use retrieved documents as context before producing an answer. section 6.1
  • Distinguish generation failures from retrieval failures when diagnosing incoherent or unsupported responses. section 6.0
  • Check whether the response answers every part of the question rather than merely repeating supported facts. section 6.0

Financial research assistant

Given earnings-report excerpts stating $2.3 billion in revenue, 15% year-over-year growth, and cloud services as the primary driver, generation produced those supported facts but also invented European expansion and a new CEO, while omitting risks affecting Q4. section 6.0

Figure 6.1 The basic single-turn Retrieval-Augmented Generation architecture. A user question is routed to a retrieval system, which queries a knowledge base and returns relevant documents. The LLM synthesizes an answer to the question and retrieves context, producing responses supported by sources.

Trace how a user question becomes a retrieval query, retrieved documents, and a synthesized answer in basic single-turn RAG. section 6.1

How LLM Evaluation and Alignment, The Foundational Ideas covers it

How deep it goes
Explains context-based synthesis and interacting component failures, but does not detail generation algorithms or evaluation metrics.
Examples
Uses a financial research assistant that combines earnings-report excerpts with a partially hallucinated and incomplete answer.
What you can do after
Diagnose whether unsupported or incomplete answers arise from generation, retrieval, or their interaction.
How it connects
Connects generation with Retrieval-Augmented Generation, information retrieval, knowledge bases, context windows, and hallucination.

Also explained in

Compare how 29 books explain generation →

Related concepts

This explainer was generated by Manning's knowledge graph from the text of LLM Evaluation and Alignment, The Foundational Ideas. The section links point to the parts of the book it draws on.