liveBook · Manning

Sequential evaluation

As explained in LLM Evaluation and Alignment, The Foundational Ideas

Sequential evaluation assesses separate quality dimensions through separate judge calls or evaluation passes.

Sequential evaluation assesses each quality dimension in a separate judge call. Focusing on one aspect at a time can improve attention and supports prompts specialized for different criteria. section 4.4.3

Use sequential evaluation when:

  • Quality dimensions interact in complex ways. section 4.4.3
  • Different dimensions need specialized prompts. section 4.4.3
  • You can accept cost increasing proportionally with the number of dimensions. section 4.4.3

Customer-service response

Evaluate helpfulness, factual accuracy, tone, and policy compliance through separate judge calls, using a prompt optimized for each dimension. section 4.4.3

How LLM Evaluation and Alignment, The Foundational Ideas covers it

How deep it goes
Explains separate calls, dimension-specific prompts, and proportional cost tradeoffs for multi-dimensional evaluation.
Examples
Uses customer-service responses assessed for helpfulness, accuracy, tone, and policy compliance.
What you can do after
Apply it by running one judge call per dimension and tailoring each prompt to the quality aspect.
How it connects
Connects sequential evaluation to evaluation rubrics, combined evaluation, specialized prompts, and weighted score aggregation.

Related concepts

This explainer was generated by Manning's knowledge graph from the text of LLM Evaluation and Alignment, The Foundational Ideas. The section links point to the parts of the book it draws on.