liveBook · Manning

Accuracy

As explained in AI Agents in Action, Second Edition

Routing accuracy is the proportion of inputs assigned to the appropriate agent or processing route.

Accuracy measures how often outputs match a known target. For routing, it is the share of inputs assigned to the appropriate agent or route. section 7.3.4

Choose the evaluation method that fits the output:

  • Use accuracy and related metrics when outputs have known targets, such as classification labels, extracted fields, or optimization decisions. section 7.3.4
  • For open-ended outputs such as summaries, explanations, plans, or creative content, use rubrics, LLM-as-judge evaluation, or human review instead. section 7.3.4

How AI Agents in Action, Second Edition covers it

How deep it goes
Explains when accuracy applies and contrasts it with rubric-based evaluation, without detailing metric calculations.
Examples
Uses classification labels, extracted fields, optimization decisions, summaries, explanations, plans, and creative content as output examples.
What it assumes
Requires outputs with known targets; contrasts these with open-ended outputs whose quality is multidimensional.
What you can do after
Evaluate structured outputs with standard metrics and select other methods when correctness is not clearly defined.
How it connects
Connects accuracy with precision, recall, F1 score, rubrics, LLM-as-judge evaluation, and human review.
Cover of AI Agents in Action, Second Edition
liveBookmanning.com
Read AI Agents in Action, Second Edition on liveBook Get the book at manning.com

Also explained in

Compare how 32 books explain accuracy →

Related concepts

This explainer was generated by Manning's knowledge graph from the text of AI Agents in Action, Second Edition. The section links point to the parts of the book it draws on.