liveBook · Manning

Reliability

As explained in Agent Design Patterns

Reliability is the degree to which a system consistently produces correct and dependable results under expected conditions.

A model can appear to follow a detailed prompt yet skip a step or produce only half the requested result. Splitting the work into one entry per call makes mistakes easier to spot and supervise, though it does not eliminate them. section 1.0

In an agent, reliability has a specific operational meaning: how often the system avoids internal failures that prevent its routines from completing. A model call that returns an unusable format or a search tool that returns no results can stop a workflow even when its design is sound. An agent design pattern improves reliability when it reduces the rate of such failures. section 1.2.2

Reliability is one reason to put a structured process around a model:

  • Narrow the model’s responsibilities, supervise its information sources, and impose fixed, interpretable processes and output restrictions rather than relying on instructions alone. section 1.1.2
  • Use guardrails to gate information flow within and around the agent; sequential, parallel, buffered, and streaming arrangements involve safety, latency, and cost trade-offs. section 4.0
  • Log internal failures on capability and robustness datasets or during real-world use to measure whether a change actually reduces them. section 1.2.2

Do not conflate reliability with capability or robustness. Capability concerns what an agent can correctly handle or do; robustness concerns whether success survives changes in input or task structure. Reliability concerns failures of the agent’s own workflow. section 1.2.2

How Agent Design Patterns covers it

How deep it goes
Defines reliability by internal failure rate, gives failure cases and a logging-based measurement approach, but does not specify a recovery procedure.
Examples
Uses skipped prompt instructions, malformed model responses, empty search results, and guarded personal-assistant behavior as concrete cases.
What you can do after
Evaluate/understand it by logging agent failures on test datasets or in real-world use and checking whether their rate decreases.
How it connects
Distinguishes reliability from capability and robustness while linking it to agent design patterns and guardrails.

Also explained in

Compare how 21 books explain reliability →

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Agent Design Patterns. The section links point to the parts of the book it draws on.