9 Evaluation and performance for LLMs and agents
This chapter covers
- Measuring and identifying hallucinations
- Implementing red-teaming and stress testing strategies for robust AI systems
- Building production-ready monitoring with frameworks
- Implementing core architectural patterns
- Evaluating agents
Throughout this book, we’ve built increasingly sophisticated large language model (LLM) applications. We started with basic prompt engineering, moved to structured outputs and function calling, and added retrieval-augmented generation (RAG) for grounding responses in real data. In chapter 8, we constructed multi-agent systems in which specialized agents collaborate on complex tasks. Each layer added capability.
Each layer also added ways for things to go wrong. A prompt can be poorly written. A structured output can fail to parse. A retrieval system can fetch irrelevant documents. An agent can call the wrong tool or loop indefinitely. But the most insidious failure—hallucination—cuts across all these problems. Unlike crashes and error messages, hallucinations don’t announce themselves. They slip past users and damage trust before anyone notices.