Reader robustness is an answer-producing component’s ability to preserve answer quality when retrieved context contains irrelevant or imperfect passages. At scale, it can matter more than retriever quality: a stronger retriever helps little if the generator cannot separate useful evidence from noise. section 15.0
Production systems expose reader robustness through contrasting outcomes:
- A dense retriever can raise recall substantially while leaving answer quality nearly unchanged when the generator is overwhelmed by retrieved documents. section 15.0
- Agentic components can improve generation as well as retrieval by decomposing queries and re-ranking results into cleaner, more focused context. section 15.7.6
- Complexity should follow diagnosed failure points rather than being applied uniformly; routing by query complexity helps control average latency. section 15.7.6
Retrieval improves, answers do not
In a legal document system, replacing BM25 with a dense retriever improved recall by 18 points but barely changed answer quality because the generator was already overwhelmed by five retrieved documents. The failure points suggested focusing on context quality and reader behavior, not retrieval alone. section 15.0
