Real-world monitoring checks whether a deployed search system continues to serve users well as documents, data, and behavior change. Strong test results are not enough: production evidence reveals whether results match user intent and support complete workflows. section 7.2.3
Track complementary signals rather than treating one metric as decisive:
- Click-through rate shows whether users select the results ranked most prominently. section 7.2.3
- Query-log analysis exposes recurring searches that produce little interaction, highlighting intent or coverage gaps. section 7.2.3
- Dwell time and bounce rate indicate whether retrieved documents appear useful or irrelevant. section 7.2.3
- Session analysis measures workflow efficiency when users issue several queries to complete a task. section 7.2.3
A refinement signal
If users repeatedly search for “feature comparison” but do not interact with the results, the system needs refinement even if offline evaluation looked strong. section 7.2.3
