Qualitative assessment evaluates a search system through observed user experience, manual judgments, and detailed feedback before teams rely on advanced metrics. It starts with quick vibe checks and can become more systematic as evidence and optimization needs grow. section 7.2.3
Useful approaches include:
- User studies: Have people perform realistic tasks and describe whether results are relevant or require searching through irrelevant documents. section 7.2.3
- Relevance assessments: Ask evaluators to judge whether results satisfy a query’s intent, creating structured feedback about what works. section 7.2.3
- Task-based evaluation: Observe system performance while users complete live work, such as retrieving customer feedback for a case study. section 7.2.3
