Retrieval precision is the share of returned chunks that are relevant to a query: relevant chunks retrieved ÷ all chunks retrieved. If seven of ten chunks are relevant, precision is 0.7. It measures the quality of the retrieved set, not whether a generated answer is factually correct. section 3.6.3
Relevant, but buried
For “What’s the policy on remote work for employees with disabilities?”, a hybrid search may find the accommodation policy yet rank it third, behind less-relevant results. Broad retrieval has found the information, but the top results still need improvement. section 4.3.3
1. "Company holiday and leave policies for 2023" [marginally relevant]
2. "Benefits enrollment deadlines" [not relevant]
3. "Employees may request remote work accommodations under ADA" [highly
relevant!]
4. "General remote work guidelines" [somewhat relevant]
5. "New office opening announcement" [not relevant]
The ranked results show the accommodation policy at position three, behind a marginally relevant result and an irrelevant one. section 4.3.3
Improve the results in stages rather than asking one search method to do everything:
- Retrieve broadly with hybrid search so relevant documents have a chance to appear. section 4.3.3
- Narrow candidates with selective filtering. In a policy search, metadata can restrict results by policy ID, department, effective date, or status. section 4.3.3 section 4.5.4
- Rerank what remains with a more sophisticated model. Efficient bi-encoder similarity compares separately encoded vectors, so the query and document do not interact directly. section 4.3.3
For health-care policy retrieval, a domain-appropriate embedding model can improve precision for specialized queries. Measure retrieval precision alongside answer relevance, factual accuracy, and human evaluation: a cleaner retrieved set is useful, but it is not itself proof of a correct answer. section 4.5.4 section 3.6.3
