Missed the top rank is a retrieval failure in which the answer-bearing passage exists in the corpus but loses to a less useful lookalike. Similarity-only search rewards shared words or embeddings without knowing which document helps answer the question, so near-duplicates can outrank the correct evidence. section 3.0
Ambiguous acronyms in a fintech knowledge base
A query containing “CMA” may match documents about Consumer Management Application, Cardholder Management Architecture, or Credit Monitoring Alert. Keyword overlap can rank the wrong CMA document first even though the correct document is present. This differs from a retrieval failure where the correct document is absent from the top K results. section 15.7.1 section 4.5.5
Treat the failure as an ordering problem and evaluate it before changing the generator:
- Measure a baseline retriever with retrieval accuracy or Hit Rate @5, rather than assuming generated-answer quality reveals ranking quality. section 15.7.1
- Train the retriever on whether a document helps answer the question; changing encoder weights moves useful passages closer while preserving nearest-neighbor search. section 3.0
- Compare simple similarity retrieval with approaches that jointly optimize retrieval and reading, while tracking latency and query types such as procedural or multi-document questions. section 3.0 section 15.7.1
- Do not expect generation to repair a missing or badly ranked foundation; diagnose retrieval separately from chunking, reranking, and generation. section 4.5.5
