TF-IDF is a statistical weighting scheme that scores terms by their frequency within a document and rarity across a collection. In retrieval-augmented generation, it supports sparse retrieval by matching query and document keywords. section 7.4
TF-IDF is useful when retrieval emphasizes efficient keyword matching:
- Uses inverted indexes to search large collections quickly with limited computational resources. section 7.4
- Fits keyword-dependent search tasks but can miss synonyms, nuanced phrasing, and deeper semantic relationships. section 7.4
- Contrasts with dense retrieval, which uses vectors and machine learning to capture semantic similarity at greater computational cost. section 7.4
Keyword-dependent RAG search
A RAG system can use a TF-IDF-based sparse retriever to search its knowledge corpus for documents sharing important query terms, then pass the retrieved information to the generator model. section 7.4
