Employees search an enterprise knowledge base for “automobile maintenance,” but the relevant documents say “car repair” or “vehicle service.” A keyword search may return nothing: the query and documents use different words for related ideas. That gap is vocabulary mismatch, also called lexical mismatch. section 2.1 section 2.0
Semantic search addresses this failure by representing meaning rather than relying only on shared keywords. Word2Vec helped establish the approach by learning word vectors from context; later transformer-based embeddings extended semantic representation to larger contexts. In RAG, better matching can help the retriever supply relevant documents when a query does not repeat their wording. section 2.0 section 2.1
Check the size of the gap before replacing keyword retrieval:
- Collect a few hundred representative queries from FAQs, customer emails, or subject-matter experts. section 2.1.6
- Lowercase and strip punctuation from queries and documents, then compare their sets of unique tokens. Calculate the share of query tokens present in the document vocabulary. section 2.1.6
- Use the result to guide evaluation, not as proof of retrieval quality. Keyword methods may be sufficient when vocabulary overlaps; specialized domains with substantial mismatch are candidates for advanced RAG. The suggested thresholds are starting points to calibrate against your own costs. section 2.1.6 section 1.5.5
Token coverage is the number of query tokens also found in the document vocabulary divided by the number of query tokens. It measures word overlap, not whether different words express the same meaning. section 2.1.6
Semantic matching has its own limits. Word2Vec represents individual words with static vectors, so it cannot adjust a word’s representation to its context: “bank” has the same vector whether it refers to finance or a river. It also does not directly embed whole sentences or documents. More sophisticated embeddings and similarity-search infrastructure address parts of these limitations, but retrieval still depends on how well meaning is represented and on search tradeoffs such as speed, recall, and memory. section 2.1.6 section 2.0
