liveBook · Manning

Vocabulary mismatch

As explained in Retrieval Augmented Generation, The Foundational Ideas

Vocabulary mismatch is a difference between the terms used in a query and the terms used in relevant source content.

Employees search an enterprise knowledge base for “automobile maintenance,” but the relevant documents say “car repair” or “vehicle service.” A keyword search may return nothing: the query and documents use different words for related ideas. That gap is vocabulary mismatch, also called lexical mismatch. section 2.1 section 2.0

Semantic search addresses this failure by representing meaning rather than relying only on shared keywords. Word2Vec helped establish the approach by learning word vectors from context; later transformer-based embeddings extended semantic representation to larger contexts. In RAG, better matching can help the retriever supply relevant documents when a query does not repeat their wording. section 2.0 section 2.1

Check the size of the gap before replacing keyword retrieval:

  • Collect a few hundred representative queries from FAQs, customer emails, or subject-matter experts. section 2.1.6
  • Lowercase and strip punctuation from queries and documents, then compare their sets of unique tokens. Calculate the share of query tokens present in the document vocabulary. section 2.1.6
  • Use the result to guide evaluation, not as proof of retrieval quality. Keyword methods may be sufficient when vocabulary overlaps; specialized domains with substantial mismatch are candidates for advanced RAG. The suggested thresholds are starting points to calibrate against your own costs. section 2.1.6 section 1.5.5
|query tokens∩ doc tokens ||query tokens|

Token coverage is the number of query tokens also found in the document vocabulary divided by the number of query tokens. It measures word overlap, not whether different words express the same meaning. section 2.1.6

Semantic matching has its own limits. Word2Vec represents individual words with static vectors, so it cannot adjust a word’s representation to its context: “bank” has the same vector whether it refers to finance or a river. It also does not directly embed whole sentences or documents. More sophisticated embeddings and similarity-search infrastructure address parts of these limitations, but retrieval still depends on how well meaning is represented and on search tradeoffs such as speed, recall, and memory. section 2.1.6 section 2.0

How Retrieval Augmented Generation, The Foundational Ideas covers it

How deep it goes
Defines the failure through an enterprise example and gives a token-overlap check, while distinguishing that check from semantic understanding.
Examples
Uses “automobile maintenance” versus “car repair” or “vehicle service,” plus query samples from FAQs and customer emails.
What it assumes
Assumes familiarity with queries and documents but introduces the keyword-matching failure through a concrete search example.
What you can do after
Evaluate/understand it by measuring query-token coverage before deciding whether keyword retrieval suffices or advanced RAG warrants testing.
How it connects
Connects keyword search, semantic embeddings, Word2Vec, Sentence-BERT, and retrieval-augmented generation through the problem of finding relevant content.

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Retrieval Augmented Generation, The Foundational Ideas. The section links point to the parts of the book it draws on.