liveBook · Manning

Similarity search

As explained in Retrieval Augmented Generation, The Foundational Ideas

Similarity search finds items whose representations are close to a query representation according to a similarity or distance function.

A support ticket says, “My internet connection keeps dropping.” A resolved ticket calls the problem “WiFi connectivity issues.” Keyword search may miss the match because the wording differs. Similarity search instead encodes the new ticket and stored tickets as vectors, then ranks stored vectors by closeness to the query vector. section 2.3 section 2.4.1

For document retrieval, embed passages and build an index before users ask questions. At query time, embed the question, search the index, and return nearby passages. A RAG system can place those passages in a generator’s prompt. Sentence-BERT makes independent sentence embeddings practical to precompute; FAISS provides indexing options for searching large collections. section 2.5 section 4.6

Choose the search method for the mismatch and the workload, not for sophistication alone:

  • Start with BM25 when exact terminology matters and query vocabulary substantially overlaps the corpus. Use dense search when different wording expresses related ideas; combine both for mixed domains. section 2.1.6
  • For similar-length texts such as duplicate tickets, Sentence-BERT supports symmetric matching. For a short question seeking a longer answer passage, DPR trains separate query and passage encoders for that asymmetric task. section 2.3 section 3.1
  • At scale, weigh exact search against approximate nearest-neighbor indexing. Measure recall, latency, and memory as the index grows; faster results are useful only if the needed passages remain retrievable. section 2.2.3

When closeness is not enough

Searching “automobile maintenance” with averaged word vectors gives a document about a car a nonzero similarity score where BM25 gives it zero. Yet an unrelated computer-crash document ranks slightly higher: averaging word vectors loses word order and sentence structure. Dense retrieval can also miss an answer when a brief, informal question differs sharply from a technical passage. Compare rankings against known relevant passages rather than assuming semantic proximity means answer usefulness. section 2.1.3 section 7.1 section 3.0

Listing 4.2 Turning a paper into a searchable index.
from langchain_community.document_loaders import PyPDFLoader 
from langchain_text_splitters import RecursiveCharacterTextSplitter 
from langchain_huggingface import HuggingFaceEmbeddings 
from langchain_community.vectorstores import FAISS
PAPER = "https://arxiv.org/pdf/2005.11401" 
splitter = RecursiveCharacterTextSplitter(
  chunk_size=512, chunk_overlap=50
)  #A
chunks = PyPDFLoader(PAPER).load_and_split(text_splitter=splitter) 
embeddings = HuggingFaceEmbeddings(  #B
  model_name="sentence-transformers/all-mpnet-base-v2"
)
index = FAISS.from_documents(chunks, embeddings)  #C
print(f"{len(chunks)} chunks indexed")

A Python indexing example splits a PDF into chunks, embeds them with a Hugging Face model, and builds a FAISS index before query-time search. section 4.6

Listing 4.3 Retrieving, then generating from what was retrieved.
from langchain_core.prompts import ChatPromptTemplate 
from providers import get_chat_model
 
llm = get_chat_model(temperature=0)  #A
prompt = ChatPromptTemplate.from_template(
  "Answer from the context only. If the context does not " 
  "contain the answer, say so.\n\n{context}\n\n" 
  "Question: {question}"
)  #B
chain = prompt | llm  #C
 
def answer(question, k=3): 
  docs = index.similarity_search(question, k=k)  #D
  context = "\n\n".join(d.page_content for d in docs) 
  reply = chain.invoke({"context": context, "question": question}) 
  pages = [d.metadata["page"] for d in docs]  #E
  return reply.content, pages

The corresponding query-time example calls similarity_search, passes the retrieved chunks to a generator, and returns the answer with source pages. section 4.6

If a short query still lands far from the right passage, a hypothetical-document query can improve retrieval; a supplied medical example raises recall at two results from 0.50 for direct dense search to 1.00 with HyDE. This is a comparison on that small corpus, not a guarantee. Retrieval can also introduce wrong context into an otherwise answerable question, so evaluate the final answers as well as search rankings. section 7.1 section 1.3.5

How Retrieval Augmented Generation, The Foundational Ideas covers it

How deep it goes
Explains vector encoding, indexing, query-time ranking, and production tradeoffs, without deriving nearest-neighbor algorithms.
Examples
Compares car-document rankings under BM25 and word vectors, and demonstrates PDF indexing and query-time retrieval in Python.
What it assumes
Assumes basic Python familiarity for implementation while explaining keyword-versus-vector retrieval through concrete rankings.
What you can do after
Apply it via LangChain, Hugging Face embeddings, and FAISS to index document chunks, retrieve passages, and inspect answer context.
How it connects
Connects vector embeddings and nearest-neighbor indexing to BM25, DPR, HyDE, and retrieval-augmented generation.

Also explained in

Compare how 37 books explain similarity search →

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Retrieval Augmented Generation, The Foundational Ideas. The section links point to the parts of the book it draws on.