A Q&A chatbot cannot fit a large knowledge base into every prompt. Similarity search finds a few relevant document chunks first, so the model can answer using retrieved context rather than the entire collection. section 6.4 section 1.4.2
During ingestion, documents are split into chunks. An embedding model converts each chunk into a vector, and a vector store saves the vectors alongside the text. At question time, the same embedding model converts the query into a vector. The store returns chunks with nearby vectors, typically ranked by distance. Because the comparison uses embeddings, a search for “feline animals” can find chunks about cats, lions, or tigers without an exact word match. section 6.1.3
Figure 6.4 RAG Q&A stage: Retrieval and generation
The retrieval-and-generation diagram shows where similarity search sits between the user question and the LLM’s answer. section 6.1.3
In LangChain, a vector store can expose a retriever with `retriever = vector_db.as_retriever()`. A RAG chain sends the question to that retriever for `context`, passes the original question through separately, then combines both in a prompt for the chat model. The retriever finds evidence; the model writes the answer. section 7.3.3 section 7.1.2
rag_chain = {"context": retriever,
"question": question_feeder}|rag_prompt|chatbot
This short chain definition shows the retriever supplying prompt context while the original question follows a separate path. section 7.3.3
Search for activities in Cornwall
A query-rewriting chain turns “Tell me some fun things I can do in Cornwall” into “fun activities to do in Cornwall.” Calling `uk_granular_collection.similarity_search(query=search_query, k=3)` returns three chunks to inspect. The shown results include surfing, walking, cycling, and festivals; they offer a broader selection than the original-question results. section 9.1.3
Check retrieval quality before trusting the generated answer:
- Inspect the returned chunks. Poor splitting can leave sentences broken or context orphaned, even when the search finds those chunks. section 6.4
- Adjust `k`, the number of chunks returned. Start with four as a baseline, then test against answer quality, content density, and the model’s context window. section 6.4
- Keep query and document embeddings consistent. Changing embedding models can require re-embedding the corpus; dimensions must match. section 6.4
- Use another retrieval method when semantic closeness is not enough. Sparse retrieval matches terms for precise keyword queries; hybrid search combines semantic and keyword results, while metadata filters can narrow the chunks searched. section 10.1 section 10.2
