A question about yesterday’s uploaded document cannot be answered from an agent’s training data alone. Retrieval finds relevant information in external storage and brings selected pieces into the current prompt. Similarity search is one way to choose those pieces: it ranks stored items by how closely they match a query, rather than looking up an exact key. section 6.0 section 6.1 section 6.1.1
The representation determines what “similar” means. A word-based vectorizer can match related words and phrases but miss a passage’s meaning. Semantic search uses embeddings to find content with similar meaning even when its wording differs. A keyword search for “dog” depends on that word appearing; a semantic search may also find a passage about a particular breed or puppy care. section 6.2 section 6.1.2
For document question answering, follow a small retrieval pipeline:
- Split documents into chunks, encode the chunks as vectors, and store them in a vector database. section 6.1.1
- Encode the user’s query with the embedding model, then retrieve the most similar chunks. A top-K setting limits how many results enter the prompt. section 6.1.1 section 6.1.2
- Give those chunks to the answer-generating model as context. Require it to answer from retrieved passages, cite them, and say when they do not contain the answer. section 6.3.2 section 11.3
Listing 6.3 document_vector_database.py
# code above omitted
vectorizer = TfidfVectorizer()
X = vectorizer.fit_transform(documents)
vector_database = X.toarray() #1
def cosine_similarity_search(query,
database,
vectorizer,
top_n=5): #2
query_vec = vectorizer.transform([query]).toarray()
similarities = cosine_similarity(query_vec, database)[0]
top_indices = np.argsort(-similarities)[:top_n] # Top n indices
return [(idx, similarities[idx]) for idx in top_indices]
while True: #3
query = input("Enter a search query (or 'exit' to stop): ")
if query.lower() == 'exit':
break
top_n = int(input("How many top matches do you want to see? "))
search_results = cosine_similarity_search(query,
vector_database,
vectorizer,
top_n)
print("Top Matched Documents:")
for idx, score in search_results:
print(f"- {documents[idx]} (Score: {score:.4f})") #4
print("\n")
###Output
Enter a search query (or 'exit' to stop): blue
How many top matches do you want to see? 3
Top Matched Documents:
- The sky is blue and beautiful. (Score: 0.4080)
- Love this blue and beautiful sky! (Score: 0.3439)
- The brown fox is quick and the blue dog is lazy! (Score: 0.2560)
The small in-memory example turns documents and a query into vectors, scores them with cosine similarity, and returns the highest-ranked document indices and scores. It makes the ranking step inspectable before adding semantic embeddings or an agent. section 6.2
From a search exercise to a RAG agent
A practical progression starts by entering a query against an in-memory document collection and inspecting its scores. Next, chunk a movie script, embed its chunks, and store them in Chroma DB; a search tool can then supply script passages to an agent answering questions about the script. Ask about an exact time as well as a general scene: the difference helps expose what vector-only retrieval misses. section 6.2 section 6.3.2 section 6.5
A close match is not proof of a correct answer. Similarity search can return outdated or loosely related passages, miss exact codes and numbers, or drop a useful result just below the top-K cutoff. Approximate nearest-neighbor indexes can also miss the true closest match. For reliable coverage, pair vector search with keyword search or other appropriate lookups; adjust K, rerank or deduplicate results, and filter by metadata such as version, date, or access role. Inspect retrieval misses before treating them as answer-generation failures. section 6.1.2 section 6.3.1 section 11.1.4
