A traditional database can find a row where an ID equals 42. A vector database answers a different question: which stored vectors are most similar to this query vector? It ranks matches by similarity score. In retrieval-augmented generation (RAG), those matches help supply relevant context for an answer. section 6.1.1
Think of document retrieval as two phases:
- Ingest: load documents, split them into chunks, turn the chunks into vectors with an embedding model, and store them in the vector database. section 6.1.1
- Retrieve: embed a user's question, find similar stored chunks, and add those chunks to the prompt. A separate language model produces the answer. section 6.1.1
Start with an in-memory search
A small Python prototype converts documents into TF-IDF vectors. It converts each query the same way, calculates cosine similarity, and returns the top document matches with scores. This makes the search mechanism inspectable, but word-and-phrase matching can miss a document's meaning. section 6.2
Listing 6.3 document_vector_database.py
# code above omitted
vectorizer = TfidfVectorizer()
X = vectorizer.fit_transform(documents)
vector_database = X.toarray() #1
def cosine_similarity_search(query,
database,
vectorizer,
top_n=5): #2
query_vec = vectorizer.transform([query]).toarray()
similarities = cosine_similarity(query_vec, database)[0]
top_indices = np.argsort(-similarities)[:top_n] # Top n indices
return [(idx, similarities[idx]) for idx in top_indices]
while True: #3
query = input("Enter a search query (or 'exit' to stop): ")
if query.lower() == 'exit':
break
top_n = int(input("How many top matches do you want to see? "))
search_results = cosine_similarity_search(query,
vector_database,
vectorizer,
top_n)
print("Top Matched Documents:")
for idx, score in search_results:
print(f"- {documents[idx]} (Score: {score:.4f})") #4
print("\n")
###Output
Enter a search query (or 'exit' to stop): blue
How many top matches do you want to see? 3
Top Matched Documents:
- The sky is blue and beautiful. (Score: 0.4080)
- Love this blue and beautiful sky! (Score: 0.3439)
- The brown fox is quick and the blue dog is lazy! (Score: 0.2560)
Inspect the vectorization, cosine-similarity calculation, and top-match selection in the in-memory prototype. section 6.2
Store chunks for a RAG agent
For a movie-script search agent, split the script into token-limited chunks, embed them, and use a persistent Chroma DB collection for later retrieval. Create the collection if it is missing and populate it if it has no vectors. The agent can then search the indexed script; instruct it to answer from retrieved passages and say when they do not contain an answer. section 6.3.2
Similarity search is useful, but it can miss exact matches, jargon, numbers, and fresh updates. For broader coverage, combine it with keyword, relational, or graph queries. Require citations to retrieved passages when answers must be grounded. section 6.6
