A vector index helps a retrieval system find relevant content without comparing a query against every stored embedding. It is a data structure for fast similarity or nearest-neighbor lookup in high-dimensional vector space. A vector database stores embeddings alongside content and provides database capabilities; the index makes searching those embeddings efficient. This matters in retrieval-augmented generation (RAG), where slow retrieval can delay the answer. section 7.4 section 7.5
To prepare an index for RAG, split source documents into manageable chunks, create an embedding for each chunk, and retain a link to its source as metadata. A query is also embedded. Vector search uses the index to find similar document or passage embeddings, whose content can then be supplied to the LLM as context. section 7.4 section 7.5.2
Index design involves practical trade-offs:
- Choose an exact or approximate nearest-neighbor approach, such as HNSW or IVF, according to speed and accuracy needs. section 7.5.1
- Choose a similarity measure suited to the data and query, such as cosine similarity or squared Euclidean distance. section 7.5.1 section 7.5.2
- Consider compression, such as quantization or pruning, when balancing speed, accuracy, and memory consumption. section 7.5.1
Searching blog posts for an answer
In a RAG chat application, blog posts are chunked, embedded, and indexed in Redis. A question becomes an embedding; a hybrid search returns the top five matching chunks. Those chunks and the original question are placed in the prompt before the LLM generates an answer. The local Redis deployment illustrates the flow, but setting up containers, ingesting data, and connecting retrieval to generation is not by itself a production-ready enterprise deployment. section 8.6 section 8.9
Listing 8.9 End-to-end RAG-powered chat
def hybrid_search(query_vector, client, top_k=5, hybrid_fields="*"):
...
return results
def get_search_results(query:str, max_token=4096,
debug_message=False) -> str:
query_vector = get_embedding(query) #1
query_vector = np.array(query_vector).astype(
np.float32).tobytes() #2
print("Searching for similar posts...")
results = hybrid_search(query_vector, conn, top_k=5) #3
token_budget = max_token - count_tokens(query) #4
if debug_message:
print(f"Token budget: {token_budget}")
message = 'Use the blog post below to answer the subsequent
question. If the answer cannot be found in the
articles, write "Sorry, I could not find an answer in
the blog posts."'
question = f"\n\nQuestion: {query}"
if results:
for i, post in enumerate(results.docs): #5
next_post = f'\n\nBlog post:\n"""\n{post.content}\n"""'
new_token_usage = count_tokens(message + question + next_post)
if new_token_usage < token_budget:
if debug_message:
print(f"Token usage: {new_token_usage}")
message += next_post
else:
break
else:
print("No results found")
return message + question
def ask_gpt(query : str, max_token = 4096, debug_message = False) -> str:
message = get_search_results( #6
query,
max_token,
debug_message=debug_message)
messages = [ #7
{"role":
"system",
"content": "You answer questions in summary from the [CA]
blog posts."},
{"role":
"user",
"content": message},]
response = openai.ChatCompletion.create( #8
model="gpt-3.5-turbo-16k",
messages=messages,
temperature=0.7,
max_tokens=2000,
top_p=0.95
)
response_message = response["choices"][0]["message"]["content"]
return response_message
if __name__ == "__main__":
# Enter a query
while True:
query = input("Please enter your query: ")
print(ask_gpt(query, max_token=15000, debug_message=False))
print("=="*20)
The listing shows the handoff from query embedding to top-five hybrid search and prompt construction. section 8.6
For production use, plan the source-data integration and ingestion pipeline as carefully as the search itself. Data shape affects chunking and embedding work, while performance, cost, security, and scale make the vector database and index operational choices—not just implementation details. section 7.5 section 8.9
