Chroma DB is a vector database for finding text by meaning rather than by keyword alone. In a question-answering chatbot, it stores document chunks and their embeddings, then retrieves chunks similar to a question. A language model can use those chunks to compose an answer without receiving the entire document collection in every prompt. section 6.0 section 6.3
A small implementation has two phases:
- Ingest: create a collection, split source text into chunks, and add each chunk with an ID and useful metadata such as its source. Chroma generates embeddings from supplied text using its default model unless you specify another. section 6.2.5
- Retrieve: query the collection with a question and choose how many results to return. The response includes matching documents, IDs, metadata, and distances. Pass relevant text, together with the original question, to the language model for answer generation. section 6.2.5 section 6.3
Listing 6.1 Creating and populating a ChromaDB collection
tourism_collection.add(
documents=[
"Paestum, Greek Poseidonia, …[shortened] … Greek temples.",
"Poseidonia was probably …[shortened] … in the 18th century.",
"The ancient Greek part of …[shortened] … from the site."
],
metadatas=[
{"source": "https://www.britannica.com/place/Paestum"},
{"source": "https://www.britannica.com/place/Paestum"},
{"source": "https://www.britannica.com/place/Paestum"}
],
ids=["paestum-br-01", "paestum-br-02", "paestum-br-03"]
)
A Paestum collection pairs three short passages with source URLs and distinct IDs. This is the data the search will retrieve, not an answer written in advance. section 6.2.5
Finding an answer about Paestum
Querying for the number of Doric temples with n_results=1 returns a passage stating that Paestum has three. Raising n_results to 3 returns that passage alongside two other matches. Retrieval supplies evidence; the language model turns it into a response. section 6.2.5 section 6.3
Choose storage deliberately. A chromadb.Client() example keeps data in memory and loses it when the notebook session ends; PersistentClient(path="./chroma_db") stores it at a specified path. A server started with chroma run can instead be reached through HttpClient. For a larger document pipeline, LangChain can load files, split them, calculate embeddings with OpenAIEmbeddings, and add the resulting chunks to Chroma. section 6.2.5 section 7.2.1
Retrieval quality depends on the chunks as well as the database. Splits that discard context can weaken answers; test the number of returned chunks against answer quality. If you change embedding models, ingestion and retrieval must still use matching dimensions, and an existing corpus may need re-embedding. section 6.4
