A retrieval index is a named, isolated collection of documents configured for search. It gives a workflow a defined body of knowledge to retrieve from, helping ground responses in relevant organizational information rather than plausible guesses. section 5.2 section 5.0
During ingestion, documents are parsed, split into chunks, and converted to embeddings. The vector store keeps the embeddings with chunk text and metadata. At query time, a query embedding is matched against stored vectors; results include source attribution, scores, and metadata. Metadata filters can distinguish content that is semantically similar but intended for different audiences. section 5.1
Figure 5.1 The retrieval pipeline from document to search result. A source document enters the ingestion pipeline, where it is parsed into clean text and then broken into semantically coherent chunks. Each chunk is passed through an embedding model to produce a dense vector representation. The vectors, along with the original chunk text and metadata, are stored in a vector database. At query time, the user's question follows the same embedding process to produce a query vector, which is compared against stored vectors using similarity search. The top matching chunks are returned with their source attribution, relevance scores, and metadata, ready to be injected into the model's context as grounding information.

The pipeline connects document ingestion to similarity search and grounded model context. section 5.1
Each index has its own embedding and chunking configuration. Searches are scoped to an index, so a support workflow can search troubleshooting guides without returning legal compliance documents. An allowed list of embedding models can also prevent costly or unapproved model choices at creation time. section 5.2.1
Listing 5.1 Index configuration dataclass
@dataclass
class IndexConfig:
name: str #A
embedding_model: str =
➥"text-embedding-3-small" #B
embedding_dimensions: int = 1536 #B
chunking_strategy: str = "fixed" #C
chunk_size: int = 512 #C
chunk_overlap: int = 50 #C
metadata_schema: Optional[Dict] = None #DThe configuration identifies the index and defines its embedding, chunking, and optional metadata schema settings. section 5.2.2
Configuration choices have operational consequences:
- Set the embedding model and dimensions at creation. Changing the model would make existing vectors incompatible with new ones; reject changes to an existing index. section 5.2.2
- To change models, create a new index, re-ingest documents, validate retrieval, and switch traffic. Retain the old index temporarily for rollback. section 5.2.2
- Use a metadata schema to catch inconsistent keys at ingestion, before filters silently miss documents. Choose chunking to suit the content: preserve self-contained FAQ entries, but split long manuals into retrievable pieces. section 5.2.2 section 5.2.1
Creating an index sets up an empty container; ingestion is a separate, ongoing operation. Management operations create, list, retrieve, and delete indexes. Counts and the last-ingested timestamp help teams inspect size and freshness. An owner field provides a foundation for access control, but production enforcement must check caller identity before destructive operations. section 5.2.3
The vector store scopes chunks by index and document, allowing updated documents or entire indexes to be removed. Search accepts an index name, query embedding, result limit, optional metadata filters, and a score threshold. Returned results carry chunk and document identifiers, text, score, and metadata. section 5.4.1
Listing 5.17 VectorStore abstract base class: search operation and result model
@dataclass
class SearchResult:
chunk_id: str
document_id: str
text: str
score: float #A
metadata: Dict[str, str]
class VectorStore(ABC):
# ... write operations from Listing 5.16 ...
@abstractmethod
def search(
self,
index_name: str,
query_embedding: List[float],
top_k: int = 5,
metadata_filters: Optional[Dict[str, str]] = None,
score_threshold: Optional[float] = None
) -> List[SearchResult]: #B
"""Find chunks most similar to the query embedding."""
passThe search interface makes index scope, filtering, and the returned result fields explicit. section 5.4.1



