liveBook · Manning

Retrieval index

As explained in Designing AI Systems

A retrieval index is a searchable data structure that organizes document representations and associated metadata so relevant information can be found efficiently.

A retrieval index is a named, isolated collection of documents configured for search. It gives a workflow a defined body of knowledge to retrieve from, helping ground responses in relevant organizational information rather than plausible guesses. section 5.2 section 5.0

During ingestion, documents are parsed, split into chunks, and converted to embeddings. The vector store keeps the embeddings with chunk text and metadata. At query time, a query embedding is matched against stored vectors; results include source attribution, scores, and metadata. Metadata filters can distinguish content that is semantically similar but intended for different audiences. section 5.1

Figure 5.1 The retrieval pipeline from document to search result. A source document enters the ingestion pipeline, where it is parsed into clean text and then broken into semantically coherent chunks. Each chunk is passed through an embedding model to produce a dense vector representation. The vectors, along with the original chunk text and metadata, are stored in a vector database. At query time, the user's question follows the same embedding process to produce a query vector, which is compared against stored vectors using similarity search. The top matching chunks are returned with their source attribution, relevance scores, and metadata, ready to be injected into the model's context as grounding information.

The pipeline connects document ingestion to similarity search and grounded model context. section 5.1

Each index has its own embedding and chunking configuration. Searches are scoped to an index, so a support workflow can search troubleshooting guides without returning legal compliance documents. An allowed list of embedding models can also prevent costly or unapproved model choices at creation time. section 5.2.1

Listing 5.1 Index configuration dataclass
@dataclass
class IndexConfig:
    name: str    #A
    embedding_model: str = 
➥"text-embedding-3-small"    #B
    embedding_dimensions: int = 1536    #B
    chunking_strategy: str = "fixed"    #C
    chunk_size: int = 512    #C
    chunk_overlap: int = 50    #C
    metadata_schema: Optional[Dict] = None    #D

The configuration identifies the index and defines its embedding, chunking, and optional metadata schema settings. section 5.2.2

Configuration choices have operational consequences:

  • Set the embedding model and dimensions at creation. Changing the model would make existing vectors incompatible with new ones; reject changes to an existing index. section 5.2.2
  • To change models, create a new index, re-ingest documents, validate retrieval, and switch traffic. Retain the old index temporarily for rollback. section 5.2.2
  • Use a metadata schema to catch inconsistent keys at ingestion, before filters silently miss documents. Choose chunking to suit the content: preserve self-contained FAQ entries, but split long manuals into retrievable pieces. section 5.2.2 section 5.2.1

Creating an index sets up an empty container; ingestion is a separate, ongoing operation. Management operations create, list, retrieve, and delete indexes. Counts and the last-ingested timestamp help teams inspect size and freshness. An owner field provides a foundation for access control, but production enforcement must check caller identity before destructive operations. section 5.2.3

The vector store scopes chunks by index and document, allowing updated documents or entire indexes to be removed. Search accepts an index name, query embedding, result limit, optional metadata filters, and a score threshold. Returned results carry chunk and document identifiers, text, score, and metadata. section 5.4.1

Listing 5.17 VectorStore abstract base class: search operation and result model
@dataclass
class SearchResult:
    chunk_id: str
    document_id: str
    text: str
    score: float    #A
    metadata: Dict[str, str]
 
class VectorStore(ABC):
    # ... write operations from Listing 5.16 ...
 
    @abstractmethod
    def search(
        self,
        index_name: str,
        query_embedding: List[float],
        top_k: int = 5,
        metadata_filters: Optional[Dict[str, str]] = None,
        score_threshold: Optional[float] = None
    ) -> List[SearchResult]:    #B
        """Find chunks most similar to the query embedding."""
        pass

The search interface makes index scope, filtering, and the returned result fields explicit. section 5.4.1

How Designing AI Systems covers it

How deep it goes
Defines configuration and lifecycle interfaces, explains migration and isolation, and defers ownership enforcement until authentication exists.
Examples
Contrasts support troubleshooting and legal compliance indexes, and compares FAQ entries with long technical manuals.
What you can do after
Operate and configure isolated indexes, migrate embedding models, and inspect ingestion freshness.
How it connects
Connects document chunking, embeddings, vector stores, metadata filtering, hybrid search, and retrieval-augmented generation.

Also explained in

Compare how 4 books explain retrieval index →

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Designing AI Systems. The section links point to the parts of the book it draws on.