liveBook · Manning

Vector database

As explained in Designing AI Systems

A vector database is a database system optimized for storing, indexing, and retrieving vector representations.

A vector database, or vector store, stores embeddings and retrieves them by similarity. It makes organizational documents searchable by meaning: related phrases can match even when they share few words. At production scale, approximate nearest-neighbor search narrows candidates without comparing every stored vector. section 5.1

Ingestion extracts text, divides it into chunks, embeds each chunk, and stores the vectors with their text and metadata. Search embeds the query using the same model as the indexed chunks, then retrieves the closest results. Results include source attribution, scores, and metadata for downstream workflows. section 5.3 section 5.4.4 section 5.1

Figure 5.1 The retrieval pipeline from document to search result. A source document enters the ingestion pipeline, where it is parsed into clean text and then broken into semantically coherent chunks. Each chunk is passed through an embedding model to produce a dense vector representation. The vectors, along with the original chunk text and metadata, are stored in a vector database. At query time, the user's question follows the same embedding process to produce a query vector, which is compared against stored vectors using similarity search. The top matching chunks are returned with their source attribution, relevance scores, and metadata, ready to be injected into the model's context as grounding information.

Trace the complete path from a source document to attributed search results. section 5.1

Filter similar results by audience

A customer troubleshooting guide and an internal engineering runbook may describe the same reset procedure and be semantically close. Filtering on audience selects customer-facing chunks instead of relying on similarity alone. section 5.1

A storage abstraction keeps ingestion and search independent of the backend. Scope each inserted chunk to an index and document so updates can delete a document's chunks and index removal can delete the whole collection. Return insertion and deletion counts for verification, and authorize destructive operations against the index owner. section 5.4 section 5.4.1

Listing 5.16 VectorStore abstract base class: write operations
class VectorStore(ABC):
 
    @abstractmethod
    def insert(
        self,
        index_name: str,
        document_id: str,
        chunks: List[Chunk],
        embeddings: List[List[float]],
        metadata: Dict[str, str]
    ) -> int:    #A
        """Insert chunks with embeddings. Returns count inserted."""
        pass
 
    @abstractmethod
    def delete_by_document(
        self, index_name: str, document_id: str
    ) -> int:    #B
        """Delete all chunks for a document. Returns count deleted."""
        pass
 
    @abstractmethod
    def delete_index
➥(self, index_name: str) -> int:    #C
        """Delete all data for an index. Returns count deleted."""
        pass

The write interface defines insertion and document- or index-level deletion. section 5.4.1

The search interface accepts an index, query embedding, result limit, optional metadata filters, and score threshold. Its results carry chunk and document IDs, text, score, and metadata. Vector similarity works well for natural-language queries but can struggle with exact keywords and identifiers; hybrid search combines it with keyword matching. section 5.4.1 section 5.4.4

Listing 5.17 VectorStore abstract base class: search operation and result model
@dataclass
class SearchResult:
    chunk_id: str
    document_id: str
    text: str
    score: float    #A
    metadata: Dict[str, str]
 
class VectorStore(ABC):
    # ... write operations from Listing 5.16 ...
 
    @abstractmethod
    def search(
        self,
        index_name: str,
        query_embedding: List[float],
        top_k: int = 5,
        metadata_filters: Optional[Dict[str, str]] = None,
        score_threshold: Optional[float] = None
    ) -> List[SearchResult]:    #B
        """Find chunks most similar to the query embedding."""
        pass

The search contract makes filtering, thresholding, and source attribution explicit. section 5.4.1

Backend choice is an operational tradeoff. The pgvector extension adds vector types, similarity operators, and indexing to PostgreSQL, reusing existing backup and monitoring infrastructure. Purpose-built vector databases optimize for similarity search and can scale further, but self-hosting adds operational work; managed services can introduce vendor lock-in and usage-based costs. A consistent interface allows a backend to be replaced without changing the surrounding service. section 5.4.2 section 5.4

Table 5.1 Vector store backend comparison
 
pgvector
Pinecone
Weaviate
Elasticsearch
What it is
PostgreSQL extension
Fully managed cloud service
Open-source vector database
Distributed search engine with native vector support
Best for
Adding vector search with minimal infrastructure overhead
Fast time-to-production with fully managed infrastructure
Combining vector and keyword search in a single query
Adding semantic search alongside existing full-text search
Scale ceiling
~10M vectors before performance degrades
Billions of vectors
Hundreds of millions of vectors
Billions of vectors with proper cluster sizing
Self-hosting
Yes
Bring-your-own-cloud available on AWS, GCP and Azure
Yes
Yes
Key tradeoff
Requires PostgreSQL expertise for production workloads; no built-in hybrid search
Higher cost at scale; bring-your-own-cloud requires dedicated plan
More complex to operate than pgvector; typically requires Kubernetes for production at scale
Already a complex system to operate; adding vector workloads increases memory and compute requirements on top of existing search infrastructure

Compare backend choices by infrastructure, scale, and operational tradeoffs. section 5.4.2

How Designing AI Systems covers it

How deep it goes
Defines write and search interfaces, compares backends, and explains orchestration; defers a full multimodal retrieval implementation.
Examples
Uses customer-facing and internal reset guides to illustrate metadata filtering, and compares pgvector with purpose-built alternatives.
What you can do after
Apply it via pgvector behind a storage interface, with scoped writes, similarity search, filters, and deletion controls.
How it connects
Connects embeddings, document chunking, semantic search, metadata filtering, and hybrid search.

Also explained in

Compare how 34 books explain vector database →

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Designing AI Systems. The section links point to the parts of the book it draws on.