liveBook · Manning

Similarity search

As explained in Designing AI Systems

Vector search retrieves items by comparing numerical vector representations of queries and candidate content.

Similarity search retrieves content by comparing a query vector with vectors representing candidate passages. In a knowledge service, it can connect a question about an opened laptop to a return-policy passage about electronics, even when their wording differs. section 5.1

During ingestion, documents are parsed into text and split into semantically coherent chunks. An embedding model converts each chunk into a dense vector, which is stored with the text and metadata. At query time, the question is embedded and compared with stored vectors; the top matches return with scores and source attribution for use as grounding context. section 5.1

Figure 5.1 The retrieval pipeline from document to search result. A source document enters the ingestion pipeline, where it is parsed into clean text and then broken into semantically coherent chunks. Each chunk is passed through an embedding model to produce a dense vector representation. The vectors, along with the original chunk text and metadata, are stored in a vector database. At query time, the user's question follows the same embedding process to produce a query vector, which is compared against stored vectors using similarity search. The top matching chunks are returned with their source attribution, relevance scores, and metadata, ready to be injected into the model's context as grounding information.

Trace the document and query paths through embedding, vector storage, and retrieval. section 5.1

A vector store interface separates storage from retrieval. Insert operations associate chunks and embeddings with an index and document; search accepts an index, query embedding, result limit, optional metadata filters, and an optional score threshold. Results carry chunk and document identifiers, text, score, and metadata. Document and index deletion support updates, but destructive operations should be restricted to authorized callers. section 5.4.1

Listing 5.17 VectorStore abstract base class: search operation and result model
@dataclass
class SearchResult:
    chunk_id: str
    document_id: str
    text: str
    score: float    #A
    metadata: Dict[str, str]
 
class VectorStore(ABC):
    # ... write operations from Listing 5.16 ...
 
    @abstractmethod
    def search(
        self,
        index_name: str,
        query_embedding: List[float],
        top_k: int = 5,
        metadata_filters: Optional[Dict[str, str]] = None,
        score_threshold: Optional[float] = None
    ) -> List[SearchResult]:    #B
        """Find chunks most similar to the query embedding."""
        pass

Inspect the search contract and result model. section 5.4.1

Backend choice affects operations. PostgreSQL with pgvector can reuse existing database infrastructure, while purpose-built vector databases can scale further. Self-hosting adds deployment and monitoring work; managed services can introduce vendor lock-in and usage-based costs. A replaceable vector store interface allows a backend to change without changing the service. section 5.4.2

When exact matches matter

A search for ERR-4012 needs the specific error code, not semantically related errors. Hybrid search runs vector and keyword searches in parallel and merges their results, retaining semantic matches while also finding precise identifiers. section 5.5

How Designing AI Systems covers it

How deep it goes
Traces document ingestion and query retrieval, defines a vector store interface, and implements a pgvector backend.
Examples
Uses laptop return policies to illustrate semantic matching and error codes to motivate exact keyword matches.
What you can do after
Apply similarity search via a Data Service interface to retrieve ranked passages and source identifiers for a model response.
How it connects
Connects embeddings and document chunking to vector databases, grounding, and hybrid search.

Also explained in

Compare how 37 books explain similarity search →

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Designing AI Systems. The section links point to the parts of the book it draws on.