liveBook · Manning

Hybrid search

As explained in Designing AI Systems

Hybrid retrieval combines multiple retrieval methods, such as semantic vector search and keyword search, into a unified result ranking.

Hybrid search combines semantic vector retrieval with keyword search into one ranked result list. It addresses a common production gap: vector similarity handles natural-language descriptions but can dilute exact identifiers, while keyword matching can miss relevant results expressed in different words. section 5.4.4 section 5.5

Identifiers and vocabulary gaps

A search for ERR-4012 benefits from a keyword path that finds the literal code. A search for “laptop return” benefits from a semantic path that can find a chunk about an “electronics refund” despite the different vocabulary. section 5.5

A retrieval service sends the same query down two paths: embed it for vector similarity and use the raw text for keyword matching. Each path returns a ranked list. Reciprocal Rank Fusion (RRF) combines positions in those lists rather than requiring the two scoring scales to be normalized. section 5.5.4 section 5.7

Figure 5.8 Hybrid search in the Data Service. A text query enters the search method and follows two parallel paths. The vector path embeds the query through the Model Service and runs a cosine similarity search against stored vectors. The keyword path converts the query into a tsquery and runs a full-text search with ts_rank scoring against the same chunks table. Both paths return ranked result lists, which Reciprocal Rank Fusion merges into a single ranked list by combining positional scores. The fused results are returned to the calling workflow.

Follow the two retrieval paths from the query through rank fusion to the combined result list. section 5.5.4

A practical service contract keeps both paths interchangeable:

  • Add keyword_search to the vector-store interface. It accepts raw query text, scopes the search to an index, applies metadata filters, and returns the same SearchResult type as vector search. A default NotImplementedError lets vector-only backends omit keyword support. section 5.5.1
  • Fetch more candidates than requested from each path before fusion. For top_k=5, fetching ten per path gives a chunk ranked sixth and seventh respectively a chance to appear near the top of the combined list, at the cost of scoring more candidates. section 5.5.4
  • Return the same result type for vector and hybrid search. A gRPC service and SDK can expose both operations without changing how workflows consume results; workflows still choose the index, result count, threshold, and context formatting. section 5.6

In PostgreSQL, a generated tsvector column can derive search data from chunk text, with a GIN index supporting full-text search. At query time, @@ matches a tsquery and ts_rank orders keyword results. This built-in ranking is simpler than BM25, but provides a dependency-light starting point for fusion. section 5.5.2

Listing 5.22 Adding full-text search to the chunks table
ALTER TABLE chunks
    ADD COLUMN search_vector tsvector
    GENERATED ALWAYS AS (
        to_tsvector('english', chunk_text)
    ) STORED;    #A
 
CREATE INDEX idx_chunks_search
    ON chunks USING gin (search_vector);    #B

Inspect the generated search column and GIN index used for the keyword path. section 5.5.2

How Designing AI Systems covers it

How deep it goes
Works through the retrieval interface, PostgreSQL keyword indexing, rank fusion, candidate oversampling, and service orchestration.
Examples
Uses an error-code query and a laptop-return vocabulary mismatch to contrast keyword and semantic retrieval.
What you can do after
Build it from scratch using a vector-store interface, PostgreSQL full-text search, and rank fusion in a retrieval service.
How it connects
Connects semantic search, full-text search, reciprocal rank fusion, metadata filtering, and retrieval-augmented generation.

Also explained in

Compare how 15 books explain hybrid search →

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Designing AI Systems. The section links point to the parts of the book it draws on.