Hybrid search combines lexical retrieval, which matches words and term-frequency signals, with semantic retrieval, which matches meaning through embeddings. The combination helps search systems find both exact terminology and relevant paraphrases. section 1.1.1
A hybrid search pipeline brings together:
- Full-text indexes for occurrence-based or keyword retrieval. section 1.1.1
- Vector indexes for semantic similarity over embedding representations. section 1.1.1
- A shared production database, such as PostgreSQL with pgvector, which supports exact and approximate similarity indexes alongside full-text search. section 10.3.5
Scaling a production search pipeline
Store document embeddings in PostgreSQL with pgvector and use its full-text indexes at the same time. This design can scale an NLP pipeline to millions of users and documents, while dimensionality limits may require PCA before storage. section 10.3.5
