4 Embeddings and vector search
This chapter covers
- How embedding models power retrieval-augmented generation systems by mapping text into vector space
- Seeing why embedding quality determines whether the right documents are retrieved
- Choosing among commercial, open-source, and domain-specific embedding models based on your domain, constraints, and goals
- Designing hybrid and multistage retrieval pipelines that combine dense, sparse, and reranking components for better precision
- Building a hybrid retriever for an employee-policy chatbot
A major health-care company launched a chatbot designed to help patients navigate their insurance plans. It was built using retrieval-augmented generation (RAG), backed by a well-trained large language model (LLM) and connected to internal policy documents (figure 4.1). On paper, everything looked solid. The system had access to accurate information, and the model could generate fluent, helpful responses.
Figure 4.1 Document embedding in RAG systems
But shortly after launch, users began reporting problems. The chatbot frequently failed to answer questions that should have been easy to answer. When someone asked whether physical therapy after surgery was covered, the model returned a vague reply or an “I don’t know” response, even though the answer was clearly stated in the documents it had access to. In some cases, it gave outdated or incorrect information.