A Q&A system needs to find relevant passages even when a question uses different words from its documents. An embedding is a numerical vector representation of text that helps make this possible: similar meanings can be matched by comparing vectors rather than exact wording. Embeddings can represent words, sentences, or document chunks. section 1.1.1 section 6.2.1
To build a searchable knowledge base, load documents, split them into chunks, generate an embedding for each chunk, and store each vector with its original text. At query time, embed the question using the same model, search for nearby stored vectors, and put the retrieved text—not just the vectors—into the LLM prompt. A search for “feline animals,” for example, can find passages about cats, lions, and tigers without an exact word match. section 6.1.3 section 1.1.1
Figure 1.2 A Q&A engine implemented with RAG design. An LLM query engine stores domain-specific document information in a vector store. When an external system sends a query, it converts the natural language question into its embeddings (or vector) representation, retrieves the related documents from the vector store, and then gives the LLM the information it needs to craft a natural language response.
Follow the two paths in the Q&A diagram: document chunks and embeddings enter the vector store; an embedded question retrieves documents that supply context for an answer. section 1.1.1
Store embeddings in a SQL table
With PostgreSQL and pgvector, an embedding can live in a VECTOR column alongside the value it represents. For a searchable first_name field, add first_name_embedding VECTOR, then populate it with vectors generated either by an available database function or externally through an embedding API. Similarity search can then use those vectors instead of relying only on exact name matches. section 10.4.3
ALTER TABLE user ADD COLUMN first_name_embedding VECTOR
The ALTER TABLE statement shows the database-field version of the idea: add a VECTOR-typed column to hold embeddings for an existing field. section 10.4.3
A vector store is another storage option. LangChain’s Embeddings abstraction converts text to vectors, while a VectorStore keeps chunks and embeddings available for similarity search; some stores generate embeddings when documents are added. For example, a document-ingestion workflow can use RecursiveCharacterTextSplitter, OpenAIEmbeddings, and Chroma to split and store source documents. section 7.1.1 section 7.2.1
Embedding quality does not rescue poor chunking. A chunk that is too large can match imprecisely; one that is too small can omit context needed for an answer. ParentDocumentRetriever addresses this by searching embeddings of small child chunks and returning their larger parent chunks. Keep embedding dimensions consistent between ingestion and retrieval; changing models may require re-embedding the stored corpus. section 8.4.1 section 6.4
