Retrieval-augmented generation (RAG) gives a language model relevant external information when it answers a question. It addresses stale knowledge and domain-specific gaps without retraining or changing model weights. A retriever finds passages; a generator receives those passages alongside the question and produces an answer. This is context engineering: changing what information the model sees at inference time, rather than what it learned during training. section 5.4 section 5.4.1
A typical pipeline has two phases:
- Prepare the collection: split documents into chunks, turn them into embeddings, and store those vectors in a searchable database. section 3.4 section 5.4.3
- Answer a question: embed the query, retrieve similar passages, optionally rerank or compress them, and pass the selected context and original question to the language model. section 3.4 section 5.4.3
Retrieval quality determines which facts the generator can use. Choose an embedding model and retrieval strategy suited to the collection; exact match, hybrid BM25-and-vector search, and reranking are possible approaches. Tune chunk size and the number of retrieved passages. Poor chunks, missing metadata, or ineffective reranking can cause retrieval to fail before generation begins. Semantic splitting is worth testing when fixed character chunks break up coherent text. section 5.4.3 section 5.4.2
A current-policy assistant
A legal assistant asked about recent policy changes can retrieve the relevant portions of current policy documents rather than rely on a model’s potentially outdated pretraining. A basic LangChain implementation loads a page, splits it into overlapping chunks, stores OpenAIEmbeddings in Chroma, and composes a retriever, prompt, language model, and output parser. The same design applies where answers need auditable sources, including compliance and financial research. Retrieval can reduce hallucinations and make answers easier to trace, but its quality depends on the passages found. section 5.4.1 section 5.4.2 section 5.4.3
Listing 5.23 Running a basic LangChain RAG pipeline
loader = WebBaseLoader( #1
web_paths=("https://lilianweng.github.io/posts/2023-06-23-agent/",),
bs_kwargs=dict(
parse_only=bs4.SoupStrainer(
class_=("post-content", "post-title", "post-header")
)
),
)
docs = loader.load()
text_splitter = RecursiveCharacterTextSplitter( #2
chunk_size=1000,
chunk_overlap=200
)
splits = text_splitter.split_documents(docs)
vectorstore = Chroma.from_documents(documents=splits, #3
embedding=OpenAIEmbeddings())
retriever = vectorstore.as_retriever()
prompt = hub.pull("rlm/rag-prompt") #4
def format_docs(docs): #5
return "\n\n".join(doc.page_content for doc in docs)
rag_chain = ( #6
{"context": retriever | format_docs, "question": RunnablePassthrough()}
| prompt
| LLM
| StrOutputParser()
)
rag_chain.invoke("What types of memory do exist in agents?") #7
The runnable pipeline shows how loading, chunking, embedding, retrieval, and prompt assembly fit together. section 5.4.2
The collection need not be limited to ordinary documents. Text extracted from video frames can be indexed alongside other knowledge sources for multimodal retrieval. The governing question remains the same: can the system retrieve the information needed for this particular answer? section 6.3.4 section 5.4.3
