Retrieval-augmented generation (RAG) answers a question using relevant external information supplied to a language model at query time. It separates finding information from writing the response. An internal documentation assistant, for example, can answer questions about recently updated company procedures without retraining its LLM or placing the entire documentation collection in every prompt. section 12.2.2 section 12.0
Build the pipeline in two phases:
- At build time, ingest documents, split them into chunks, create embeddings, and index them in a vector store. An embedding model maps semantically similar wording to nearby vectors; FAISS can search those vectors. section 12.1.3 section 12.2.1
- At query time, embed the question, retrieve relevant chunks, place their text alongside the question in a prompt, and ask the LLM to generate an answer. Retrieve selectively rather than sending the whole collection. section 12.2.1 section 12.2.2
Listing 12.2 Manual RAG vs. LangChain’s declarative pipeline
def manual_rag_query(query: str):
try:
query_embedding = embedding_client.embed(query)
results = vector_db.similarity_search(query_embedding, k=5)
context = "\n".join([doc.content for doc in results])
prompt = f"Context: {context}\nQuestion: {query}\nAnswer:"
response = llm_client.generate(prompt)
return validate_response(response)
except Exception as e:
return handle_error(e)
from langchain.chains import RetrievalQA
from langchain.retrievers import VectorStoreRetriever
chain = RetrievalQA.from_chain_type(
llm=llm,
retriever=vector_store.as_retriever(),
return_source_documents=True
)
result = chain.invoke({"query": query})
Compare the explicit sequence—embed, search, assemble context, generate, validate—with a LangChain RetrievalQA chain that connects an LLM and retriever. section 12.1.3
Answering a key-rotation question
A question about rotating API keys can retrieve documentation even if it uses different wording, such as updating credentials. Put the retrieved text under a Context label, the user’s request under Question, and invoke the LLM for Answer. The labels distinguish retrieved facts from the request and encourage an answer grounded in the documentation. section 12.1.3 section 12.2.6
A working chain is not yet a dependable service. Track retrieval quality, prompts, and LLM behavior with tracing; version prompts, embeddings, retrieval settings, and model configuration independently. Budget tokens before generation: excessive retrieved context can exceed input limits or consume money without improving the answer. A zero-temperature setting improves consistency but does not make output perfectly reproducible, so evaluate answers semantically rather than relying only on exact matches. section 12.5 section 12.2.6
Treat both questions and retrieved content as safety boundaries. Questions influence which documents enter the prompt, creating opportunities for prompt injection or answers outside the assistant’s intended scope. Sanitize inputs and test adversarial cases, hallucinations, and guardrails before exposing a RAG service to users. section 13.2.2 section 2.1.7
