A large language model (LLM) generates text, but a production application needs more than a model call. In a documentation assistant, the LLM answers a user’s question using retrieved company documents. Unlike an embedding model, which converts text into vectors for search, the LLM generates the response. section 12.0 section 12.1.3
Build the application around the model:
- Ingest and embed documents, then store their vectors for semantic search. section 12.1.3 section 12.2.1
- At query time, embed the question, retrieve relevant documents, add them to the prompt, and generate an answer with the LLM. LangChain can connect these steps into a retrieval pipeline. section 12.1.3 section 12.2.1
- Test prompts and observe each step. The same question can produce different answers, so a single successful response is not enough to establish reliable behavior. section 12.0 section 12.2.1
Listing 12.2 Manual RAG vs. LangChain’s declarative pipeline
def manual_rag_query(query: str):
try:
query_embedding = embedding_client.embed(query)
results = vector_db.similarity_search(query_embedding, k=5)
context = "\n".join([doc.content for doc in results])
prompt = f"Context: {context}\nQuestion: {query}\nAnswer:"
response = llm_client.generate(prompt)
return validate_response(response)
except Exception as e:
return handle_error(e)
from langchain.chains import RetrievalQA
from langchain.retrievers import VectorStoreRetriever
chain = RetrievalQA.from_chain_type(
llm=llm,
retriever=vector_store.as_retriever(),
return_source_documents=True
)
result = chain.invoke({"query": query})
Compare the explicit sequence—embed, search, assemble context, generate, validate—with a LangChain RetrievalQA pipeline. The shorter pipeline still depends on the same components. section 12.1.3
Choose a model for the workload
A documentation question about an API endpoint need not use the same model as a request to synthesize an onboarding guide from security documents. Compare response quality against token cost and deployment cost: a lower-cost hosted model may handle routine questions, while an open-weight model also requires infrastructure investment. section 13.4.1
Treat the LLM as one component of an observable ML system. Track behavior and spending, and retain the deployment and monitoring practices used for other models; generative output adds testing, debugging, and safety concerns rather than replacing those practices. section 12.0 section 12.2.1
