A financial assistant fine-tuned on historical earnings transcripts can learn financial language yet still miss what happened on yesterday’s call. Retrieval-augmented generation (RAG) addresses that knowledge gap by supplying relevant documents when the model answers, without changing its weights. section 2.0 section 4.5
A RAG pipeline indexes documents, retrieves relevant chunks for a query, and places them alongside the question in a prompt. The model conditions its response on that context. Updating the corpus can make new information available without retraining; retrieval quality limits what the model can use. section 1.6.2 section 4.5
Start with a measured prompting baseline. Add RAG when errors point to missing or outdated facts, particularly when information changes frequently or answers need verifiable citations. Fine-tuning addresses a different problem: behavior such as applying conventions consistently. A production system can combine retrieval for knowledge with an adapter for behavior. section 1.7 section 1.6.2 section 4.5
Contract review
For a legal review system, retrieve current regulatory text at inference time and use fine-tuning to shape structured tables and the organization’s risk-rating taxonomy. The retrieved documents can change without repeating the behavior training. section 6.5
Evaluate and debug the pipeline in layers:
- Measure retrieval hit rate, answer faithfulness, and end-to-end accuracy separately; each points to a different part of the system. section 4.10
- If the right chunk is retrieved but ignored, instruct the model to answer only from context and admit when context is insufficient. Test whether it follows that instruction using deliberately wrong context. section 4.5.6
- If the wrong chunk is returned, inspect production failures: determine whether the answer was dropped or split during chunking, or whether an indexed answer was ranked too low. Revisit chunking, hybrid search, or reranking accordingly. section 4.5.6
- Guard against invented answers on retrieval gaps with an explicit uncertainty instruction and a downstream check. Monitor retrieval, prompt assembly, and inference latency, and cap retrieved context length. section 4.5.6
A minimal pipeline can use vector embeddings and NumPy retrieval. Production extensions include chunking strategies, reranking, query rewriting, and hybrid retrieval. Choose additions based on measured failures rather than assuming a larger pipeline will fix every gap. section 4.10 section 4.5.6













































