A retrieval system can find the right topic and still produce an incomplete answer. If its documents are stored as isolated chunks, it may retrieve a useful fact but lose the relationships needed to explain a larger story. An architectural style addresses this kind of problem through the organization of components and their interactions, not just a better prompt. section 10.0 section 1.4.2
Retrieval-augmented generation (RAG) is a durable architectural pattern for connecting a generative model to external knowledge. Its role is to supply relevant, verifiable context for a task, including information beyond the model’s knowledge or from private sources. Longer context windows, fine-tuning, and caching can improve a system without replacing that role. A long prompt containing entire document collections may instead obscure the facts that matter. section 1.3.4
A modern retrieval style coordinates four component roles:
- Semantic encoding represents queries and documents by meaning. section 2.4.1
- Similarity search finds candidate documents; ranking and retrieval return the most relevant results. section 2.4.1
- Scalable infrastructure keeps those operations usable across large knowledge bases. section 2.4.1
When flat retrieval is not enough
A search over scattered historical paragraphs might answer when an armistice was signed but struggle to explain how a political figure’s foreign policy evolved. RAPTOR changes the retrieval organization from a flat list of vectors to a tree built by recursively clustering and summarizing chunks. The choice addresses missing context and unhelpful retrieval granularity rather than treating every question as a single-fact lookup. section 10.0
Choose architectural changes against observed failures. Start with a simple retrieve-then-generate pipeline and an accuracy measure; add a component when the baseline exposes a failure it can address. Complexity alone is not evidence of improvement. section 1.1 section 1.4.2


