What you will learn in Retrieval Augmented Generation, The Foundational Ideas
Principles for architecting reliable and verifiable AI
Chapter by chapter
Chapter 1: How RAG research prevents disasters
The introduction defines RAG, shows why access to a correct source does not guarantee a grounded answer, and frames reliability as an engineering problem.
What you learn in chapter 1 →Chapter 2: Revolutions in semantics, scale, and similarity
The chapter introduction explains how advances in semantic representation, scalable vector search, and sentence similarity converged to enable RAG, while identifying retrieval trade-offs that still matter.
What you learn in chapter 2 →Chapter 3: REALM: Birth of end-to-end trainable RAG
The introduction explains why retrieving similar documents is not enough to answer questions and presents REALM’s central shift toward training retrieval for usefulness to the reader.
What you learn in chapter 3 →Chapter 4: Retrieval-augmented generation for knowledge tasks
Introduces canonical RAG’s modular design, its evidence-weighting distinction from naive RAG, and the practical advantages that helped it spread.
What you learn in chapter 4 →Chapter 5: Fusion-in-Decoder for multi-document processing
The introduction explains why answers spread across retrieved documents are difficult for conventional RAG and presents FiD’s separation of independent encoding and fused decoding as a scalable way to synthesize them.
What you learn in chapter 5 →Chapter 6: Atlas: Few-shot learning with retrieval augmentation
Introduces Atlas as a way to achieve strong few-shot performance by aligning a retriever and reader around external knowledge rather than relying on ever-larger models.
What you learn in chapter 6 →Chapter 7: HyDE: Imagining the answer before you search
Introduces query-document mismatch and HyDE’s idea of searching with an imagined answer while grounding the final response in retrieved documents.
What you learn in chapter 7 →Chapter 8: RAG-Fusion: Multi-query retrieval enhancement
Introduces the retrieval coverage problem and explains how RAG-Fusion uses varied queries and rank-based result merging to give an answer generator broader context.
What you learn in chapter 8 →Chapter 9: Self-RAG: Retrieval with reflection and self-critique
This introduction explains why Self-RAG makes retrieval and answer checking adaptive, how reflection tokens make those decisions visible, and how its original training approach differs from common practice today.
What you learn in chapter 9 →Chapter 10: RAPTOR: Recursive abstractive processing for tree-organized retrieval
The introduction explains why flat retrieval loses context and how RAPTOR’s tree of chunks and summaries supports retrieval at different levels of detail.
What you learn in chapter 10 →Chapter 11: Correction, planning, and reasoning
This introduction explains why better retrieval alone cannot fix single-pass RAG and contrasts two ways to add control: correcting retrieval afterward and planning it beforehand.
What you learn in chapter 11 →Chapter 12: Graph-based RAG
This introduction explains when relationship-heavy questions call for graph-based retrieval and distinguishes the roles a graph can play in RAG.
What you learn in chapter 12 →Chapter 13: Context compression and pruning
This introduction explains why long retrieved context can hurt RAG answers and outlines ways to reduce or prioritize what the generator processes.
What you learn in chapter 13 →Chapter 14: Evaluating RAG systems
The introduction shows why RAG evaluation needs separate retrieval and answer measures, while cautioning that LLM-judge scores are most useful for diagnosing and comparing systems rather than certifying quality.
What you learn in chapter 14 →Chapter 15: Production RAG: Metrics, agentic systems, and continuous improvement
The introduction argues that production RAG performance depends heavily on how well the reader handles noisy retrieval, and distinguishes two ways readers respond to more context.
What you learn in chapter 15 →Concepts explained in this book
- retrieval-augmented generationRetrieval-augmented generation (RAG) supplies retrieved external information to a language model before it generates a response.
- similarity searchSimilarity search finds items whose representations are close to a query representation according to a similarity or distance function.
- RetrieverA retriever is a model or system component that searches an indexed collection and returns potentially relevant documents or passages.
- embedding-based retrievalA bi-encoder is a neural architecture that independently encodes two inputs into representations that can be compared afterward.
- fusion-in-decoderFused decoding generates an answer while jointly attending to representations of multiple independently encoded documents.
- independent encoding architectureAn independent encoding architecture encodes a query and each candidate document separately so their representations can be precomputed and compared efficiently.
- cross-attentionCross-attention is an attention mechanism in which one sequence selects information from the representations of another sequence.
- embeddingA dense vector is a vector whose coordinates are generally populated with informative numerical values.
- reflection tokenIsUse is a Self-RAG reflection token that signals the overall usefulness of a generated answer.
- self-ragSKILL-RAG is a retrieval-augmented generation technique that filters retrieved information according to the model's self-knowledge.
- agentic RAGAgentic RAG is a retrieval-augmented generation architecture in which runtime decisions control retrieval, reasoning, or tool actions based on system state and signals.
- fine-tuningFine-tuning is further training of a pretrained model on task-specific or domain-specific data.
- cross-encoderA cross-encoder jointly processes two input sequences through one encoder so that tokens from both sequences can interact directly during encoding.
- recallRecall is the proportion of relevant items that a retrieval or classification system successfully returns.
- end-to-end learningEnd-to-end training jointly optimizes connected system components by propagating learning signals through the complete input-to-output pipeline.
- retrieve-then-generate pipelineSingle-pass RAG is a retrieval-augmented generation architecture that performs one retrieval pass and then generates an answer without iterative planning or correction.
- fusion ragRAG-Fusion is a retrieval technique that generates multiple queries, retrieves documents for each, and fuses the ranked results before generation.
- LongLLMLinguaLongLLMLingua is a prompt-compression method that uses question-conditioned perplexity to remove less informative tokens while preserving information useful for downstream generation.
- encoder-decoder architectureAn encoder-decoder architecture transforms input into an encoded representation and generates output conditioned on that representation.
- CRAGThe Comprehensive RAG Benchmark is a benchmark for evaluating retrieval-augmented generation systems on question-answering performance.
- readerA reader model is a question-answering component that extracts or generates an answer from retrieved documents or passages.
- compactionContext compression reduces retrieved or prompt context while retaining information relevant to the query and generation task.
- marginalizationMarginalization is the mathematical operation of summing or integrating over unobserved alternatives to obtain a combined probability or result.
- compression ratioA compression ratio is the proportion or factor by which an input representation is reduced in size relative to the original.
- Word2VecWord2Vec is a neural language model that learns dense vector representations of words from their distributional contexts.
- lexical searchLexical search retrieves text by matching words or other surface forms rather than representing meaning directly.
- marginal likelihoodMarginal likelihood is the probability of an observed outcome after summing or averaging over possible latent variables or retrieved documents.
- retrieve tokenA retrieve token is a control token whose predicted occurrence signals that external information should be retrieved during generation.
- REALM++REALM++ is a method for improving REALM through changes to training scale, retrieval during evaluation, and supervision without changing its architecture.
- T5T5 is a text-to-text Transformer model family that frames diverse natural-language tasks as sequence-to-sequence text transformations.
- Sentence-BERTSentence-BERT is a transformer-based model architecture that produces semantically meaningful sentence embeddings for efficient similarity comparison.
- retrieval evaluatorA retrieval evaluator is a component that assesses retrieval results and can trigger searches for additional supporting sources.
- LevelRAGLevelRAG is a retrieval approach that decomposes a complex question into atomic queries, routes them to searchers, summarizes results, and iterates until verification passes.
- rerankingA reranker is a model or retrieval component that reorders candidate results according to estimated relevance.
- AtlasAtlas is a retrieval-augmented language model designed for knowledge-intensive natural-language tasks.
- reciprocal rank fusionRank fusion is a retrieval technique that combines ranked result lists from multiple searches into a single ranking.
- fidelityFaithfulness measures whether a generated response is supported by the retrieved context rather than introducing unsupported claims.
- generatorA generator is a model component that produces a natural-language answer conditioned on retrieved documents or passages.
- RagasRAGAS is a framework for evaluating retrieval-augmented generation systems with metrics including faithfulness, answer relevance, and context relevance.
- evaluation datasetAn evaluation set is a collection of examples used to measure system performance under specified evaluation conditions.
- retrieval rankingRanking is the ordering or filtering of retrieved candidates according to a relevance score, including ordering passages by relevance to a query.
- LLM-as-a-judgeAn LLM judge is a large language model configured to evaluate the quality or correctness of generated outputs.
- context relevanceContext relevance is a metric measuring how much retrieved context is useful for answering the question.
- hierarchical summarizationHierarchical summarization is summarization performed across multiple levels of organization, from granular items to broader aggregates.
- relevanceAnswer relevance is a metric measuring how well a generated answer addresses the user's question.
- groundingGrounding is conditioning a generated answer on retrieved evidence so that its claims can be checked against supporting passages.
- self-attentionSelf-attention is an attention mechanism that computes relationships among tokens by allowing each token to incorporate information from other tokens in the same sequence.
- Precision@kPrecision@k is the proportion of the top k retrieved results that are relevant to the query.
- knowledge-augmented encoderA knowledge-augmented encoder is an encoder that jointly processes an input and retrieved document text to predict an output.
- AttentionRAGAttentionRAG is a retrieval-context compression technique that uses a transformer’s attention to select relevant tokens and sentences before generation.
- document encoderA document encoder is a model component that transforms documents or passages into representations used for indexing and retrieval.
- RAPTORRAPTOR is a retrieval technique that recursively summarizes document chunks into a tree to support retrieval at multiple levels of abstraction.
- hybrid searchHybrid search combines dense and sparse retrieval methods to improve the range and quality of retrieved results.
- pruningPruning is the removal of less relevant branches or items from a candidate set during retrieval.
- query-side fine-tuningQuery-side fine-tuning adapts the query-processing component of a retrieval system without changing document representations in the corpus index.
- chunkingChunking is the division of source documents into smaller units for processing and retrieval.
- continuous improvementA closed-loop system observes an intermediate or resulting state and uses that evaluation to decide subsequent actions.
- contrastive perplexityContrastive perplexity is a query-conditioned relevance measure based on the reduction in a token's perplexity after the user question is added to its conditioning context.
- HyDEHyDE (Hypothetical Document Embeddings) is a retrieval technique that generates a hypothetical document or answer and embeds it to improve document retrieval.
- atomic queryA sub-query is an individual query produced by dividing a larger information need into smaller retrieval or reasoning steps.
- GraphRAGGraph RAG is a retrieval-augmented generation architecture that represents corpus relationships as a graph and retrieves evidence through graph structure.
- few-shot learningFew-shot prompting conditions a model with a small set of input-output examples that demonstrate the desired task or format.
- information retrievalRetrieval finds candidate documents or passages from a knowledge base in response to a query.
- latencyInference time is the time required for a trained model or system to produce an output for an input.
- hypothetical documentA hypothetical document is a synthetic passage generated to provide a document-like semantic representation of a user's information need.
- stale dataStaleness is the condition in which stored or indexed information no longer reflects the current underlying source.
- routingA query router is a system component that analyzes a query and directs it to an appropriate retrieval or processing path.
- knowledge boundaryA self-knowledge boundary is the limit distinguishing information a language model can reliably know internally from information requiring external retrieval.
- modular RAGModular RAG is a retrieval-augmented generation architecture composed of interchangeable specialized components that can be recombined or reconfigured.
- masked language modelingMasked language modeling is a self-supervised training objective in which a model predicts tokens hidden from an input sequence.
- maximum inner product searchMaximum inner product search finds database vectors having the largest inner products with a query vector.
- perplexity distillationPerplexity distillation is a training objective that converts reader-derived document utility scores into a target distribution and trains the retriever to match it.
- HippoRAGHippoRAG is a retrieval-augmented generation architecture that uses personalized PageRank over an entity graph to retrieve evidence for answering questions.
- LightRAGLightRAG is a retrieval-augmented generation architecture that combines graph-based query routing with retrieval over unstructured vector representations.
- soft clusteringSoft clustering assigns an item degrees of membership in multiple clusters instead of assigning it to exactly one cluster.
- Gaussian mixture modelA Gaussian mixture model represents a distribution as a weighted combination of Gaussian component distributions and can provide probabilistic membership in multiple clusters.
- multi-query retrievalMulti-query retrieval generates or uses multiple formulations of an information need to retrieve complementary result sets.
- architectural styleAn architectural pattern is a recurring high-level organization of software components and their interactions.
- BERTBERT is a bidirectional Transformer-based language model pretrained to represent text using contextual token representations.
- PPRPersonalized PageRank is a PageRank variant that repeatedly teleports a random walk to specified seed nodes, producing query-focused importance scores.
- retrieval indexA document index is a stored structure containing representations of documents that supports retrieval from a document collection.
- synthetic dataSynthetic data is artificially generated data produced to resemble or serve the role of real data.
- modularityModularity is a graph-partition quality measure comparing the observed density of within-community edges with a null-model expectation.
- RAG-TokenRAG-token is a retrieval-augmented generation model variant that can condition each generated token on a different retrieved document.
- alpha-ndcgα-nDCG is an intent-aware ranking metric that rewards relevant coverage of multiple query facets while reducing credit for redundancy.
- g-indexingG-Indexing extracts a graph, organizes it into hierarchical communities, and precomputes summaries for later retrieval.
- FAISSFAISS is a library for efficient similarity search and clustering of dense vectors, including at very large scale.
- tokenizerA tokenizer converts text into token identifiers and related tensors that a neural language model can process.
- T5ForConditionalGenerationT5ForConditionalGeneration is a Hugging Face Transformers model class that performs conditional text generation with a T5 encoder-decoder model.
- transformersHugging Face Transformers is a library providing pretrained transformer models and tools for natural-language and other machine-learning tasks.
- failure modeA failure mode is a specific way a system or component can behave incorrectly or fail to meet its requirements.
- zero-shot learningZero-shot retrieval is retrieval performed without relevance-labeled examples or task-specific fine-tuning of the retriever.
- query decompositionQuery decomposition is the technique of splitting a complex information need into smaller sub-queries that can be searched or answered separately.
- query expansionQuery expansion generates alternative query terms or formulations to improve the recall of relevant information.
- hallucinationA generated claim that contradicts the evidence or source context available to the generation system.
- reference-free evaluationReference-free RAG evaluation assesses retrieval-augmented generation outputs without requiring reference answers.
- High-Level SearcherThe High-Level Searcher is LevelRAG's planning component that decomposes a user query into atomic sub-queries, summarizes evidence, verifies answers, and issues supplemental queries.
- asynchronous MIPS refreshAsynchronous MIPS refresh is an index-maintenance strategy that rebuilds document embeddings and search indexes periodically while training continues on a stale index.
- ContrieverContriever is a dense retrieval model that learns representations for retrieving relevant documents without supervised retrieval labels.
- decompose-then-recomposeDecompose-then-recompose is a context-processing technique that splits retrieved documents into fine-grained segments, filters them, and recombines the useful text.
- knowledge baseThe internal knowledge base is the case study's repository of Mastercard support information supplied to the assistant.
- native pruningNative pruning is token or context pruning that derives importance scores directly from a model's internal attention weights.
- vocabulary mismatchVocabulary mismatch is a difference between the terms used in a query and the terms used in relevant source content.
- semantic gapA semantic gap is a discrepancy between the meaning or representation of an input and the meaning or representation of a relevant target.
- semantic flatteningSemantic flattening is the loss of document-level structure when a corpus is represented as independent, disjoint vector items.
- modularityA modular architecture organizes a system into independently developable, deployable, scalable, and replaceable components.
- performance-based trainingA performance-based training signal is feedback derived from a system's downstream task performance and used to optimize a component.
- vector databaseA vector database is a database system designed to store and retrieve vector representations using similarity search.
- pretrainingPretraining is the initial training of a model on broad data before adaptation to a specialized task.
- perspective shiftingPerspective shifting is a query-generation technique that formulates searches from different facets or interpretations of an information need.
- interpretabilityInterpretability is the degree to which a model's information sources or decision behavior can be inspected and understood.
- query generationA query generator is a component that transforms a user input into multiple alternative queries for information retrieval.
- product quantizationProduct quantization compresses vectors by splitting them into subvectors and replacing each subvector with the index of its nearest entry in a learned codebook.
- retrieval coverage problemThe coverage problem is the gap between information available to a retrieval system and all information needed to generate a correct and complete answer.
- document-to-document similarityA retrieval comparison that evaluates similarity between a generated document-like passage and candidate documents rather than directly comparing the query with documents.
- foundation modelA foundation model is a broadly trained model that can be adapted to multiple downstream tasks.
- inverse cloze taskThe Inverse Cloze Task trains a retriever to match a sentence with the passage from which that sentence was removed.
- salient span maskingSalient span masking is a masking strategy that hides information-bearing spans such as named entities or dates so retrieval affects prediction.
- traversalGraph traversal is the process of visiting nodes by following edges through a graph from one or more starting points.
- RAGGEDRAGGED is an evaluation framework for testing retrieval-augmented generation assumptions across retrievers, reader models, datasets, and retrieval conditions.
- surrogate modelA proxy model is an approximate model used to estimate another model's behavior or to provide computationally cheaper scores for an input.
- abstractive compressorAn abstractive compressor generates a shorter passage that preserves important information from source text rather than merely selecting source sentences.
- system promptA system prompt is a high-priority instruction context that establishes a language model's behavior and constraints.
- encoderAn encoder is a model component that transforms an input sequence into a numerical representation.
- UMAPUMAP (Uniform Manifold Approximation and Projection) is a nonlinear dimensionality-reduction technique that constructs a low-dimensional representation while preserving local and some global structure.
- intent-aware metricsAn intent-aware metric evaluates retrieval by rewarding coverage of distinct facets or subtopics of an information need.
- query rewritingQuery reformulation is the transformation of an existing search query into an alternative formulation intended to retrieve more suitable information.
- query encoderA query encoder is a model component that converts a search query into a representation suitable for matching indexed documents.
- cache-augmented generationA generation approach that augments retrieval with cached information or computation to improve efficiency and reduce inference cost.
- context engineeringThe deliberate selection, organization, and presentation of information in a model's input context.
- distributional hypothesisThe distributional hypothesis is the linguistic principle that words occurring in similar contexts tend to have similar meanings or semantic relationships.
- G-EvalG-Eval is an LLM-based evaluation technique that generates evaluation steps and assigns probability-weighted scores to outputs.
- RAG-SequenceRAG-Sequence is a RAG variant that conditions the entire generated sequence on a single retrieved document selected from the candidate set.
- tupleA triple is a structured statement consisting of a subject, predicate, and object.
- contrastive learningContrastive learning trains representations by bringing related examples closer and pushing unrelated examples farther apart in representation space.
- orchestration layerAn orchestration layer coordinates modules, interfaces, routing decisions, and execution across a larger software system.
- confidence intervalA 95% confidence interval is an estimated range constructed by a statistical procedure intended to contain the relevant population quantity with 95% confidence.
- sub-intentA sub-intent is a distinct constituent information need or facet implied by a broader user query.
- g-generationG-Generation is the final generation step that gives a language model the original query and merged dual-path retrieval context in one call.
- reasoning questionA reasoning question or query requires interpretation, comparison, inference, or logical combination beyond direct extraction of a stated fact.
- abstractive summarizationAbstractive synthesis generates a shorter novel representation by integrating and rewording information from source documents.
- open-domain taskOpen-domain question answering is the task of answering questions on broad topics by retrieving evidence from a large external collection.
- Kullback–Leibler divergenceKullback-Leibler divergence is a measure of how one probability distribution differs from a reference distribution.
- test suiteA test suite is a collection of tests designed to assess specified behaviors or failure conditions in a system.
- advanced ragAdvanced RAG is a retrieval-augmented generation architecture that optimizes retrieval, document processing, and post-retrieval components while retaining the basic pipeline.
- hierarchical softmaxHierarchical softmax computes vocabulary probabilities through paths in a binary tree instead of evaluating every vocabulary item directly.
- negative samplingNegative sampling trains a predictive model by contrasting observed positive examples with a sampled set of negative examples.
- iterative retrievalIterative retrieval is a retrieval strategy that performs multiple retrieval steps, using intermediate results or reasoning to formulate subsequent searches.
- decoder-only transformerA decoder-only model generates text from a single autoregressive sequence without a separate encoder for input representation.
- long-context RAGLong-context RAG is a retrieval-augmented generation pattern that concatenates many retrieved documents into a large model input context.
- retrieval-extractive pipelineA retrieval-extractive pipeline retrieves relevant passages and then extracts a literal answer span from each passage.
- extractive summarizationExtractive pruning reduces context by deleting selected material without generating replacement text.
- attention dilutionAttention dilution is the spreading of attention probability mass across more tokens as a sequence becomes longer.
- query misalignmentQuery misalignment is the failure of a multi-token query to produce a single clear attention focus because its tokens attend to different context regions.
- exact-match accuracyExact Match is a question-answering metric that counts an answer as correct only when it exactly matches the reference answer after normalization.
- adaptive retrievalAdaptive retrieval is a retrieval strategy that dynamically decides whether or when to retrieve information during generation.
- CORECORE is a retrieval-augmented generation compression framework that trains a compressor with reinforcement learning against downstream task rewards.
- kv-cache evictionKey-value cache eviction is the removal of stored transformer attention states during decoding to control memory use.
- causal language modelA causal language model predicts each token from preceding tokens and is trained for autoregressive sequence modeling.
- coarse document rankingCoarse document ranking orders whole documents by estimated relevance before finer-grained compression.
- dynamic token budgetingDynamic token budgeting allocates different compression budgets to items according to their estimated importance or rank.
- LLMLinguaLLMLingua is a prompt-compression framework and software library for reducing prompt length while preserving task-relevant information.
- LLMLingua-2LLMLingua-2 is a prompt-compression system that uses a distilled token classifier to remove less important tokens.
- synthesisSynthesis generates a novel answer by integrating information from multiple evidence sources rather than merely extracting text.
- g-retrievalG-Retrieval is a graph-based RAG stage that anchors a query to graph nodes and traverses relationships to gather evidence.
- Natural QuestionsNatural Questions is a dataset of naturally occurring questions paired with answers for training and evaluating question-answering systems.
- leaf nodeA leaf node is a terminal node in a tree that has no child nodes.
- RECOMPRECOMP is a retrieval-augmented generation framework that trains compressor models to optimize downstream task performance and selectively omit unhelpful retrieved context.
- community detectionCommunity detection is the algorithmic identification of densely connected groups of nodes in a graph.
- HNSWHNSW is a graph-based approximate nearest-neighbor indexing method that navigates layered connections between vectors.
- Inverted File IndexIVF is an inverted-file vector indexing method that partitions vectors into clusters and searches selected clusters.
- ivfpqIVF-PQ is a composite vector index that combines inverted-file clustering with product-quantized vector storage.
- adapter methodAdapter-based fine-tuning adapts a pretrained model by training added parameter modules while leaving the base model weights largely frozen.
- rag stability scoreThe RAG Stability Score measures how consistently retrieval-augmented generation performance remains near its optimal retrieval depth.
- cosine similarityCosine similarity is a metric that measures the angular similarity between two vectors.
- context windowContext length is the maximum amount of input text that a model can process in one input sequence.
- parametric knowledgeParametric memory is knowledge encoded in the learned parameters of a trained model rather than retrieved from an external source.
- parametric-only modelA parametric-only model is a model whose accessible knowledge is stored entirely in its learned parameters without consulting external information.
- visual retrievalA visual retriever is a retrieval component that finds relevant documents or pages from visual representations rather than text alone.
- decoderA decoder is a neural-network component that generates an output sequence from encoded representations and its previously generated tokens.
- BICThe Bayesian Information Criterion is a model-selection metric that balances likelihood-based goodness of fit against a penalty for model complexity.
- proxy perplexityProxy perplexity is a prompt-compression approach that uses a language model's perplexity as a proxy for token informativeness.
- beam searchBeam search is a heuristic decoding algorithm that retains a fixed number of highest-scoring partial sequences at each step.
- Leiden algorithmThe Leiden algorithm is a community-detection algorithm that iteratively optimizes a graph partition while refining communities into internally connected groups.
- evaluation triadThe evaluation triad is a three-dimensional evaluation framework comprising faithfulness, answer relevance, and context relevance.
- mean reciprocal rankMean reciprocal rank is the mean of the reciprocal ranks assigned to the first correct result across evaluation queries.
- hierarchical indexA hierarchical index organizes retrieval representations at multiple levels of abstraction, from detailed items to broader summaries.
- lost in the middleA language-model performance degradation caused by relevant information being placed amid distracting or competing context.
- precisionPrecision is the proportion of retrieved items that are relevant according to the evaluation criterion.
- missed the top rankMissed the top rank is a retrieval or ranking failure in which the most relevant result is not prioritized.
- [CLS] tokenThe [CLS] token is a special BERT input token whose final hidden state aggregates the input sequence for classification.
- transformer architectureA transformer model is a neural-network architecture based on attention mechanisms for processing sequences and modeling contextual relationships.
- summary nodeA summary node is a node in a hierarchical retrieval structure whose content summarizes source chunks or lower-level nodes.
- variable-resolution retrievalVariable-resolution retrieval retrieves information at different levels of detail, ranging from specific source segments to broad summaries.
- Continuous Bag of WordsCBOW is a Word2Vec architecture that predicts a center word from multiple surrounding context words.
- projection layerA projection layer is the intermediate layer of a Word2Vec network whose rows serve as learned word-vector representations.
- skip-gramSkip-gram is a Word2Vec architecture that predicts surrounding context words from a target word.
- bottleneck layerA dense bottleneck is a constrained continuous vector representation through which an encoder compresses input information.
- HydeRetrieverHydeRetriever is a LangChain retriever component that combines hypothetical-document embedding with vector-store retrieval behind a single query interface.
- HypotheticalDocumentEmbedderHypotheticalDocumentEmbedder is a LangChain component that generates hypothetical text and encodes it into an embedding for retrieval.
- lossy compressionA lossy compressor produces a smaller representation by discarding some information from the original input.
- normalized discounted cumulative gainNDCG@10 is normalized discounted cumulative gain computed over the ten highest-ranked retrieval results.
- pseudo-supervisionPseudo-supervision is training or retrieval guidance supplied by synthetic signals generated from an existing model rather than direct human labels.
- key-value cacheA decoder cache stores key and value attention states from previously processed tokens for reuse during autoregressive generation.
- model calibrationCalibration is the degree to which a model's confidence or categorical assessments agree with externally observed outcomes or labels.
- attributionThe association of generated claims or responses with the information sources supporting them.
- BARTBART is a denoising sequence-to-sequence Transformer model trained to reconstruct text from corrupted inputs.
- tensorA tensor is a multidimensional numerical array used to represent data and intermediate computations in machine-learning models.
- component-wise evaluationComponent-wise evaluation maps separate evaluation metrics to the system components responsible for the measured behavior.
- failure point mappingFailure point mapping links evaluation dimensions to specific categories of system failure.
- metrics-driven developmentMetrics-Driven Development is an iterative development approach that uses evaluation metric scores to guide system changes.
- collapsed treeCollapsed Tree is a retrieval approach that performs a single-shot vector search over a flattened tree representation rather than traversing its layers.
- in-context learningIn-context learning is a model adaptation method in which a model learns from information provided in its prompt without modifying its internal weights.
- asymmetric searchAsymmetric search retrieves documents or passages for queries whose length, structure, and communicative role differ substantially from the targets.
- joint probability distributionJoint probability is the probability that multiple events or variables take specified values together.
- marginal probabilityMarginal probability is the probability of an event obtained by summing joint probabilities over another variable.
- approximate nearest neighbor searchApproximate nearest-neighbor search efficiently finds vectors close to a query vector without exhaustively comparing every stored vector.
- multi-document synthesis taskMulti-document synthesis is the generation of an answer by integrating information from multiple source documents.
- RA-DITRA-DIT, or Retrieval-Augmented Dual Instruction Tuning, is a retrieval-augmented instruction-tuning method for training model components to cooperate.
- student modelA student model is a model trained to imitate or absorb behaviors provided by a teacher model, typically with lower deployment cost.
- distillationKnowledge distillation is a training technique in which a smaller student model learns to reproduce the behavior or outputs of a larger teacher model.
- ClusteringEngineClusteringEngine is the companion notebook component that combines dimensionality reduction, cluster selection, and probabilistic clustering for RAPTOR tree construction.
- DeepEvalDeepEval is a software framework for evaluating large language model applications and generating evaluation data.
- crag pipelineA CRAG pipeline is an orchestration workflow that retrieves documents, evaluates them, and branches to refinement or corrective retrieval.
- retrieval depthRetrieval depth is the configured number or extent of candidate results considered during retrieval.
- ColPaliColPali is a vision-language retrieval model that represents document pages visually for late-interaction search.
- multimodal embedding modelA multimodal embedder is a model that maps inputs from multiple modalities, such as text and images, into vector representations.
- page imageA page image is a rendered visual representation of an entire document page.
- text pipelineA text pipeline is a retrieval workflow that represents and searches documents using extracted textual content.
- proactive planningProactive planning is a retrieval strategy that structures an information need and plans retrieval steps before initial retrieval.
- extractive compressorAn extractive compressor is a compressor model that selects and combines useful sentences from retrieved documents without generating new wording.
- self-critiqueSelf-critique is a model technique in which generated content or intermediate decisions are evaluated by the model itself.
- treeAn entity tree is a hierarchical data structure whose nodes represent entities and their relationships across levels.
- eRAGeRAG is a retrieval-augmented generation evaluation method that uses per-document language-model judgments to estimate actual system performance.
- communityA graph community is a group of nodes identified as more strongly or densely connected to one another than to the rest of the graph.
- accuracyAccuracy is the proportion of evaluated system outputs judged correct according to a specified criterion.
- evaluationSystematic evaluation is a structured process for measuring system behavior against defined criteria and test conditions.
- benchmarkA benchmark is a standardized evaluation task, dataset, or reference result used to compare system performance.
- BM25BM25 is a probabilistic term-weighting and document-ranking function based on term frequency, inverse document frequency, and document-length normalization.
- source knowledgeExternal knowledge is information outside a model's learned parameters that can be supplied from an external source.
- fact-single questionA fact_single question is an evaluation question whose answer can be obtained as a single factual item from context.
- self-correctionSelf-monitoring and correction is a system capability that evaluates outputs and information quality and initiates corrective actions when problems are detected.
- peak-then-decline behaviorPeak-then-decline behavior is a performance pattern in which adding context improves results up to a point and then reduces them.
- response generationGeneration produces a natural-language response from a language model conditioned on retrieved evidence.
- tree depthMax depth is a recursion limit that bounds the number of hierarchical levels generated by a tree-building process.
- critic-to-generator distillationCritic-to-generator distillation is a training technique that transfers critique behavior from a stronger critic model into a generator through supervised traces.
- inference-time controlInference-time control is the adjustment of a model’s behavior during decoding by modifying how generated control signals affect generation.
- passive retrieval architecturePassive retrieval is a retrieval strategy that performs search without dynamically deciding whether retrieval is necessary.
- multilingual semantic searchMultilingual semantic search retrieves semantically related content across different languages using language-aligned representations.
- interaction informationInteraction information is an information-theoretic quantity that measures how the information supplied by one variable changes when another variable is known.
- perplexityAbsolute perplexity is a token-predictability measure computed from general language statistics without conditioning on the user's question or task.
- probability theoryProbability is a numerical measure representing the likelihood of an event or outcome.
- anchor vectorAn anchor vector is a numerical summary representation formed from important sentences selected from retrieved passages.
- cross-attention sparsificationCross-attention sparsification reduces the number of decoder cross-attention layers or operations to lower memory access costs.
- FiDOFiDO is a Fusion-in-Decoder architecture optimized by reducing cross-attention and reallocating parameters toward the decoder.
- graph neural networkA graph neural network is a neural network that learns representations from graph-structured data by aggregating information across connected nodes.
- KG-FiDKG-FiD is a Fusion-in-Decoder extension that uses graph-based passage refinement before decoding.
- MGFiDMGFiD is a Fusion-in-Decoder architecture that selects fine-grained evidence and represents key sentences with an anchor vector.
- rationale signalA rationale signal is information passed to a decoder that indicates how strongly passages support an answer.
- RFiDRFiD is a Fusion-in-Decoder architecture that guides decoding with scores identifying passages containing answer rationales.
- rag scalability coefficientThe RAG Scalability Coefficient measures how much additional retrieved context continues to improve model performance across retrieval depths.
- multi-hop reasoningMulti-hop reasoning is reasoning that connects evidence or inferences across multiple intermediate steps or sources.
- temporal reasoningTemporal reasoning is reasoning that relates events, facts, or entities according to their times and chronological relationships.
- regressionA regression is a deterioration in system behavior caused by a change that previously improved or preserved performance.
- reinforcement learningReinforcement learning is a machine-learning paradigm in which an agent learns a policy from reward signals produced by interactions with an environment.
- SimRAGSimRAG is a self-improving retrieval-augmented generation method that creates training questions from an unlabeled domain corpus and retains examples supported by retrieved context.
- QueryFusionRetrieverQueryFusionRetriever is a LlamaIndex component that generates multiple queries, retrieves results, and fuses the resulting rankings.
- BookRAGBookRAG is a retrieval method that maps queries and retrieval operations onto a document's inherent table-of-contents hierarchy.
- CFT-RAGCFT-RAG is a hierarchical retrieval method that combines an entity tree with a Cuckoo Filter for fast probabilistic entity membership testing.
- Cuckoo FilterA Cuckoo Filter is a probabilistic membership data structure that supports compact, fast approximate lookup.
- HiRAGHiRAG is a hierarchical retrieval method that applies Gaussian mixture clustering to an entity graph rather than to text chunks.
- intrinsic structureIntrinsic structure is the preexisting hierarchical organization explicitly provided by a document, such as chapters, sections, or numbered filing sections.
- LLM wikiThe LLM Wiki pattern compiles a corpus into a persistent, interlinked Markdown knowledge base that an agent reads to answer questions.
- PageIndexPageIndex is a vectorless retrieval approach that builds a hierarchy from a document's native structure and uses an LLM agent to select relevant pages.
- automated synthesisThe automated integration and summarization of information from retrieved sources into a coherent result.
- checkabilityThe property of an answer being verifiable by inspecting an identified supporting passage or source.
- confidenceA confidence score is a numerical indication of a system's estimated certainty in an answer or prediction.
- f-scoreF1 score is the harmonic mean of precision and recall.
- brute-force similarity searchBrute-force similarity search compares a query vector with every vector in a dataset to identify the most similar items.
- baselineA baseline is a reference system or result against which another system's performance is compared.
- position biasPosition bias is an evaluation bias in which a judge favors whichever option appears first in a comparison.
- damping factorA damping factor is a random-walk parameter controlling the balance between following graph edges and teleporting back to designated seed nodes.
- MapReduceMapReduce is a programming model and distributed computation pattern that independently maps input records and then reduces intermediate results into an aggregate output.
- root nodeA root node is the topmost node in a hierarchical tree that represents or summarizes the tree's overall content.
- teacher modelA teacher model is a larger or more capable model whose outputs or behavior provide supervision for training a student model.
- reflection patternA reflection pattern is an agentic control pattern in which a system evaluates its own output or intermediate results and revises them when needed.
- summarization-based retrievalSummarization as indexing is the creation of synthetic retrieval representations by summarizing groups of source content.
- asynchronous index refreshAn asynchronous index refresh is the process of rebuilding or updating a retrieval index during ongoing model training.
- non-parametric approachA non-parametric approach stores task knowledge in an external information source rather than exclusively in learned model parameters.
- verificationVerification is the structured assessment of whether gathered evidence sufficiently answers a question.
- element extractionElement extraction is the identification of entities and relationships in text to produce structured graph data.
- LangExtractLangExtract is a library for extracting structured entities and relationships from text using prompted language models and preserving source offsets.
- cachingCaching stores reusable computation results or retrieved information to avoid repeating expensive operations.
- ARESARES is an automated evaluation framework that trains specialized judges and applies statistical inference to evaluate retrieval-augmented generation systems.
- multilingual supportA multilingual language model understands and generates natural-language text across multiple languages.
- chain-of-thought promptingChain-of-thought reasoning is a reasoning technique that generates or follows intermediate steps between a problem and its answer.
- greedy algorithmGreedy decoding generates a sequence by selecting the single most probable next token at each step.
- ablation studyAn ablation study is an experiment that removes or disables system components to measure their individual contributions.
- sufficient contextSufficient context is the condition in which the retrieved set collectively contains the information needed to answer the query.
- external memoryInformation stored outside a model's learned parameters in an external, searchable collection that can be updated independently.
- separation of concernsSeparation of concerns is a design principle that keeps distinct responsibilities in separate system components.
- GPGPUGPU acceleration is the use of graphics processing units to execute computations faster or in greater parallelism than general-purpose processors.
- dimensionality reductionDimensionality reduction is the transformation of high-dimensional data into a lower-dimensional representation while retaining selected structure or information.
- gleaning loopA gleaning loop is an iterative extraction procedure in which a model reviews its previous output and adds entities or relationships it missed.
- autonomous query formulationAutonomous query formulation is the generation of search queries by a system from its current generation context, predicted information needs, or detected knowledge gaps.
- bounded autonomyBounded autonomy is an agentic system design in which autonomous behavior is restricted by explicit operational, budgetary, or depth limits.
- dynamic retrieval triggeringDynamic retrieval triggering is the runtime decision to obtain additional information when generation confidence, content, or task requirements indicate a need.
- hard negative exampleA hard negative is a candidate passage that appears plausible for a query but is labeled irrelevant during ranking training.
- chain-of-thoughtMulti-step reasoning coordination is the organization of subproblem solving, information gathering, and result synthesis across iterative reasoning cycles.
- query refinementQuery refinement is the transformation of an information need into a more effective retrieval query or sequence of sub-queries.
- RQ-RAGRQ-RAG is a retrieval technique that trains a language model to decide whether to retrieve, rewrite, decompose, or disambiguate a query.
- soft hydeSoft HyDE is a HyDE variant that generates a concise outline, keyword expansion, or bulleted pseudo-passage instead of a polished hypothetical document.
- robustnessNoise robustness is the ability of a system to maintain performance when inputs or retrieved evidence contain irrelevant or corrupted information.
- corpusA corpus is a collection of documents available as a body of text for retrieval or analysis.
- indexing pipelineAn indexing pipeline is an offline workflow that transforms source documents into stored representations suitable for retrieval.
- centroidA cluster centroid is the representative mean vector of the items assigned to a cluster.
- vectorA vector is an ordered list of numbers representing a position or direction in a mathematical space.
- inner productAn inner product is an operation that maps two vectors to a scalar expressing their algebraic alignment.
- K-means clusteringK-means is a clustering algorithm that partitions data points into a fixed number of clusters by assigning each point to one cluster.
- vector clusteringVector clustering partitions vectors into groups whose members are similar according to their positions in vector space.
- vector compressionVector compression reduces the memory required to store vector representations, usually by encoding them with fewer or lower-precision values.
- reader robustnessReader robustness is the ability of an answer-producing component to maintain answer quality despite variation or imperfections in retrieved context.
- closed-book settingA closed-book knowledge-task setting requires a system to answer without consulting external information during inference.
- span-corruption objectiveA span-corruption objective trains a model to reconstruct contiguous spans of tokens replaced by sentinel markers.
- unanswerable questionAn unanswerable question is an evaluation question whose answer is not supported by the system's knowledge base or supplied context.
- planningPlanning is an agentic workflow pattern in which a system constructs an action sequence before or during task execution.
- vector spaceA vector space is a mathematical space whose elements are vectors that support operations such as addition and scalar multiplication.
- latent variableA latent variable is an unobserved variable whose possible values are integrated or inferred when modeling observed outcomes.
- language modelA language model generates or evaluates sequences of natural-language tokens.
- sparse vectorA sparse vector is a vector in which most coordinates are zero or otherwise inactive.
- RankRAGRankRAG is a retrieval-augmented model fine-tuned to rerank passages and generate answers within a single model.
- T5-v1.1T5-v1.1 is a T5 model variant pretrained only on unlabeled text rather than supervised datasets.
- active correctionReactive correction is a retrieval pattern that audits an initial retrieval result and intervenes only when that result is inadequate.
- manifold learningManifold learning is a dimensionality-reduction approach that models high-dimensional observations as lying near a lower-dimensional manifold.
- hard clusteringHard clustering assigns each item exclusively to one cluster rather than allowing graded membership across clusters.
- learning signalA supervisory signal is information derived from observed outcomes that guides a model's learning or parameter updates.
- FLAREFLARE (Forward-Looking Active Retrieval Augmented Generation) is a method that triggers retrieval based on predicted future generation content.
- LangChainLangChain is a software framework for building applications that coordinate language models with prompts, data sources, tools, and workflows.
- vector indexA vector index is an indexed structure that stores vector representations and supports similarity-based retrieval.
- semantic similaritySemantic similarity is the degree to which two linguistic items have similar meanings or conceptual content.
- probability-weighted scoringProbability-weighted scoring aggregates evaluation judgments according to the model's assigned outcome probabilities.
- text chunkA chunk is a smaller contiguous segment of source text created for processing or retrieval.
- retrieval validationRetrieval validation is the assessment of retrieved evidence to determine whether it is sufficiently reliable for answering.
- local node profileA node profile is a textual description associated with a graph node that summarizes the entity's mentions across a corpus.
These guides are generated from the text of Manning books by Manning's knowledge graph and link to the sections they draw on. The full books are available on liveBook and at manning.com.
