Chapter 8: Advanced indexing

Introduction

The chapter introduction explains why a basic RAG setup can produce poor answers and why improving it requires refining retrieval and application design.

• Limits of basic RAG: Explain why the stripped-down RAG setup used for learning may disappoint in practice, and why building a robust application takes design refinement rather than just connecting components.
  Why it matters: It explains why having the right documents in a vector store does not, by itself, ensure good answers.

Key terms: Retrieval-Augmented Generation (RAG), vector store, indexing, metadata, retrieval strategies

8.1 Improving RAG accuracy

This section frames RAG accuracy as something to improve by examining both content ingestion and question answering, step by step.

• Accuracy across RAG workflows: Explain why improving RAG accuracy calls for examining each step of both the content ingestion and question-answering workflows.
  Why it matters: This gives you a way to think about where RAG accuracy can be improved without assuming the problem lies in only one stage.

Key terms: RAG accuracy, content ingestion, question-answering (Q&A) workflows

8.1.1 Content ingestion stage

This subsection explains why simple content indexing can weaken RAG retrieval and identifies chunking and embeddings as ways to improve it.

• Ingestion-stage retrieval accuracy: Explain why retrieval accuracy depends on how content is indexed, and describe the two improvement areas identified here: embedding calculations and links between embeddings and related text chunks.
  Why it matters: A clear question alone cannot compensate for content that was indexed in a way that makes it hard to retrieve.
• Chunk size retrieval tradeoffs: Describe how chunk size affects retrieval for precise versus broad questions, and explain the stated purpose of adding embeddings based on distinct chunk features.
  Why it matters: The way content is divided and represented affects which kinds of questions can retrieve it.

Key terms: content ingestion, basic indexing, embedding calculations, text chunks, vector store, chunk size, overlap length, additional embeddings, chunk features

8.1.2 Question-answering stage

This section identifies common reasons a RAG question-answering stage gives weak answers and pairs each problem with a high-level remedy.

• Question formulation and retrieval: Distinguish an unclear question from a broad question that is ineffective for retrieval, and describe the remedy for each.
  Why it matters: Improving the question can improve the context retrieved for the answer.
• Structured content store access: Explain why relying only on a vector store can limit answers, and describe how routing and generated structured queries help access other data sources.
  Why it matters: Choosing and querying the right source can make relevant structured content available to the answer.
• Irrelevant result filtering: Explain why retrieved results may still need filtering before they are fed to the LLM.
  Why it matters: A good question does not guarantee that every retrieved result will help the LLM answer.
• Combined accuracy improvements: State why improving just one part of the RAG process may be insufficient and describe the proposed high-level response.
  Why it matters: Accuracy problems may call for several complementary improvements rather than one isolated fix.

Key terms: poor question formulation, ineffective question for retrieval, subquestions, vector stores, structured data sources, relational databases, graph databases, structured queries, routing, irrelevant search results

8.2 Advanced document indexing

This section explains how document indexing choices affect the relevance and accuracy of chunks retrieved for RAG answers.

• Chunk quality and splitting tradeoff: Explain why retrieved chunk quality matters to an LLM’s answers and how chunk size trades precise matches against broader context.
  Why it matters: The split size affects what useful evidence the LLM receives.
• Chunk indexing choices: Describe how the way chunks are indexed can help capture both fine details and broader context.
  Why it matters: Indexing affects which aspects of a chunk retrieval can find.
• Retrieval-time sentence expansion: Explain how surrounding sentences can add context to a small retrieved chunk.
  Why it matters: It offers a way to balance a precise match with enough context to use it.
• Indexing nonprose content: Describe why structured or semi-structured content needs specialized retrieval techniques for unstructured queries, and give an example from the section.
  Why it matters: Not all content being searched is ordinary prose.

Key terms: text chunks, splitting strategy, chunk overlap, document hierarchy, embedding strategy, embeddings, metadata, multiple indexes, sentence expansion, surrounding sentences

8.3 Splitting strategy

This section explains how chunk size affects RAG retrieval and answer context, and why it should match the kinds of questions users will ask.

• Chunk indexing and retrieval: Describe how documents become searchable during RAG ingestion: they are split into chunks, which are indexed with embeddings and sometimes metadata.
  Why it matters: The splitting decision affects the chunks that retrieval can find and pass to the LLM.
• Chunk size and question fit: Explain the tradeoff between small and large chunks, and justify choosing chunk size according to whether expected questions are detailed or broad.
  Why it matters: Choosing chunks that fit likely questions helps retrieval supply relevant material with enough context for accurate answers.

Key terms: RAG ingestion phase, chunks, embeddings, metadata, vector similarity searches, granular chunks, coarse chunks, chunk size, expected query type, retrieval performance

8.3.1 Splitting strategies

This section compares splitting documents by their natural hierarchy with splitting them into fixed-size chunks, including the trade-offs of each approach.

• Document splitting strategy trade-offs: Distinguish splitting by document hierarchy from splitting by absolute size, explaining when each is useful and what each can sacrifice. Describe why fixed chunk sizes need testing.
  Why it matters: The way you set chunk boundaries affects both how coherent and how consistently sized the resulting chunks are.

Key terms: splitting by document hierarchy, splitting by absolute size, chunk size

8.3.2 Factors to consider

This section identifies document type and search type as factors when choosing a document-splitting strategy.

• Document and search considerations: Explain why mixed-content documents favor chunks that keep related content together, and how chunk granularity and tagging affect metadata search.
  Why it matters: These considerations help you choose chunks that preserve useful context and support flexible retrieval.

Key terms: document type, document hierarchy, fixed-size approach, metadata search, chunk granularity, tags

8.3.3 Choosing the right strategy

This section explains that choosing a document-splitting strategy depends on the documents and the desired balance between hierarchy and chunk size.

• Splitting strategy tradeoffs: Explain why choosing a splitting strategy depends on the document structure and specific needs, and may take trial and error to balance document hierarchy against absolute size.
  Why it matters: This helps you choose a splitter for the documents you actually have rather than assuming one method always wins.

Key terms: document hierarchy, absolute size, splitting strategy

8.3.4 Splitting by HTML header

This section sets up granular HTML-header chunks and coarser chunks in separate Chroma collections so their retrieval results can be compared.

• HTML Header Splitting Strategy: Explain why an HTML page can be split at H1 and H2 tags to produce granular chunks, and distinguish that approach from using RecursiveCharacterTextSplitter for larger, coarser chunks.
  Why it matters: The splitting choice changes how much content each retrieved chunk contains.
• Chunk Retrieval Comparison: Describe how the section sets up a comparison of granular and coarse retrieval: load the same Cornwall HTML content, store the two kinds of chunks in separate Chroma collections, and search them with queries. Explain how adding other destination pages broadens the searches.
  Why it matters: Comparing searches over both collections helps choose a balance between detailed and general content retrieval.

Key terms: H1 and H2 tags, granular chunks, coarse chunks, HTMLSectionSplitter, RecursiveCharacterTextSplitter, ChromaDB collection, uk_destinations

8.4 Embedding strategy

This section introduces multi-vector indexing, explains its two-layer chunk structure, and recommends testing strategies against the structure of your text.

• Multi-vector chunk structure: Explain how storing multiple vectors per document supports retrieval, and distinguish the roles of retrieval chunks and synthesis chunks.
  Why it matters: This distinction explains why the chunks used to find content need not be the chunks given to the LLM.
• Strategy selection by testing: Justify testing different multi-vector retriever techniques for a particular use case.
  Why it matters: Testing helps you choose a strategy suited to your content rather than assuming one works best everywhere.

Key terms: multiple vectors per document, synthesis chunks, retrieval chunks, embeddings, multi-vector retriever techniques, structure of your text

8.4.1 Embedding child chunks with ParentDocumentRetriever

This section explains how ParentDocumentRetriever uses small child chunks for precise matching while returning larger parent chunks with enough context for an LLM answer.

• Chunk Size Tradeoff: Explain why using only large or only small chunks can hurt RAG answers: large chunks may retrieve imprecisely for detailed questions, while small chunks may omit needed context.
  Why it matters: This tradeoff explains why the section combines two chunk sizes.
• Parent Child Index Structure: Describe how content is divided into coarse parent chunks and smaller child chunks, with parents in a document store and children and their embeddings in a vector store. Explain that each child retains a reference to its parent.
  Why it matters: The child-to-parent link makes it possible to search at one granularity and supply context at another.
• Precise Matches With Parent Context: Trace how matching a detailed query against child chunks can retrieve a context-rich parent chunk, and contrast that result with directly returning a child chunk.
  Why it matters: It explains why a granular search result can still provide enough surrounding information for an answer.

Key terms: large chunks, small chunks, context, detail, ParentDocumentRetriever, parent chunks, child chunks, document store, vector store, embeddings

8.4.2 Embedding child chunks with MultiVectorRetriever

The section presents MultiVectorRetriever as a way to link embedded child chunks to larger parent chunks and compares its search results with direct search on child chunks.

• MultiVectorRetriever parent-child setup: Describe how this section sets up MultiVectorRetriever to embed child chunks and link them to larger parent chunks: define parent and child splitters, supply them to the retriever, and ingest the content.
  Why it matters: This setup lets retrieval use granular chunks while retaining larger chunks for synthesis.
• Parent context in search results: Distinguish the result retrieved through MultiVectorRetriever from a direct semantic search on child chunks, and explain why the larger parent chunk is useful.
  Why it matters: The comparison shows why retrieving a relevant small chunk alone may not provide enough context for a detailed answer.

Key terms: MultiVectorRetriever, parent splitters, child splitters, ingestion, parent chunks, child chunks, direct semantic search, synthesis

8.4.3 Embedding document summaries

This section shows how embedding summaries of coarse chunks can improve retrieval while still returning larger chunks for context.

• Summary embeddings for precision: Explain why embeddings of coarse chunks can be unfocused and how embedding a concise summary can improve retrieval.
  Why it matters: It explains why summarizing a chunk before embedding it can make a search more precise.
• Focused search with broad context: Distinguish what a direct search on summaries returns from what the MultiVectorRetriever returns, and explain why the latter result helps the LLM.
  Why it matters: It connects precise retrieval with the context the LLM needs to synthesize an answer.
• Summary ingestion and cost: Describe the section’s setup for generating summaries of coarse chunks and state why it can run more slowly than using child embeddings.
  Why it matters: The retrieval benefit comes with an LLM summarization step that takes additional time during ingestion.

Key terms: coarse chunks, summary embeddings, search precision, MultiVectorRetriever, direct semantic search, summaries, broader context, summarization chain, InMemoryByteStore, child embeddings

8.4.4 Embedding hypothetical questions

The section explains how embeddings of questions a document chunk could answer can improve retrieval when a user’s question does not closely match the chunk’s wording.

• Question embedding retrieval rationale: Explain why a relevant chunk can be missed by vector search and how embedding hypothetical questions for that chunk can improve its chances of being retrieved.
  Why it matters: It explains why indexing only a chunk’s own wording can miss an answer that is actually relevant.
• Hypothetical question indexing workflow: Describe the section’s workflow for generating hypothetical questions, storing them for a MultiVectorRetriever, and searching after ingestion.
  Why it matters: The retrieval benefit depends on connecting generated questions to the chunks they are intended to help find.

Key terms: hypothetical questions, embeddings, vector store, semantic similarity, MultiVectorRetriever, coarse chunks, structured output, hypothetical question embeddings

8.5 Granular chunk expansion

This section explains how chunk expansion preserves granular search while giving the LLM more surrounding context for complete answers.

• Granular chunk context tradeoff: Explain why small granular chunks can work well for detailed questions yet fall short when generating complete answers, and contrast their direct retrieval with expanded-chunk retrieval.
  Why it matters: It explains why finding a relevant passage is not always enough to answer well.
• Expanded chunk retrieval: Trace how a MultiVectorRetriever searches granular chunks but returns linked expanded chunks containing adjacent content.
  Why it matters: This separates precise retrieval from the broader context supplied for answer generation.

Key terms: granular chunks, chunk expansion, MultiVectorRetriever, vector store, document store, expanded chunks

8.6 Semi-structured content

This section explains how to index mixed text and tables so a search can retrieve the full content needed for an answer.

• Mixed content indexing and retrieval: Explain how to index documents containing both text and tables, and trace what happens when a search matches a table summary.
  Why it matters: A summary can make a table searchable without depriving the LLM of the full table when it answers.

Key terms: semi-structured content, MultiVectorRetriever, document store, vector store, table summary

8.7 Multimodal RAG

This section sketches how multimodal RAG uses an image summary to find and return a raw image for response generation.

• Image summary indexing: Describe how multimodal RAG prepares an image for retrieval: a multimodal LLM summarizes it, the summary is embedded, and those embeddings are linked to the raw image in a document store.
  Why it matters: This explains how an image can become searchable without treating the raw image as the stored search text.
• Image retrieval and response: Trace what happens when a user query matches an image summary’s embeddings, through retrieval of the raw image and its use by the multimodal LLM.
  Why it matters: It distinguishes what the system searches from what the model receives to answer.

Key terms: multimodal LLM, image summary, embeddings, document store, MultiVectorRetriever, raw image

8.8 Summary

This summary explains how advanced indexing can improve RAG retrieval by choosing suitable document splits, preserving context around matches, and preparing semi-structured content.