Chunking divides large documents into smaller, coherent segments for retrieval. In an offline indexing pipeline, parsed and normalized text is split into chunks, converted into embeddings, and stored in a vector database. The chunk boundaries influence which evidence can later be retrieved for an LLM. section 3.2.1 section 3.2.3
Chunk size balances retrieval precision against contextual richness. Fragments that are too small can lose necessary context; chunks that are too large can return irrelevant material. The choice also affects output quality, latency, and cost. section 3.2.3
Keeping a warranty exception with its rule
A policy gives 24 months of coverage but limits batteries to 6 months. Splitting by sentence can separate the exception from the general rule. Indexing the entire 40-page policy keeps them together but retrieves many unrelated clauses. A chunk containing the complete warranty clause preserves the exception while remaining specific enough to rank accurately. section 3.2.3
Foundational strategies are a practical starting point for most RAG pipelines:
- Fixed-size splitting cuts text every N characters or tokens, but can break sentences or ideas. section 3.2.3
- Recursive splitting tries separators in priority order, such as paragraphs, sentences, and words; related ideas can still fall across boundaries. section 3.2.3
- Document-based splitting follows source structure, such as Markdown headings or code functions; it is less effective when structure is poor. section 3.2.3
Table 3.2 Advanced chunking strategies, with their typical use cases, trade-offs, and example documents
| Strategy | Best for | Trade-offs | Examples |
Semantic |
Dense narrative text where topical continuity matters. |
Higher cost and slower indexing. |
Research papers, textbooks, whitepapers. |
Hierarchical |
Very long documents with mixed query granularity. |
Increased storage and pipeline complexity. |
Manuals, regulations, large documentation sets. |
LLM-based |
Maximum semantic coherence when quality is critical. |
Very high indexing cost. |
Legal opinions, medical records. |
Agentic |
Mixed document collections requiring adaptive handling. |
Highest implementation complexity. |
Corporate policies, regulatory filings. |
Compare advanced strategies for semantic continuity, mixed query granularity, and adaptive handling, including their indexing and implementation costs. section 3.2.3
In production, chunking belongs to the knowledge-base lifecycle. Validate incoming documents, update changed index records to limit staleness, and evaluate candidate indexes against test queries before deployment. Canary or A/B releases can compare retrieval precision and generation groundedness on live traffic. section 9.3.5









