liveBook · Manning

Chunking

As explained in Context Engineering

Chunking is the division of large documents into smaller, coherent text segments for downstream processing and retrieval.

Chunking divides large documents into smaller, coherent segments for retrieval. In an offline indexing pipeline, parsed and normalized text is split into chunks, converted into embeddings, and stored in a vector database. The chunk boundaries influence which evidence can later be retrieved for an LLM. section 3.2.1 section 3.2.3

Chunk size balances retrieval precision against contextual richness. Fragments that are too small can lose necessary context; chunks that are too large can return irrelevant material. The choice also affects output quality, latency, and cost. section 3.2.3

Keeping a warranty exception with its rule

A policy gives 24 months of coverage but limits batteries to 6 months. Splitting by sentence can separate the exception from the general rule. Indexing the entire 40-page policy keeps them together but retrieves many unrelated clauses. A chunk containing the complete warranty clause preserves the exception while remaining specific enough to rank accurately. section 3.2.3

Foundational strategies are a practical starting point for most RAG pipelines:

  • Fixed-size splitting cuts text every N characters or tokens, but can break sentences or ideas. section 3.2.3
  • Recursive splitting tries separators in priority order, such as paragraphs, sentences, and words; related ideas can still fall across boundaries. section 3.2.3
  • Document-based splitting follows source structure, such as Markdown headings or code functions; it is less effective when structure is poor. section 3.2.3
Table 3.2 Advanced chunking strategies, with their typical use cases, trade-offs, and example documents
Strategy Best for Trade-offs Examples

Semantic

Dense narrative text where topical continuity matters.

Higher cost and slower indexing.

Research papers, textbooks, whitepapers.

Hierarchical

Very long documents with mixed query granularity.

Increased storage and pipeline complexity.

Manuals, regulations, large documentation sets.

LLM-based

Maximum semantic coherence when quality is critical.

Very high indexing cost.

Legal opinions, medical records.

Agentic

Mixed document collections requiring adaptive handling.

Highest implementation complexity.

Corporate policies, regulatory filings.

Compare advanced strategies for semantic continuity, mixed query granularity, and adaptive handling, including their indexing and implementation costs. section 3.2.3

In production, chunking belongs to the knowledge-base lifecycle. Validate incoming documents, update changed index records to limit staleness, and evaluate candidate indexes against test queries before deployment. Canary or A/B releases can compare retrieval precision and generation groundedness on live traffic. section 9.3.5

How Context Engineering covers it

How deep it goes
Compares foundational and advanced strategies, their use cases and trade-offs, and outlines index evaluation and deployment.
Examples
Illustrates boundary choices with a warranty policy whose battery exception must remain attached to the general coverage rule.
What you can do after
Evaluate chunking strategies for contextual completeness, retrieval precision, and indexing cost.
How it connects
Connects chunking to embeddings, vector databases, retrieval precision, and RAG operations.

Also explained in

Compare how 10 books explain chunking →

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Context Engineering. The section links point to the parts of the book it draws on.