liveBook · Manning

Chunking

As explained in LLM Customization and Fine-Tuning

The process of dividing documents into smaller units for indexing or retrieval.

Chunking divides documents into smaller units for indexing and retrieval in a RAG pipeline. The choice matters: fixed windows can work for narrative documents but split structured content at the wrong boundaries. section 4.5.5

Choose boundaries to match the corpus, then measure retrieval quality before adding preprocessing complexity:

  • Use fixed windows as a starting point; the baseline uses 400-word windows with 50-word overlap. section 4.5.5
  • For structured content, split policy documents at numbered clauses, code repositories at function boundaries, and transcripts at speaker turns. section 4.5.5
  • Consider semantic chunking when topic boundaries matter; it is more flexible but adds preprocessing cost and non-determinism. section 4.5.5

How LLM Customization and Fine-Tuning covers it

How deep it goes
Compares fixed, structure-aware, and semantic boundaries, then recommends measuring retrieval quality before switching strategies.
Examples
Uses numbered policy clauses, code functions, transcript speaker turns, and 400-word windows with 50-word overlap.
What it assumes
Assumes a minimal RAG pipeline and retrieval-quality measurement as the basis for tuning chunk boundaries.
What you can do after
Apply a corpus-appropriate chunking strategy and change it when measured failures point to chunk boundaries.
How it connects
Connects chunking to retrieval-augmented generation, vector similarity, and retrieval quality.

Also explained in

Compare how 10 books explain chunking →

Related concepts

This explainer was generated by Manning's knowledge graph from the text of LLM Customization and Fine-Tuning. The section links point to the parts of the book it draws on.