Recursive splitting preserves natural text boundaries while meeting chunk-size limits. It tries separators from coarse to fine, retaining pieces that fit and recursively splitting oversized pieces. section 5.3.5
Use it as a replaceable ingestion component:
- Implement it behind a common chunking-strategy interface that returns Chunk objects. section 5.3.5
- Try paragraph boundaries first, then line breaks or finer separators when a piece exceeds the configured size. section 5.3.5
- Register alternative strategies through the SDK without changing the surrounding ingestion pipeline. section 5.3.5
Oversized paragraph
A document first splits into four paragraphs. Three fit and become final chunks; the oversized fourth paragraph splits at line breaks into two fitting chunks. section 5.3.5
Figure 5.5 Recursive splitting in action. A document is first split at paragraph boundaries (\n\n), producing four pieces. Three fit within the chunk size and become final chunks. The fourth exceeds the limit, so the algorithm recurses and splits it at line breaks (\n), producing two pieces that both fit. The diagram shows the hierarchy of separators tried at each level, with the final chunks highlighted.

Shows the separator hierarchy and the recursive path from paragraphs to line breaks. section 5.3.5




