Hierarchical chunking organizes long documents into nested levels, such as chapters and sections, before dividing them into smaller retrieval chunks. It preserves document structure while helping users locate either broad context or specific analysis. section 7.2.4
Choose the structure and chunk size according to the document and retrieval goal:
- Use it for annual reports, research papers, and other long documents whose chapters or sections carry meaningful context. section 7.2.4
- Favor shorter chunks for precise details, and longer chunks when users need background for open-ended drafting. section 7.2.4
- Pair the structure with contextual retrieval when isolated chunks lose essential information about the subject, process, or time period. section 7.2.4
