Chunking divides documents into smaller pieces for embedding and retrieval. Granularity affects what a retrieval-augmented generation (RAG) system can find: small chunks can separate necessary context, while large chunks can dilute relevance with unrelated content. section 4.4.2 section 4.7
Policy eligibility
A policy passage names a leave program and states its eligibility requirements. At 100–200 tokens, a split may separate those facts, so a query retrieves the policy description but misses the criteria. A chunk over 500 tokens can keep them together, but unrelated policies may dilute its relevance score. section 4.4.2
Content-aware chunking uses document structure to keep related information together rather than applying one split everywhere. section 4.4.2
Choose boundaries to suit the content:
- For legal documents, identify section headers or numbered clauses and preserve complete concepts. section 4.4.2
- For narrative content, split on paragraph breaks and use a minimum length to limit fragmentation. section 4.4.2
- For code documentation, detect code fences, function signatures, and other delimiters so examples remain paired with explanations. section 4.4.2
When specialized parsing does not apply, a sliding window can overlap adjacent chunks. Label chunks with metadata such as section headings, page numbers, filenames, and publication dates to retain structured context and support filtering. section 4.4.2 section 4.7
def sliding_window_chunks(text, chunk_size=500, overlap=100):
"""Create overlapping chunks using a sliding window"""
tokens = text.split()
chunks = []
for i in range(0, len(tokens), chunk_size - overlap):
chunk = tokens[i:i + chunk_size] #1
if chunk: #2
chunks.append(" ".join(chunk))
return chunks
This Python fallback splits on whitespace and advances by chunk size minus overlap; its example uses a 500-word window and 100-word overlap. section 4.4.2









