Chunk granularity controls how much text each independently processed or retrieved chunk contains. Smaller chunks focus on precise topics; larger chunks preserve broader context but reduce fine-grained focus. section 8.1.1 section 8.3
Choose granularity by matching the expected query and document characteristics:
- Use smaller chunks for detailed questions that need precise semantic matches, accepting that surrounding context may be missing. section 8.3
- Use larger chunks for broad questions that need background context, accepting less precise retrieval. section 8.3
- Split by hierarchy when preserving coherent topics or related text, tables, and images matters; use fixed character, token, sentence, or word sizes for consistency. section 8.3.1 section 8.3.2
- Test a range of fixed sizes because consistent chunk lengths can still split sentences and lose context. section 8.3.1
Parent and child chunks
A hybrid design stores larger parent chunks for answer context and embeds smaller child chunks for precise matching. In one configuration, a 3,000-character parent splitter and 500-character child splitter feed ParentDocumentRetriever; child matches retrieve the context-rich parent. section 8.4.1
Figure 8.3 Impact of chunk size on answer accuracy. Coarse chunks provide more context but less focus, and they are suitable for broader questions. Granular chunks provide less context but more focus, and they are suitable for more detailed questions.
Compare coarse chunks, which provide context for broad questions, with granular chunks, which provide focus for detailed questions. section 8.3
Listing 8.3 Parent and child splitters for coarse and granular chunks
parent_splitter = RecursiveCharacterTextSplitter(
chunk_size=3000) #1
child_splitter = RecursiveCharacterTextSplitter(
chunk_size=500) #2
child_chunks_collection = Chroma( #3
collection_name="uk_child_chunks",
embedding_function=OpenAIEmbeddings(openai_api_key=OPENAI_API_KEY),
)
child_chunks_collection.reset_collection() #4
doc_store = InMemoryStore() #5
parent_doc_retriever = ParentDocumentRetriever( #6
vectorstore=child_chunks_collection,
docstore=doc_store,
child_splitter=child_splitter,
parent_splitter=parent_splitter
)
Inspect a concrete LangChain configuration that separates parent and child splitter sizes and connects child embeddings to parent documents. section 8.4.1
