A text chunk is a smaller piece of a larger text. In LangChain, a text splitter can turn one Document into a list of smaller Document objects. This lets an application process text that would be too large to send to a model at once. section 7.1.1 section 1.2.1
For a question-answering chatbot, chunking happens during ingestion: load content, split it, create an embedding for each chunk, and store the chunks and embeddings in a vector store. At query time, retrieval selects relevant chunks to supply as context. Smaller pieces also keep a search from matching only a large section with mixed relevance. section 6.1.3
Start with a searchable collection
For travel documents, create a RecursiveCharacterTextSplitter with chunk_size=500 and chunk_overlap=0. Pass loaded Documents to split_documents(), then add the resulting chunks to Chroma with an embedding model. This is a starting configuration, not a universally best size. section 7.2.1
Choose boundaries and size for the questions users will ask:
- Use smaller chunks for precise questions. They are more focused, but may omit context needed for a complete answer. section 8.3
- Use larger chunks for broad questions. They carry more context, but can obscure fine details during retrieval. section 8.3
- Consider structure as well as size. An HTMLSectionSplitter can split a travel page at H1 and H2 headings; a RecursiveCharacterTextSplitter can produce coarser chunks instead. section 8.3.4
Figure 8.3 Impact of chunk size on answer accuracy. Coarse chunks provide more context but less focus, and they are suitable for broader questions. Granular chunks provide less context but more focus, and they are suitable for more detailed questions.
Compare how coarse and granular chunks trade surrounding context for focus when answering different kinds of questions. section 8.3
When neither size works well alone, split large parent chunks into smaller child chunks. Search embeddings of the children, then return their context-rich parent chunks. ParentDocumentRetriever demonstrates this arrangement with separate document and vector stores. Chunk overlap and metadata are further choices to test when retrieval misses useful information. section 8.4.1 section 8.2
