A text chunk is a section of a larger document processed as its own unit. In dense vector search, chunking lets each section have a separate embedding, so a query can match a relevant passage rather than relying on one embedding for the entire document. section 13.2.2
Choose chunk boundaries and an indexing strategy together:
- Split at sentence, paragraph, or other conceptual boundaries. Overlap adjacent chunks when a split might remove context needed to understand a passage. section 13.2.2
- Store multiple embeddings in one document if the search engine supports multivalued vector fields. Otherwise, index each chunk separately and retain the original document ID. section 13.2.2
Chunks in retrieval-augmented generation
A RAG pipeline can split documents into sections and index an embedding for each one. When a prompt needs supporting information, it embeds a query, retrieves and ranks matching sections by vector similarity, and passes those sections with the prompt to the generative model. section 15.2.1
Chunk size is a trade-off. A large chunk may be too broad for one embedding to represent fully; a small chunk may omit essential context. Splitting a document into many independent chunks also increases the number of vectors to index and search, and can complicate ranking when several matches come from the same original document. Tune granularity for recall without ignoring context and search cost. section 13.2.2 section 15.2.1
