liveBook · Manning

Text chunk

Also called document chunk

A text chunk is a bounded segment of a larger text treated as a unit for processing, storage, or retrieval.

How 24 Manning books explain it

Grokking AI Applications

A contiguous portion of text created by splitting a larger document into smaller units for processing or retrieval.

How it is used
Uses chunks as retrievable units of document knowledge for a RAG assistant.
How deep it goes
Explains ingestion and retrieval stages and discusses chunk-size, overlap, and result-count tuning.
Examples
Uses a restaurant PDF with opening hours, delivery information, and menu prices to build a question-answering assistant.
What you can do after
Apply it via Langflow to split document text and tune chunk size and overlap for retrieval.

Read how Grokking AI Applications explains text chunk →

AI Agents and Applications

A text splitter is a component that divides large text into smaller chunks or document objects.

How it is used
Uses chunks to fit processing limits and supply focused context for retrieval and summarization.
How deep it goes
Explains size, overlap, structural boundaries, retrieval trade-offs, and parent-child indexing through configured splitters and stores.
Examples
Splits travel articles for Chroma search and compares granular HTML sections with coarser character chunks.
What you can do after
Apply chunking via LangChain splitters and evaluate retrieval using different chunk sizes and parent-child retrieval.

Read how AI Agents and Applications explains text chunk →

Generative AI in Action

Chunking is the partitioning of data, especially text, into smaller units for processing, indexing, or retrieval.

How it is used
Uses chunking to make enterprise text retrievable within RAG context-window constraints.
How deep it goes
Compares boundary and size strategies, selection factors, and adaptive retrieval, with PDF extraction examples.
Examples
Illustrates conversational context with a mobile-provider chatbot and PDF extraction with PyPDF2.
What you can do after
Evaluate chunking strategies for content and query needs, including overlap, context limits, and PDF extraction constraints.

Read how Generative AI in Action explains text chunk →

Spring AI in Action

A document splitter divides a document into smaller chunks suitable for downstream processing.

How it is used
Supports RAG ingestion by preparing document chunks for embedding, vector-store storage, and later retrieval as prompt context.
How deep it goes
Explains token limits, sentence boundaries, reactive integration, and builder options, but leaves embedding and distance calculations to providers and vector stores.
Examples
Uses my-document.txt, uploaded game-rules documents, TokenTextSplitter settings, and a Board Game Buddy loading pipeline.
What you can do after
Build it from scratch by reading documents, configuring TokenTextSplitter, and sending the resulting chunks to a vector store.

Read how Spring AI in Action explains text chunk →

Fast Python: High performance techniques for large datasets

How it is used
Uses chunks to process files in bounded parts and batch work sent between processes.
How deep it goes
Develops iterator-based batching and chunked CSV-to-Parquet conversion, including result flattening and cross-chunk type consistency.
Examples
Uses a novel word-count benchmark and a taxi CSV-to-Parquet conversion.
What you can do after
Apply it via pandas and Arrow to convert CSV data to Parquet incrementally.

Read how Fast Python: High performance techniques for large datasets explains text chunk →

Build AI-Enhanced Web Apps

Chunking is the division of a document or text stream into smaller units for processing or analysis.

How it is used
Prepares documents for vector ingestion and summarization within LLM context limits.
How deep it goes
Explains recursive separators, size and overlap parameters, document splitting, and downstream processing trade-offs.
Examples
Uses PDF and DOCX summarization and a 1,500-character splitter with 200-character overlap.
What you can do after
Apply it via LangChain.js to split loaded documents for summarization or vector ingestion.

Read how Build AI-Enhanced Web Apps explains text chunk →

Designing AI Systems

A document chunk is a segment of an ingested document, with associated source text or metadata, stored as an independently searchable unit.

How it is used
Supports passage-level semantic search and grounds model context with retrieved document text.
How deep it goes
Defines chunk fields and a strategy interface, then implements fixed-size splitting and illustrates recursive splitting and overlap.
Examples
Illustrates overlap with a defective-item return sentence and recursive splitting with paragraph and line boundaries.
What you can do after
Builds a chunk model and pluggable fixed-size strategy for an ingestion pipeline.

Read how Designing AI Systems explains text chunk →

Essential GraphRAG

Text chunking is the process of splitting source text into smaller pieces for downstream processing and retrieval.

How it is used
Improves embedding specificity and retrieval relevance within vector-search and retrieval-augmented generation pipelines.
How deep it goes
Defines chunking, compares boundary strategies, and gives sliding-window parameters and Python implementation guidance.
Examples
Uses papers, books, whitespace splitting, 500-character windows, 1,000-word limits, and 40-character or word overlaps.
What you can do after
Apply it via Python by splitting, overlapping, measuring, and experimentally tuning chunks for retrieval or extraction.

Read how Essential GraphRAG explains text chunk →

Build an Advanced RAG Application (From Scratch)

A chunking strategy is a method for dividing source text or documents into smaller units for processing or retrieval.

How it is used
Supports RAG retrieval by turning long documents into embed-ready units that can return focused evidence for generation.
How deep it goes
Explains context-limit motivation and strategy trade-offs, then implements recursive splitting with separator fallbacks and overlap.
Examples
Demonstrates hotel dining and spa segments, then splits research-paper abstracts into 657 overlapping chunks.
What you can do after
Build it from scratch with a recursive Python splitter that prepares overlapping chunks for embedding and vector search.

Read how Build an Advanced RAG Application (From Scratch) explains text chunk →

MongoDB 8.0 in Action, Third Edition

A text chunk is a bounded, coherent segment of source text, such as a paragraph or section, treated as one unit for processing.

How it is used
Supplies retrievable source-document context for a retrieval-augmented generation response.
How deep it goes
Explains chunking within a high-level retrieval workflow but does not specify a segmentation algorithm or chunk-size rule.
Examples
Uses PDFs and Microsoft Word documents as sources, with paragraphs and sections as possible chunks.
What you can do after
Evaluate/understand how coherent source-text segments become retrievable context for a grounded response.

Read how MongoDB 8.0 in Action, Third Edition explains text chunk →

AI-Powered Search

Chunking is the process of dividing a document or other content into smaller sections for separate processing or retrieval.

How it is used
Uses chunks to retrieve document passages for dense vector search and retrieval-augmented generation.
How deep it goes
Explains chunk boundaries, overlap, indexing alternatives, and granularity trade-offs without specifying a chunking algorithm.
Examples
Describes sentence and paragraph chunks, overlapping chunks, and separate chunk documents carrying original document IDs.
What you can do after
Evaluate chunk granularity and indexing approaches against context preservation, recall, computational cost, and relevance.

Read how AI-Powered Search explains text chunk →

The Art of AI Product Development

Semantic chunking divides text into segments according to meaningful topics or semantic boundaries.

How it is used
Uses chunks to retrieve relevant document content for search and response generation.
How deep it goes
Compares fixed-size, sentence, paragraph, semantic, and hierarchical chunking, including size trade-offs, without specifying an implementation.
Examples
Uses guides, case studies, market reports, financial statistics, and research papers to illustrate chunking choices.
What you can do after
Evaluate chunking methods and sizes against document structure and users’ information needs.

Read how The Art of AI Product Development explains text chunk →

Clojure: The Essential Reference

Sequence chunking is an internal optimization that computes a group of sequence elements ahead of their individual demand.

How it is used
Uses chunks to improve sequential processing by working through groups of elements rather than strictly one at a time.
How deep it goes
Explains buffering and chunk construction, then demonstrates a chunked map; notes the implementation-detail status of the public API.
Examples
Builds a three-element chunk, implements a chunked map, and contrasts iterator-seq with enumeration-seq.
What you can do after
Build it from scratch with chunk-buffer, chunk-append, chunk, and chunk-cons to process a custom lazy sequence in groups.

Read how Clojure: The Essential Reference explains text chunk →

Effective Conversational AI

A coherent segment of text treated as an individual unit for embedding, indexing, or retrieval.

How it is used
Prepares document content for indexing and retrieval in a retrieval-augmented generation pipeline.
How deep it goes
Defines chunks as coherent units and mentions paragraph boundaries, but does not specify how to choose chunk sizes.
Examples
Uses paragraph-boundary splitting and discusses partitioning large documents while retaining headings, tables, and lists.
What you can do after
Evaluate chunking choices as part of preparing documents for more granular retrieval.

Read how Effective Conversational AI explains text chunk →

Machine Learning Platform Engineering

A text splitter is a component that divides text into smaller chunks according to specified or semantic boundary rules.

How it is used
Supports document retrieval by dividing source text into searchable segments before embedding and vector indexing.
How deep it goes
Demonstrates configured chunk size, overlap, and separator order, but defers empirical tuning and advanced boundary evaluation.
Examples
Uses recursively loaded Markdown, heading-aware separators, 1,000-character chunks, 200-character overlap, FAISS indexing, and a three-result query.
What you can do after
Build it from scratch with LangChain by loading documents, splitting them, indexing chunks in FAISS, and testing similarity retrieval.

Read how Machine Learning Platform Engineering explains text chunk →

Data Pipelines with Apache Airflow, Second Edition

A chunk is a smaller piece of source text created by dividing a larger document for retrieval and processing.

How it is used
Improves RAG retrieval and source tracing by splitting recipes before embedding and vector-database updates.
How deep it goes
Defines chunks, compares oversized and sentence-sized splits, and implements a two-part recipe split without specifying a general sizing algorithm.
Examples
Splits each recipe into named ingredient and instruction rows in a pandas DataFrame, then assigns recipe and chunk UUIDs.
What you can do after
Build it from scratch with Python and pandas by turning recipe files into source-identifiable chunks for downstream processing.

Read how Data Pipelines with Apache Airflow, Second Edition explains text chunk →

AI Agents in Action

Chunking is the technique of dividing a larger document or text collection into smaller units for processing or retrieval.

How it is used
Uses document chunks to supply relevant context for retrieval-augmented generation and agent knowledge.
How deep it goes
Demonstrates character- and token-based splitting with overlap and retrieval tests, while deferring a semantic splitting function.
Examples
Splits an HTML nursery-rhyme collection and tests retrieval with questions phrased using both similar and different words.
What you can do after
Apply chunking via LangChain to a document and test its effect on retrieval.

Read how AI Agents in Action explains text chunk →

Microsoft Azure in Action

Chunking is the division of text into smaller segments for processing by a language model or retrieval system.

How it is used
Manages text that exceeds what a language model can view at once.
How deep it goes
Defines chunking through a suitcase analogy but gives no splitting procedure or configuration.
Examples
Uses an oversized book or conversation as the case for splitting text into smaller pieces.
What you can do after
Evaluate/understand when splitting text may help keep an LLM request within its input limit.

Read how Microsoft Azure in Action explains text chunk →

Build an AI Agent (From Scratch)

Text chunking is the division of source text into smaller segments for storage, retrieval, or processing.

How it is used
Makes long documents searchable as smaller units for retrieval-augmented agent context.
How deep it goes
Explains size and overlap tradeoffs and implements fixed-length character slicing, without developing other chunking methods.
Examples
Uses a 283-character overlap test and a web-results search for quantum computing.
What you can do after
Build from scratch a fixed-length chunker and use its output in a vector search pipeline.

Read how Build an AI Agent (From Scratch) explains text chunk →

AI Agents in Action, Second Edition

Semantic chunking is the segmentation of source material into sections that preserve meaningful units of content.

How it is used
Supports document ingestion and retrieval by turning source material into searchable context for language-model answers.
How deep it goes
Explains token-limited implementation and semantic-coherence guidance, but does not detail advanced boundary algorithms.
Examples
Uses a movie script, a 200-token Python-style function, Chroma DB indexing, and retrieved context for question answering.
What you can do after
Build it from scratch by splitting documents, embedding chunks, indexing vectors, and grounding responses in retrieved context.

Read how AI Agents in Action, Second Edition explains text chunk →

LLM Customization and Fine-Tuning

A segment of a document that can be indexed or returned as an individual retrieval result.

How it is used
Uses chunks as searchable units that supply retrieved context to a RAG prompt.
How deep it goes
Defines chunk roles and provides a failure-diagnosis procedure without specifying chunking algorithms.
Examples
Illustrates semantic proximity with quarterly revenue, Q3 earnings, and lunch passages, and retrieval with a NumPy matrix and top-k sort.
What you can do after
Diagnoses wrong-chunk failures by distinguishing missing or split answers from ranking errors.

Read how LLM Customization and Fine-Tuning explains text chunk →

The Art of Code

Chunking is the grouping of detailed operations into meaningful higher-level units that can be understood as a whole.

How it is used
Improves code reading by exposing intent through manageable, named units.
How deep it goes
Explains cognitive grouping, demonstrates its memory effect, and applies it through named methods without pursuing implementation internals.
Examples
Uses letters versus words, an over-literal walking story, and a createUser method to demonstrate chunking.
What you can do after
Apply chunking by extracting coherent operations into clearly named methods that reveal intent and expose omissions.

Read how The Art of Code explains text chunk →

Building Reliable AI Systems

A segment of text produced by dividing a larger document for processing or retrieval.

How it is used
Prepares loaded documents for embedding and precise retrieval within a retrieval-augmented generation indexing pipeline.
How deep it goes
Describes splitting loaded data into manageable chunks, including paragraphs or sections, without detailing splitter configuration.
Examples
Uses paragraphs or sections as chunk boundaries when dividing loaded documents.
What you can do after
Apply text splitters to segment loaded documents into manageable units for embedding and retrieval.

Read how Building Reliable AI Systems explains text chunk →

Retrieval Augmented Generation, The Foundational Ideas

A chunk is a smaller contiguous segment of source text created for processing or retrieval.

How it is used
Uses contiguous text segments to refine retrieved context and connect raw evidence within graph-based retrieval.
How deep it goes
Explains segmentation, query rescoring, filtering, recomposition, and graph bridging, but does not specify chunk-size selection.
Examples
Uses an AML policy paragraph or transaction-limit table row and a path through entities, knowledge units, and chunks.
What you can do after
Apply chunking to isolate query-relevant evidence and preserve retrieval paths through sparse knowledge graphs.

Read how Retrieval Augmented Generation, The Foundational Ideas explains text chunk →

Also covered in

What to know first

Related concepts

Questions about text chunk

What is text chunk?

A text chunk is a bounded segment of a larger text treated as a unit for processing, storage, or retrieval.

Which Manning books explain text chunk?

Grokking AI Applications, AI Agents and Applications, Generative AI in Action, Spring AI in Action, Fast Python: High performance techniques for large datasets, Build AI-Enhanced Web Apps, Designing AI Systems, Essential GraphRAG and 16 more.

What should I know before learning text chunk?

Manning books that teach text chunk usually introduce language model, similarity search, embedding and token first.

What kinds of text chunk are there?

Kinds of text chunk include chunking.

What is text chunk also called?

Text chunk is also called document chunk.

These guides are generated from the text of Manning books by Manning's knowledge graph and link to the sections they draw on. The full books are available on liveBook and at manning.com.