liveBook · Manning

Structure-aware chunking

As explained in Designing AI Systems

Structure-aware chunking is the division of document text into chunks while preserving or using structural boundaries such as headings and sections.

Structure-aware chunking divides extracted document text using boundaries such as headings, paragraphs, and sections. Preserving those boundaries during parsing gives the chunking strategy context that rigid fixed-size splitting does not use. section 5.3.2 section 5.3.5

In an ingestion pipeline, format-specific parsers first extract text and structural information. The index’s configured strategy then produces chunks for embedding and storage alongside their text and metadata. The strategy is selected when the index is created and applied consistently to ingested documents. section 5.3.2 section 5.3.5

A chunk carries both searchable content and source context:

  • Text is embedded and returned during search. A heading identifies the source section when the parser extracted it; fixed-size chunking leaves this field empty. section 5.3.5
  • Start and end offsets support linking back to or highlighting the source passage. Per-chunk metadata can support more granular filtering, such as by section name. section 5.3.5
  • Document-level metadata also travels with each derived chunk and supports filtered search. section 5.3.2
Listing 5.8 Chunk dataclass
@dataclass
class Chunk:
    text: str    #A
    heading: Optional[str] = None    #B
    start_offset: int = 0    #C
    end_offset: int = 0    #C
    metadata: Dict[str, str] = field(
        default_factory=dict
    )    #D

The chunk data model makes the optional heading, offsets, and metadata explicit. section 5.3.5

A shared strategy interface accepts an extracted document and returns chunks; size and overlap arrive through the strategy constructor. This lets the pipeline swap strategies without changing downstream code. The interface does not itself specify how a structure-aware strategy splits oversized sections or applies overlap. section 5.3.5

Listing 5.9 Chunking strategy interface
class ChunkingStrategy(ABC):
    def __init__(
        self,
        chunk_size: int = 512,
        chunk_overlap: int = 50
    ):    #A
        self.chunk_size = chunk_size
        self.chunk_overlap = chunk_overlap
 
    @abstractmethod
    def chunk(
        self, document: ExtractedDocument
    ) -> List[Chunk]:    #B
        """Break a document into chunks for embedding and retrieval."""
        pass

The interface defines the common input, output, and configuration for chunking strategies. section 5.3.5

How Designing AI Systems covers it

How deep it goes
Defines chunk fields and a pluggable strategy interface but leaves the structure-aware splitting algorithm unspecified.
Examples
Illustrates chunk representation and strategy configuration with a Python dataclass and abstract interface.
What you can do after
Evaluate/understand how preserved headings and chunk metadata support source tracing and filtered retrieval.
How it connects
Connects document parsing, metadata, embeddings, vector stores, and filtered search.

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Designing AI Systems. The section links point to the parts of the book it draws on.