liveBook · Manning

Nlp-based chunking

As explained in Designing AI Systems

Semantic chunking detects topic boundaries by comparing embeddings of consecutive sentences and starting chunks where semantic similarity drops sharply.

Semantic chunking splits documents at inferred topic boundaries rather than fixed token counts. It can produce useful retrieval chunks for unstructured content such as transcripts and email threads, but adds embedding cost during ingestion. section 5.3.5

To find boundaries:

The choice is an index-level configuration. Recursive splitting is the platform default for broadly varied content; structure-aware chunking is an alternative for well-structured documents. Fixed-token cuts can split sentences or code blocks in policy and engineering documents. section 5.3.5 section 10.5

Listing 10.8 Populating the knowledge base
# Ingest company documents (typically done as a setup step or scheduled job)
platform.data.create_index(
    name="company-knowledge",
    embedding_model="text-embedding-3-small",
    chunking_strategy="semantic",    #A
    metadata_schema={"department": "string", "doc_type": "string"})
 
# Batch ingest documents
for doc_path in company_docs:
    platform.data.ingest(
        index_name="company-knowledge",
        document_id=doc_path.stem,
        file_path=doc_path,
        metadata={"department": "engineering", "doc_type": "policy"})

Configure semantic chunking when creating a company knowledge index, then ingest documents with metadata. section 10.5

Once populated, the index supports retrieval that grounds answers in organizational knowledge. The additional ingestion cost buys topic-sensitive boundaries, not a guarantee that every numbered clause or code block remains intact. section 10.5 section 5.3.5

How Designing AI Systems covers it

How deep it goes
Explains sentence embedding and consecutive similarity comparisons, but leaves the sharp-drop threshold unspecified.
Examples
Uses transcripts, email threads, policy documents, and engineering documents to illustrate content and boundary concerns.
What you can do after
Operate/configure a knowledge index with the semantic chunking strategy through the platform data interface.
How it connects
Connects sentence embeddings and similarity to vector search and agentic RAG.

Also explained in

Compare how 2 books explain nlp-based chunking →

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Designing AI Systems. The section links point to the parts of the book it draws on.