RecursiveCharacterTextSplitter is a LangChain component for turning documents into size-bounded chunks suitable for model processing, embeddings, summarization, or vector storage. It begins with larger separators such as newlines and recursively falls back to smaller character separators, preserving semantic structure where possible. section 6.2.1 section 7.1.3
Configure the splitter around two primary controls and one processing method:
- Set chunkSize to the maximum number of characters in each chunk. section 6.2.1
- Set chunkOverlap to repeat characters between adjacent chunks, retaining context across boundaries. section 6.2.1 section 7.1.3
- Call createDocuments([text]) for Document instances, or splitDocuments(documents) for already loaded documents. section 6.2.1 section 7.1.3
Split text for retrieval
const splitter = new RecursiveCharacterTextSplitter({ chunkSize: 100, chunkOverlap: 20, }); const output = await splitter.createDocuments([text]); // output contains Document instances with pageContent and metadata. section 6.2.1
Prepare documents for summarization
const splitter = new RecursiveCharacterTextSplitter({ chunkSize: 1500, chunkOverlap: 200, }); const documentChunks = await splitter.splitDocuments(documents); section 7.1.3
Table 6.2 LangChain text splitters
| Name |
Splits |
Adds metadata? |
Description/use case |
| Recursive |
Specific character |
Yes |
Recursively splits text while keeping related pieces together; recommended for initial splitting |
| HTML |
HTML characters |
Yes |
Splits text based on HTML tags and structures |
| Markdown |
Markdown syntax |
Yes |
Splits text according to Markdown syntax |
| Code |
Code-specific characters |
Yes |
Splits code based on syntax specific to various programming languages |
| Token |
A token |
No |
Splits text on tokens, with various methods available for token measurement |
| Character |
A single user-defined character |
No |
Simple splitting based on a specified character |
Compare recursive, HTML, Markdown, code, token, and character splitters when document structure or measurement requirements call for an alternative. section 6.2.1
