A Document is a structured LangChain object containing a text fragment in pageContent and associated metadata. Loaders produce lists of Documents from source files, making parsed content ready for chunking, summarization, embeddings, or vector storage. section 7.1.3 section 6.2.1
Document ingestion typically detects a file type, selects a loader, parses the source, and splits the resulting Documents into smaller overlapping chunks. RecursiveCharacterTextSplitter preserves semantic structure while constraining chunk size for model context windows. section 7.1.3 section 6.2.1
Loading and chunking uploaded files
Use PDFLoader or DocxLoader according to the file extension, then pass the returned Documents to RecursiveCharacterTextSplitter. For example, a PDF such as aesops-fables.pdf becomes parsed Documents, which can be split with a 1,500-character chunk size and 200-character overlap before summarization. section 7.1.3
Listing 6.5 Splitting a text into a list of chunks for retrieval
const splitter = new RecursiveCharacterTextSplitter({
chunkSize: 100, #1
chunkOverlap: 20, #2
});
const output = await splitter.createDocuments([text]); #3
/* Example output structure:
[
Document {
pageContent: "Artificial Intelligence (AI) is intelligence",
metadata: { id: 2 }
},
Document {
...
},
...
]
*/
Inspect the JavaScript output showing each Document's pageContent and metadata after text splitting. section 6.2.1
