Document loader
A document loader is a software component that reads content from a source and converts it into a standardized document representation for downstream processing.
Build Python Web Apps with Streamlit
A document loader is a component that reads source data and converts it into structured documents for downstream processing.
- How it is used
- Supports RAG ingestion by converting files into searchable structured documents.
- How deep it goes
- Explains loader selection and folder loading, but does not detail parsing internals for formats beyond text files.
- Examples
- Uses DirectoryLoader, TextLoader, a recursive text splitter, and an articles folder containing text files.
- What you can do after
- Build it from scratch by loading a folder of text files and preparing Document objects for vector-store ingestion.
Read how Build Python Web Apps with Streamlit explains document loader →
AI Agents and Applications
A data loader is a component that ingests source data into a form suitable for downstream language-model processing.
- How it is used
- Introduces external content into summarization and Q&A ingestion workflows as standardized Document objects.
- How deep it goes
- Defines the loader and traces its place in ingestion, without detailing a loader implementation.
- Examples
- Uses CSV ingestion and summarization of a Wikipedia page and local TXT, DOCX, and PDF files.
- What you can do after
- Apply it via LangChain by selecting a loader for a source and placing its Documents in a processing workflow.
Read how AI Agents and Applications explains document loader →
Effective Conversational AI
A document loader is a component that reads content from a configured source or file format and represents it as documents with text and metadata.
- How it is used
- Starts retrieval-augmented generation ingestion by converting diverse source files into documents for downstream processing.
- How deep it goes
- Defines loader responsibilities and names common implementations, but defers detailed parsing, chunking, and transformation internals.
- Examples
- Uses text files, CSV files, directories, PDFs, HTML, markdown, webpages, transcripts, and Microsoft Office documents.
- What you can do after
- Operate LangChain loaders to ingest configured files and hand structured documents to retrieval pipelines.
Read how Effective Conversational AI explains document loader →
Related concepts
- Documentproduces · A document is a discrete information-bearing item or structured representation of content, often including text, fields, or associated metadata.
- PyPDFLoaderhas part · PyPDFLoader is a LangChain document-loader class that reads PDF files, extracts their contents, and produces document objects.
- vector databaseuses · A vector database is a data-management system that stores and indexes numerical vector representations and supports retrieval through similarity or distance searches.
- text chunkproduces · A text chunk is a bounded segment of a larger text treated as a unit for processing, storage, or retrieval.
These guides are generated from the text of Manning books by Manning's knowledge graph and link to the sections they draw on. The full books are available on liveBook and at manning.com.


