Document loader
A document loader is a software component that reads content from a source and converts it into a standardized document representation for downstream processing.
Build Python Web Apps with Streamlit
A document loader is a component that reads source data and converts it into structured documents for downstream processing.
- How it is used
- Supports RAG ingestion by converting files into searchable structured documents.
- How deep it goes
- Explains loader selection and folder loading, but does not detail parsing internals for formats beyond text files.
- Examples
- Uses DirectoryLoader, TextLoader, a recursive text splitter, and an articles folder containing text files.
- What you can do after
- Build it from scratch by loading a folder of text files and preparing Document objects for vector-store ingestion.
Read how Build Python Web Apps with Streamlit explains document loader →
AI Agents and Applications
A data loader is a component that ingests source data into a form suitable for downstream language-model processing.
- How it is used
- Introduces external content into summarization and Q&A ingestion workflows as standardized Document objects.
- How deep it goes
- Defines the loader and traces its place in ingestion, without detailing a loader implementation.
- Examples
- Uses CSV ingestion and summarization of a Wikipedia page and local TXT, DOCX, and PDF files.
- What you can do after
- Apply it via LangChain by selecting a loader for a source and placing its Documents in a processing workflow.
Read how AI Agents and Applications explains document loader →
Effective Conversational AI
A document loader is a component that reads content from a configured source or file format and represents it as documents with text and metadata.
- How it is used
- Starts retrieval-augmented generation ingestion by converting diverse source files into documents for downstream processing.
- How deep it goes
- Defines loader responsibilities and names common implementations, but defers detailed parsing, chunking, and transformation internals.
- Examples
- Uses text files, CSV files, directories, PDFs, HTML, markdown, webpages, transcripts, and Microsoft Office documents.
- What you can do after
- Operate LangChain loaders to ingest configured files and hand structured documents to retrieval pipelines.
Read how Effective Conversational AI explains document loader →
Also covered in
What to know first
- retrieval-augmented generationRetrieval-augmented generation is a technique that retrieves relevant information from external sources and supplies it as context to a generative model when producing an output.
- embeddingAn embedding is a numerical vector representation of an item in a feature space, encoding selected properties or relationships for computational tasks such as comparison or retrieval.
Related concepts
- Documentproduces · A document is a discrete information-bearing item or structured representation of content, often including text, fields, or associated metadata.
- PyPDFLoaderhas part · PyPDFLoader is a LangChain document-loader class that reads PDF files, extracts their contents, and produces document objects.
- vector databaseuses · A vector database is a data-management system that stores and indexes numerical vector representations and supports retrieval through similarity or distance searches.
- text chunkproduces · A text chunk is a bounded segment of a larger text treated as a unit for processing, storage, or retrieval.
Questions about document loader
What is document loader?
A document loader is a software component that reads content from a source and converts it into a standardized document representation for downstream processing.
Which Manning books explain document loader?
Build Python Web Apps with Streamlit, AI Agents and Applications and Effective Conversational AI.
What should I know before learning document loader?
Manning books that teach document loader usually introduce retrieval-augmented generation and embedding first.
What are the parts of document loader?
Document loader includes CSVLoader, JSONLoader and PyPDFLoader.
These guides are generated from the text of Manning books by Manning's knowledge graph and link to the sections they draw on. The full books are available on liveBook and at manning.com.


