liveBook · Manning

Document loader

A document loader is a software component that reads content from a source and converts it into a standardized document representation for downstream processing.

How 3 Manning books explain it

Build Python Web Apps with Streamlit

A document loader is a component that reads source data and converts it into structured documents for downstream processing.

How it is used
Supports RAG ingestion by converting files into searchable structured documents.
How deep it goes
Explains loader selection and folder loading, but does not detail parsing internals for formats beyond text files.
Examples
Uses DirectoryLoader, TextLoader, a recursive text splitter, and an articles folder containing text files.
What you can do after
Build it from scratch by loading a folder of text files and preparing Document objects for vector-store ingestion.

Read how Build Python Web Apps with Streamlit explains document loader →

AI Agents and Applications

A data loader is a component that ingests source data into a form suitable for downstream language-model processing.

How it is used
Introduces external content into summarization and Q&A ingestion workflows as standardized Document objects.
How deep it goes
Defines the loader and traces its place in ingestion, without detailing a loader implementation.
Examples
Uses CSV ingestion and summarization of a Wikipedia page and local TXT, DOCX, and PDF files.
What you can do after
Apply it via LangChain by selecting a loader for a source and placing its Documents in a processing workflow.

Read how AI Agents and Applications explains document loader →

Effective Conversational AI

A document loader is a component that reads content from a configured source or file format and represents it as documents with text and metadata.

How it is used
Starts retrieval-augmented generation ingestion by converting diverse source files into documents for downstream processing.
How deep it goes
Defines loader responsibilities and names common implementations, but defers detailed parsing, chunking, and transformation internals.
Examples
Uses text files, CSV files, directories, PDFs, HTML, markdown, webpages, transcripts, and Microsoft Office documents.
What you can do after
Operate LangChain loaders to ingest configured files and hand structured documents to retrieval pipelines.

Read how Effective Conversational AI explains document loader →

Also covered in

What to know first

Related concepts

Questions about document loader

What is document loader?

A document loader is a software component that reads content from a source and converts it into a standardized document representation for downstream processing.

Which Manning books explain document loader?

Build Python Web Apps with Streamlit, AI Agents and Applications and Effective Conversational AI.

What should I know before learning document loader?

Manning books that teach document loader usually introduce retrieval-augmented generation and embedding first.

What are the parts of document loader?

Document loader includes CSVLoader, JSONLoader and PyPDFLoader.

These guides are generated from the text of Manning books by Manning's knowledge graph and link to the sections they draw on. The full books are available on liveBook and at manning.com.