A retrieval system cannot search a PDF, CSV, webpage, or corporate file until its contents are ingested. A document loader reads a configured source and represents each extracted piece as a document containing text and metadata. section 6.4.3
Use loaders as the first stage of an ingestion pipeline:
- Choose a loader for the source, such as TextLoader, CSVLoader, DirectoryLoader, or an HTML loader. section 6.4.3
- Call the loader’s load method to produce documents with text and metadata. section 6.4.3
- Pass the loaded documents to transformers for splitting and structuring, then to retrievers for query-time retrieval. section 6.4.3
Choosing a loader
For a text file, configure TextLoader with its path; for a spreadsheet, use CSVLoader. For Microsoft Office-type documents, AzureAIDocumentIntelligenceLoader can extract content while retaining useful structure. section 6.4.3
from langchain_community.document_loaders import TextLoader
# Load text data from a file using TextLoader
loader = TextLoader("./your_data/YourText.txt")
document = loader.load()
Shows the minimal TextLoader configuration and load call; related listings cover CSV and Azure document loaders. section 6.4.3
