Corpus
A corpus is an organized collection of documents, texts, or other language data assembled for processing, retrieval, analysis, training, or evaluation.
Software Testing with Generative AI
A corpus is a curated collection of documents or other information assembled for retrieval and analysis.
- How it is used
- Supplies searchable context that RAG retrieves and inserts into prompts for more accurate testing-oriented LLM responses.
- How deep it goes
- Explains corpus contents, relevance selection, context-window rationale, file loading, and cosine-distance matching.
- Examples
- Uses labeled HTML pages, user-story files, booking API requirements, and Java resource-folder loading as concrete corpus examples.
- What you can do after
- Build it from scratch by loading documents, matching a query with cosine distance, and placing the closest document into a prompt.
Read how Software Testing with Generative AI explains corpus →
Natural Language Processing in Action, Second Edition
A corpus is a structured collection of documents or language samples used for linguistic analysis or model development.
- How it is used
- Supports statistical pattern-finding across documents for linguistic analysis and language-model development.
- How deep it goes
- Defines document collection, indexing, counting, and vectorization, while deferring deeper language-model mechanics to Word2Vec.
- Examples
- Uses German and English greeting messages, document indexes, token-sorting analogies, and Python Counter outputs.
- What you can do after
- Apply it via word counts and document indexes to inspect recurring language patterns.
Read how Natural Language Processing in Action, Second Edition explains corpus →
Generative AI in Action
A corpus is a collection of documents or passages assembled for information retrieval or language processing.
- How it is used
- Supports retrieval-augmented generation by supplying documents or passages for retrieval and grounded response generation.
- How deep it goes
- Explains sparse and dense retrieval mechanisms, trade-offs, selection criteria, and production quality controls.
- Examples
- Uses proprietary enterprise documents, diverse sources, current information, internal policies, and industry regulations as corpus contents.
- What you can do after
- Operate/configure a retrieval corpus and choose sparse or dense retrieval according to query, domain, and resource requirements.
AI-Powered Search
A document corpus is a collection of documents assembled for linguistic analysis or information retrieval.
- How it is used
- Frames documents as a searchable collection and supplies corpus evidence for query interpretation, concept extraction, and autocomplete.
- How deep it goes
- Explains corpus structure, text-analysis stages, and practical extraction workflows, but does not develop a full knowledge graph.
- Examples
- Uses phrase lookup, behavioral co-occurrence, and an outdoors dataset processed with spaCy for frequent concept extraction.
- What you can do after
- Apply it via spaCy and search data to interpret queries, extract vocabulary, and filter low-frequency autocomplete candidates.
LLM Customization and Fine-Tuning
A body of text or other training data used to train a model or, here, to build the retrieval index for RAG.
- How it is used
- Supplies text chunks that an encoder converts into vectors for retrieval-index construction.
- How deep it goes
- Explains re-encoding chunks after encoder changes, but does not specify corpus preparation or chunking procedures.
- Examples
- Uses a NumPy flat index for tens of thousands of chunks and recommends other index layers at larger scales.
- What you can do after
- Operate retrieval indexing by re-encoding every corpus chunk when replacing the sentence-transformer encoder.
Read how LLM Customization and Fine-Tuning explains corpus →
Retrieval Augmented Generation, The Foundational Ideas
A corpus is a collection of documents available as a body of text for retrieval or analysis.
- How it is used
- Supplies the searchable knowledge base for retrieval-augmented generation over corporate documents, academic papers, or web content.
- How deep it goes
- Defines corpus contents and pipeline role, then connects growth to retrieval dilution, chunk size, retrieval depth, and monitoring.
- Examples
- Uses PDFs, office documents, web pages, corporate documents, academic papers, and web content as corpus examples.
- What you can do after
- Diagnose retrieval versus generation failures by logging retrieved context and monitoring RAGAS and RAGGED trends.
Read how Retrieval Augmented Generation, The Foundational Ideas explains corpus →
Also covered in
- Learn Go with Pocket-Sized Projects
- LLMs in Production
- Go in Action, Second Edition
- Data Pipelines with Apache Airflow, Second Edition
- Machines that ThinkIn early access (MEAP)
- LLM Evaluation and Alignment, The Foundational IdeasIn early access (MEAP)
- Coding with AI
What to know first
- language modelA language model is a computational model that represents patterns in linguistic sequences to assign probabilities, predict, generate, or evaluate language.
- vector spaceA vector space is a set equipped with vector addition and scalar multiplication over a field that satisfy the vector-space axioms.
- context windowA context window is a bounded span of sequence elements or contextual information that a model or processing operation can consider together.
- deep learningDeep learning is a machine-learning approach that uses neural networks with multiple computational layers to learn representations from data.
Related concepts
- JSONhas part · JSON is a language-independent text-based data-interchange format that represents structured values using objects, arrays, strings, numbers, booleans, and null.
- language modelused by · A language model is a computational model that represents patterns in linguistic sequences to assign probabilities, predict, generate, or evaluate language.
- retrieval-augmented generationused by · Retrieval-augmented generation is a technique that retrieves relevant information from external sources and supplies it as context to a generative model when producing an output.
- fuzzingused by · Fuzzing is an automated testing technique that supplies varied, malformed, unexpected, or mutated inputs to software to reveal failures, vulnerabilities, or other defects.
Questions about corpus
What is corpus?
A corpus is an organized collection of documents, texts, or other language data assembled for processing, retrieval, analysis, training, or evaluation.
Which Manning books explain corpus?
Software Testing with Generative AI, Natural Language Processing in Action, Second Edition, Generative AI in Action, AI-Powered Search, LLM Customization and Fine-Tuning and Retrieval Augmented Generation, The Foundational Ideas.
What should I know before learning corpus?
Manning books that teach corpus usually introduce language model, vector space, context window and deep learning first.
What kinds of corpus are there?
Kinds of corpus include Brown Corpus.
What are the parts of corpus?
Corpus includes document set and JSON.
What is corpus also called?
Corpus is also called corpora.
These guides are generated from the text of Manning books by Manning's knowledge graph and link to the sections they draw on. The full books are available on liveBook and at manning.com.





