liveBook · Manning

Corpus

Also called corpora

A corpus is an organized collection of documents, texts, or other language data assembled for processing, retrieval, analysis, training, or evaluation.

How 6 Manning books explain it

Software Testing with Generative AI

A corpus is a curated collection of documents or other information assembled for retrieval and analysis.

How it is used
Supplies searchable context that RAG retrieves and inserts into prompts for more accurate testing-oriented LLM responses.
How deep it goes
Explains corpus contents, relevance selection, context-window rationale, file loading, and cosine-distance matching.
Examples
Uses labeled HTML pages, user-story files, booking API requirements, and Java resource-folder loading as concrete corpus examples.
What you can do after
Build it from scratch by loading documents, matching a query with cosine distance, and placing the closest document into a prompt.

Read how Software Testing with Generative AI explains corpus →

Natural Language Processing in Action, Second Edition

A corpus is a structured collection of documents or language samples used for linguistic analysis or model development.

How it is used
Supports statistical pattern-finding across documents for linguistic analysis and language-model development.
How deep it goes
Defines document collection, indexing, counting, and vectorization, while deferring deeper language-model mechanics to Word2Vec.
Examples
Uses German and English greeting messages, document indexes, token-sorting analogies, and Python Counter outputs.
What you can do after
Apply it via word counts and document indexes to inspect recurring language patterns.

Read how Natural Language Processing in Action, Second Edition explains corpus →

Generative AI in Action

A corpus is a collection of documents or passages assembled for information retrieval or language processing.

How it is used
Supports retrieval-augmented generation by supplying documents or passages for retrieval and grounded response generation.
How deep it goes
Explains sparse and dense retrieval mechanisms, trade-offs, selection criteria, and production quality controls.
Examples
Uses proprietary enterprise documents, diverse sources, current information, internal policies, and industry regulations as corpus contents.
What you can do after
Operate/configure a retrieval corpus and choose sparse or dense retrieval according to query, domain, and resource requirements.

Read how Generative AI in Action explains corpus →

AI-Powered Search

A document corpus is a collection of documents assembled for linguistic analysis or information retrieval.

How it is used
Frames documents as a searchable collection and supplies corpus evidence for query interpretation, concept extraction, and autocomplete.
How deep it goes
Explains corpus structure, text-analysis stages, and practical extraction workflows, but does not develop a full knowledge graph.
Examples
Uses phrase lookup, behavioral co-occurrence, and an outdoors dataset processed with spaCy for frequent concept extraction.
What you can do after
Apply it via spaCy and search data to interpret queries, extract vocabulary, and filter low-frequency autocomplete candidates.

Read how AI-Powered Search explains corpus →

LLM Customization and Fine-Tuning

A body of text or other training data used to train a model or, here, to build the retrieval index for RAG.

How it is used
Supplies text chunks that an encoder converts into vectors for retrieval-index construction.
How deep it goes
Explains re-encoding chunks after encoder changes, but does not specify corpus preparation or chunking procedures.
Examples
Uses a NumPy flat index for tens of thousands of chunks and recommends other index layers at larger scales.
What you can do after
Operate retrieval indexing by re-encoding every corpus chunk when replacing the sentence-transformer encoder.

Read how LLM Customization and Fine-Tuning explains corpus →

Retrieval Augmented Generation, The Foundational Ideas

A corpus is a collection of documents available as a body of text for retrieval or analysis.

How it is used
Supplies the searchable knowledge base for retrieval-augmented generation over corporate documents, academic papers, or web content.
How deep it goes
Defines corpus contents and pipeline role, then connects growth to retrieval dilution, chunk size, retrieval depth, and monitoring.
Examples
Uses PDFs, office documents, web pages, corporate documents, academic papers, and web content as corpus examples.
What you can do after
Diagnose retrieval versus generation failures by logging retrieved context and monitoring RAGAS and RAGGED trends.

Read how Retrieval Augmented Generation, The Foundational Ideas explains corpus →

Also covered in

What to know first

Related concepts

Questions about corpus

What is corpus?

A corpus is an organized collection of documents, texts, or other language data assembled for processing, retrieval, analysis, training, or evaluation.

Which Manning books explain corpus?

Software Testing with Generative AI, Natural Language Processing in Action, Second Edition, Generative AI in Action, AI-Powered Search, LLM Customization and Fine-Tuning and Retrieval Augmented Generation, The Foundational Ideas.

What should I know before learning corpus?

Manning books that teach corpus usually introduce language model, vector space, context window and deep learning first.

What kinds of corpus are there?

Kinds of corpus include Brown Corpus.

What are the parts of corpus?

Corpus includes document set and JSON.

What is corpus also called?

Corpus is also called corpora.

These guides are generated from the text of Manning books by Manning's knowledge graph and link to the sections they draw on. The full books are available on liveBook and at manning.com.