liveBook · Manning

Synthetic data

As explained in Retrieval Augmented Generation, The Foundational Ideas

Synthetic data is artificially generated data produced to resemble or serve the role of real data.

Synthetic data is artificially generated data designed to resemble or serve the role of real data. In retrieval-augmented generation (RAG), it can supply domain-specific training signals when labeled examples are scarce. section 14.3.2 section 15.4.3

ARES generates query–passage–answer triples from in-domain passages, including positive examples and negatives such as unrelated passages or contradictory answers. It uses these examples to train specialized judges for context relevance, faithfulness, and answer relevance. section 14.3.2

SimRAG generates candidate answer spans and questions from unlabeled domain documents. A round-trip consistency check retains a pair only when its answer appears in the top-k passages retrieved for the generated question, filtering data that is not sufficiently grounded or retrievable. section 15.4.3

Synthetic data supports two distinct RAG workflows:

  • Train lightweight evaluation judges that score production outputs at scale. section 14.3.2
  • Adapt a model to domain-specific retrieval and answer generation without manually labeled QA pairs. section 15.4.3
  • Filter generated examples with retrieval and faithfulness checks before using them for continued fine-tuning. section 15.4.3

From unlabeled documents to domain adaptation

A SimRAG-style pipeline generates a question and answer from each document, retrieves passages for the question, and keeps the pair only if retrieval returns the source context. The filtered pairs are mixed with general retrieval-oriented training data for continued fine-tuning. section 15.4.3

Figure 14.4 The ARES three-stage evaluation pipeline.

The three-stage ARES pipeline shows how synthetic data feeds trained judges, which then support statistically grounded evaluation. section 14.3.2

How Retrieval Augmented Generation, The Foundational Ideas covers it

How deep it goes
Explains generation, positive and negative construction, retrieval-based filtering, and downstream fine-tuning, but not generator internals.
Examples
Uses query-passage-answer triples, contradictory answers, generated QA pairs, and top-k retrieval checks as concrete artifacts.
What it assumes
Assumes familiarity with RAG, retrieval, QA pairs, fine-tuning, and evaluation dimensions such as faithfulness and context relevance.
What you can do after
Build synthetic training sets, filter them with retrieval consistency, and apply them to judge training or domain fine-tuning.
How it connects
Connects synthetic data to RAG evaluation, faithfulness, context relevance, retrieval, fine-tuning, and prediction-powered inference.

Also explained in

Compare how 24 books explain synthetic data →

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Retrieval Augmented Generation, The Foundational Ideas. The section links point to the parts of the book it draws on.