liveBook · Manning

Synthetic data

As explained in Build an Advanced RAG Application (From Scratch)

Synthetic data generation produces artificial evaluation data, such as questions and curated answers, from source documents.

Synthetic data generation creates artificial evaluation data from document datasets, especially diverse questions paired with curated answers for RAG testing. section 6.5.2

A practical evaluation workflow uses synthetic data to:

  • Generate comprehensive question-and-answer pairs from source documents. section 6.5.2
  • Use the generated pairs directly or adapt them through human review. section 6.5.2
  • Establish a baseline, then measure retrieval and generation changes against it. section 6.5.2

How Build an Advanced RAG Application (From Scratch) covers it

How deep it goes
Defines document-to-question-and-answer generation and human adaptation, but does not detail prompts or generation internals.
Examples
Generates diverse artificial questions and curated answers from a dataset of documents.
What it assumes
Assumes a dataset of source documents and teaches how to turn it into evaluation questions and curated answers.
What you can do after
Apply it via RAGAS to create reusable evaluation pairs and compare system changes against a baseline.
How it connects
Connects synthetic data with RAG evaluation, retrieval, generation, and human review.

Also explained in

Compare how 24 books explain synthetic data →

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Build an Advanced RAG Application (From Scratch). The section links point to the parts of the book it draws on.