liveBook · Manning

Learning signal

As explained in Retrieval Augmented Generation, The Foundational Ideas

A supervisory signal is information derived from observed outcomes that guides a model's learning or parameter updates.

A learning signal is information derived from observed outcomes that guides model learning or parameter updates. In retrieval-augmented generation, reader performance can supervise a retriever even when no document-level relevance labels exist. section 6.2.1

A reader's probability of the correct answer supplies the observable outcome. Retrieved documents remain latent variables: documents that make the answer easier to generate, reflected by lower perplexity, receive stronger retriever scores. This feedback loop addresses irrelevant retrieval that a static retriever cannot correct. section 6.2.1

Retrieval feedback without relevance labels

A retriever supplies several documents to a reader. If one document helps the reader generate the correct answer, the training objective increases that document's retrieval score, without requiring a human-annotated query-document pair. section 6.2.1

How Retrieval Augmented Generation, The Foundational Ideas covers it

How deep it goes
Explains latent documents, answer probability, and perplexity-based feedback, without detailing implementation procedures.
Examples
Uses irrelevant retrieval, correct-answer probability, lower perplexity, and unlabeled query-document pairs as examples.
What it assumes
Assumes retriever-reader architecture while teaching latent variables through unrecorded document usefulness.
What you can do after
Diagnose retrieval failures by interpreting reader success as feedback for improving document ranking.
How it connects
Connects retrievers, readers, latent variables, probabilistic models, perplexity, and relevance labels.

Also explained in

Compare how 4 books explain learning signal →

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Retrieval Augmented Generation, The Foundational Ideas. The section links point to the parts of the book it draws on.