liveBook · Manning

BART

As explained in Retrieval Augmented Generation, The Foundational Ideas

BART is a denoising sequence-to-sequence Transformer model trained to reconstruct text from corrupted inputs.

BART is a denoising sequence-to-sequence Transformer that combines a bidirectional encoder with an autoregressive decoder. It reconstructs corrupted text during pretraining, learning to produce fluent, coherent language rather than merely extract spans. section 4.1.2

BART’s design supports generation from retrieved information:

  • The encoder reads context bidirectionally, like BERT, while the decoder generates text token by token, like GPT. section 4.1.2
  • A retrieved passage can be concatenated with a question and supplied as ordinary model input; changing the passage changes the generated answer without changing model weights. section 4.1.2
  • In RAG, BART serves as the generator alongside a separately pretrained DPR retriever, forming a modular retrieve-then-generate pipeline. section 4.1
  • Unlike extractive systems, BART can synthesize an answer whose wording does not appear verbatim in any one retrieved document. section 4.5

Retrieved question answering

A RAG system places a retrieved passage before the user’s question and feeds the combined text to BART. BART interprets the context and generates a fluent answer, even when the answer requires combining facts rather than copying one span. section 4.1.2 section 4.5

Figure 4.1 Compares the high-level architecture of REALM and canonical RAG, showing that RAG connects a pretrained retriever to a pretrained encoder-decoder generator while REALM jointly trains retrieval and the language model in a single, integrated system.

Compare BART-based RAG’s modular retriever-generator architecture with REALM’s jointly trained retrieval and language-model system. section 4.1

How Retrieval Augmented Generation, The Foundational Ideas covers it

How deep it goes
Explains encoder-decoder roles, denoising pretraining, and retrieved-input concatenation, but stops short of training or decoding internals.
Examples
Uses retrieved passages concatenated with questions and open-domain answers requiring synthesis across documents.
What it assumes
Assumes familiarity with Transformer language models and retrieval, while contrasting BERT’s understanding role with GPT-style generation.
What you can do after
Evaluate/understand when an encoder-decoder generator can synthesize retrieved evidence instead of extracting verbatim spans.
How it connects
Connects BART with Transformers, BERT, GPT, DPR, RAG, REALM, and extractive question answering.

Also explained in

Compare how 3 books explain BART →

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Retrieval Augmented Generation, The Foundational Ideas. The section links point to the parts of the book it draws on.