BART

As explained in Causal AI

BART is a denoising sequence-to-sequence transformer language model developed for generating text from input text.

BART is a Transformer sequence-to-sequence model that maps an input text sequence to a corresponding output, supporting tasks such as translation, summarization, and vignette generation. Its encoder-decoder structure makes it suitable for representing conditional text distributions rather than only completing text from an empty prompt. section 13.3.2

In a causal language model, fine-tuning aligns pretrained language generation with a causal Markov kernel. A BART model can generate a Prince’s action from a King’s action, generate a Kingdom’s fate from both actions, or infer a King’s action from a Prince’s action. Hugging Face Transformers supplies tokenization, pretrained weights, and sequence-to-sequence training utilities. section 13.3.2

Conditional story generation

Fine-tune separate BART models on paired King–Prince and action–fate vignettes, then generate a full story by passing each generated sequence to the next model. PyTorch and Seq2SeqTrainer support tokenization, validation splits, training, generation, and log-probability calculations. section 13.3.2

Figure 13.6 A BART sequence-to-sequence model is fine-tuned to represent the Prince’s action given the King’s action.

Shows BART mapping the King’s action vignette to the Prince’s action. section 13.3.2

How Causal AI covers it

How deep it goes

Explains sequence mapping, fine-tuning, tokenization, training, generation, and log probabilities, but does not derive Transformer internals.

Examples

Uses King, Prince, and Kingdom action vignettes as paired inputs, targets, and generated story components.

What it assumes

Assumes familiarity with pretrained models, tokenization, paired datasets, and causal graphs while demonstrating the training workflow.

What you can do after

Builds and fine-tunes BART sequence-to-sequence models with Hugging Face Transformers for conditional text generation.

How it connects

Connects BART with sequence-to-sequence learning, causal Markov kernels, tokenization, PyTorch, Hugging Face Transformers, and Pyro.

liveBookmanning.com

Read Causal AI on liveBook Get the book at manning.com

Also explained in

• 

  liveBookmanning.com
  Build a Text-to-Image Generator (from Scratch)BART is a denoising sequence-to-sequence transformer model whose encoder is bidirectional and whose decoder is autoregressive.

Compare how 2 books explain BART →

Related concepts

• retrieval-augmented generationused by · Retrieval-augmented generation is a technique that retrieves relevant information from external sources and supplies it as context to a generative model when producing an output.
• encoder-decoder architecturekind of · An encoder-decoder architecture is a model design in which an encoder transforms input into an intermediate representation and a decoder uses that representation to produce output.
• generatorimplements · A generator is a computational component or construct that produces data or values from inputs, representations, context, or specified constraints.
• transformer-based text-to-image modelpart of · A transformer-based text-to-image model generates images from text by modeling relationships between textual and visual token sequences with transformer architectures.
• bidirectional encoderhas part · A bidirectional encoder represents each sequence position using contextual information from both preceding and following positions.

This explainer was generated by Manning's knowledge graph from the text of Causal AI. The section links point to the parts of the book it draws on.