Encoder-decoder architecture

As explained in Causal AI

A sequence-to-sequence model maps an input sequence to an output sequence, often for tasks such as translation, summarization, or conditional generation.

How Causal AI covers it

How deep it goes

Works through tokenization, dataset construction, fine-tuning, generation, and log probabilities, but does not detail encoder-decoder internals.

Examples

Generates Prince actions from King actions, Kingdom outcomes from paired actions, and King actions from Prince actions.

What it assumes

Assumes Python tensor and dataset workflows while providing tokenizer, dataset, training, decoding, and generation functions.

What you can do after

Apply it via Hugging Face Transformers and PyTorch to train, load, generate from, and score conditional text models.

How it connects

Connects sequence-to-sequence models with tokenization, Transformer models, PyTorch datasets, conditional generation, and causal models.

liveBookmanning.com

Read Causal AI on liveBook Get the book at manning.com

Also explained in

• 

  liveBookmanning.com
  Natural Language Processing in Action, Second EditionA neural architecture that encodes an input sequence into a representation and decodes that representation into an output sequence.
• 

  liveBookmanning.com
  Inside Deep Learning: Math, Algorithms, ModelsA neural sequence transduction model that maps an input sequence to an output sequence, potentially of different length, using an encoder and decoder.
• 

  liveBookmanning.com
  Deep Learning with Python, Third EditionA model that maps an input sequence to an output sequence, often used for translation or generation tasks.
• 

  liveBookmanning.com
  Knowledge Graphs and LLMs in ActionAn encoder-decoder architecture is a model design in which an encoder transforms input data into a learned representation and a decoder transforms that representation into predictions or reconstructed outputs.
• 

  liveBookmanning.com
  Deep Learning with R, Third EditionA sequence-to-sequence model predicts a target sequence from a separate source sequence, typically conditioning each target token on prior target tokens.
• 

  liveBookmanning.com
  Sutskever's ListA sequence-to-sequence model maps an input sequence to an output sequence using learned encoder and decoder components or an equivalent transformation.
• 

  liveBookmanning.com
  Optimization Algorithms: AI techniques for design, planning, and control problemsA neural model that transforms an input sequence into an output sequence, typically allowing the two sequences to have different lengths.
• 

  liveBookmanning.com
  Learn Generative AI with PyTorch
• 

  liveBookmanning.com
  Effective Conversational AIAn encoder-decoder architecture is a language-model architecture that encodes an input sequence and decodes a separate output sequence.

Compare how 10 books explain encoder-decoder architecture →

Related concepts

• BARTincludes · BART is a denoising sequence-to-sequence Transformer model with a bidirectional encoder and an autoregressive decoder.
• decodehas part · Decoding is the process of transforming an encoded or internal representation into an interpretable or target-form output.
• encoderhas part · An encoder is a component that transforms input data into an encoded representation for subsequent processing, reconstruction, comparison, or transmission.
• attentionuses · Attention is a neural-network mechanism that computes input-dependent weights to combine information according to its relevance to a computation.
• link predictionuses · Link prediction is a graph-learning task that estimates whether a connection exists or may occur between nodes, often by scoring candidate links.
• retrieval-augmented generationused by · Retrieval-augmented generation is a technique that retrieves relevant information from external sources and supplies it as context to a generative model when producing an output.

This explainer was generated by Manning's knowledge graph from the text of Causal AI. The section links point to the parts of the book it draws on.