liveBook · Manning

Encoder-decoder architecture

As explained in Build an Advanced RAG Application (From Scratch)

An encoder-decoder architecture is a neural-network design in which an encoder transforms an input sequence into representations that a decoder uses to produce an output sequence.

An encoder-decoder architecture combines two neural-network roles: the encoder transforms an input sequence into embeddings, and the decoder uses those representations to generate an output sequence. This design is especially useful for sequence-to-sequence tasks. section 2.3.3

The encoder processes semantic content into dense numerical vectors that capture contextual relationships. The decoder then converts those encoded representations into the requested output, extending the Transformer approach beyond using either component alone. section 2.3.3 section 2.0

Choose the architecture according to the amount and form of information being processed:

  • Use an encoder to embed large document collections or queries when building retrieval systems. section 2.3.3
  • A decoder alone may summarize one document when the article fits within its context length. section 2.3.3

Semantic search for enhanced retrieval

Given the query “Impact of climate change on marine ecosystems Jane Scientist,” the encoder represents the query and repository documents as vectors. The retrieval system can then identify articles matching the query’s meaning, context, author, and related concepts rather than relying only on keywords. section 2.3.3

How Build an Advanced RAG Application (From Scratch) covers it

How deep it goes
Explains encoder embeddings, decoder generation, and a retrieval trade-off, without detailing layer-level computation or implementation code.
Examples
Uses climate-change articles, an author-specific semantic query, document embeddings, and retrieval as concrete examples.
What you can do after
Evaluate when to combine encoders and decoders, use encoders for retrieval-scale data, or choose decoder-only summarization.
How it connects
Connects sequence-to-sequence models with Transformers, attention mechanisms, embeddings, semantic search, and machine translation.

Also explained in

Compare how 14 books explain encoder-decoder architecture →

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Build an Advanced RAG Application (From Scratch). The section links point to the parts of the book it draws on.