liveBook · Manning

Encoder-decoder architecture

As explained in Build AI Drug Discovery Pipelines

A sequence-to-sequence architecture maps an input sequence to an output sequence, typically through an encoder and decoder.

An encoder–decoder architecture maps one sequence to another. In molecular design, it converts a SMILES string into a compressed representation, then decodes that representation into an output SMILES string. section 10.4.3 section 10.0

A molecular variational autoencoder specializes the general pattern as follows:

  • The encoder embeds input SMILES tokens and uses a bidirectional GRU to read the sequence from both directions. section 10.4.3
  • The encoder projects its final hidden states into the parameters of a latent distribution, from which a latent vector is sampled. section 10.4.3
  • The decoder uses a unidirectional GRU to generate SMILES tokens sequentially, conditioned on the latent vector. section 10.4.3
  • The implementation initializes decoder hidden states from the latent vector and concatenates that vector with token embeddings at every timestep. section 10.4.3
  • During training, teacher forcing supplies ground-truth tokens; during generation, temperature sampling selects successive tokens until completion. section 10.4.3

Molecule reconstruction and generation

An input molecule is represented as SMILES, encoded into latent distribution parameters, sampled into z, and decoded into a reconstructed SMILES string. Sampling different latent vectors supports exploration of chemical space, while reconstruction tests whether molecular information survives compression. section 10.4.3 section 10.5

Figure 10.7 Architecture of a bidirectional RNN for SMILES string encoding. The network processes the input sequence (x₀, x₁, ..., x_{T-1}, x_T) in both forward (left-to-right, top row) and backward (right-to-left, bottom row) directions simultaneously. Each time step produces a hidden state that concatenates the forward hidden state and backward hidden state. The bidirectional processing allows the model to capture dependencies flowing in both directions through the molecular sequence, with each token's representation informed by both its predecessors and successors in the SMILES string.

The bidirectional encoder processes every SMILES token left-to-right and right-to-left, then concatenates the two hidden-state representations. section 10.4.3

How Build AI Drug Discovery Pipelines covers it

How deep it goes
Explains bidirectional GRU encoding, latent projection, sequential decoding, teacher forcing, and temperature sampling with implementation details.
Examples
Uses SMILES reconstruction, bidirectional encoder diagrams, latent-vector decoding, teacher forcing, and temperature-based molecule generation.
What you can do after
Build it from scratch with PyTorch by implementing bidirectional encoding, latent sampling, sequential decoding, and molecule generation.
How it connects
Connects SMILES strings, recurrent neural networks, GRUs, variational autoencoders, latent spaces, teacher forcing, and temperature sampling.

Also explained in

Compare how 14 books explain encoder-decoder architecture →

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Build AI Drug Discovery Pipelines. The section links point to the parts of the book it draws on.