liveBook · Manning

Encoder

As explained in Build AI Drug Discovery Pipelines

An encoder is a model component that transforms an input into a lower-dimensional representation while retaining selected information.

An encoder transforms an input into a representation that another model component or task can use. In a molecular autoencoder, it compresses a molecule’s representation into a lower-dimensional latent vector: a compact set of coordinates intended to retain information needed for reconstruction. section 10.2.2

A decoder attempts to recreate the input from that vector. Training minimizes the difference between the input and reconstruction. The limited latent space creates an information bottleneck, encouraging the encoder to retain features useful for reconstruction—not necessarily every detail of the input. section 10.2.2

Figure 10.2 Core components of an autoencoder neural network applied to molecular structures. The process flows from left to right: an input molecule (X) is fed into an encoder network that compresses the molecular representation into a low-dimensional latent vector. This compressed representation captures the most essential features of the molecule in a compact form. The decoder network then attempts to reconstruct the original molecule (X') from this latent vector. The reconstruction loss (shown by the dashed red arrow) measures the difference between the input molecule and the reconstructed output, providing the training signal that forces the network to learn meaningful molecular representations. The information bottleneck created by the latent vector ensures the autoencoder must learn to encode only the most important molecular features necessary for accurate reconstruction.

Follow a molecule from encoder to latent vector and decoder, with reconstruction loss providing the training signal. section 10.2.2

Encoding a SMILES string

A SMILES string is a text representation of a molecule. For an autoencoder, its characters are mapped to token IDs, then a learnable embedding layer converts those IDs into dense vectors. The vectors are flattened and passed to the encoder; the decoder uses the resulting latent vector to reconstruct the string. section 10.2.2 section 10.2.3

A sequence-to-sequence variational autoencoder uses a different encoder: a bidirectional GRU reads the SMILES sequence in both directions. Its final forward and backward hidden states are combined and projected to parameters of a latent distribution. A latent vector is sampled from that distribution for the decoder to generate a SMILES sequence. section 10.4.3

Figure 10.8 Bidirectional RNN Encoder Architecture for VAE-CYC. The encoder processes input molecules by first converting them from chemical structure to SMILES representation, then through an embedding layer that converts text to numerical vectors. A multi-layer bidirectional RNN encoder processes the sequence in both forward and backward directions. The final hidden states from both directions are concatenated and passed through a linear projection layer to generate latent distribution parameters. These parameters, combined with random noise through the reparameterization trick, produce the final latent vector representation of the input molecule.

Trace the bidirectional encoder from molecular input through embeddings and hidden states to latent distribution parameters. section 10.4.3

Not every encoder compresses an entire input into one vector. A Transformer encoder produces a contextual representation for each input token. An encoder-only Transformer can use these representations for protein folding and property prediction without a decoder; a standard Transformer decoder can instead attend to the encoder’s sequence of representations. section 12.3.2

How Build AI Drug Discovery Pipelines covers it

How deep it goes
Explains bottleneck training and embedding preparation, then traces bidirectional GRU and Transformer encoder architectures.
Examples
Uses SMILES molecular reconstruction, a bidirectional GRU variational autoencoder, and protein-sequence representations.
What it assumes
Introduces SMILES tokenization and embeddings before building a molecular encoder.
What you can do after
Understand how molecular encoders prepare sequences and produce latent representations for decoding.
How it connects
Connects encoders to autoencoders, latent spaces, decoders, recurrent neural networks, and Transformers.

Also explained in

Compare how 28 books explain encoder →

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Build AI Drug Discovery Pipelines. The section links point to the parts of the book it draws on.