liveBook · Manning

Autoencoder

As explained in Build AI Drug Discovery Pipelines

An autoencoder is a neural model that encodes inputs into a latent representation and decodes that representation to reconstruct the inputs.

An autoencoder is a neural network trained to reconstruct its own input. An encoder compresses the input into a lower-dimensional latent vector; a decoder uses that vector to recreate the input. Training minimizes reconstruction error, the difference between the original and reconstructed data. section 10.2.2

The latent vector acts as an information bottleneck: the model must retain enough information in a compact representation for the decoder to reconstruct the input. For molecular design, these vectors form a latent space that can serve as a map for exploring chemical structures. section 10.2.2 section 10.0

Figure 10.2 Core components of an autoencoder neural network applied to molecular structures. The process flows from left to right: an input molecule (X) is fed into an encoder network that compresses the molecular representation into a low-dimensional latent vector. This compressed representation captures the most essential features of the molecule in a compact form. The decoder network then attempts to reconstruct the original molecule (X') from this latent vector. The reconstruction loss (shown by the dashed red arrow) measures the difference between the input molecule and the reconstructed output, providing the training signal that forces the network to learn meaningful molecular representations. The information bottleneck created by the latent vector ensures the autoencoder must learn to encode only the most important molecular features necessary for accurate reconstruction.

Follow a molecule through the encoder, latent vector, decoder, and reconstruction loss. section 10.2.2

Reconstructing a SMILES string

A SMILES string represents a molecule as text. For example, a vocabulary can turn CCO into character token IDs, adding start and end tokens. A learnable embedding layer converts those IDs to dense vectors; the encoder compresses them, and the decoder predicts the original sequence. section 10.2.2 section 10.2.3

In a basic implementation, embedded token vectors are flattened before entering the encoder. The decoder produces predictions for each sequence position. Cross-entropy measures reconstruction loss, with padding positions ignored so they do not contribute to the training signal. section 10.2.3

Listing 10.5 Understanding the Loss Function
    def loss_function(self, recon, target, z=None):
        target = self._pad_or_truncate(target)    #A
        
        # Shape: (batch * max_length, vocab_size)
        recon_flat = recon.reshape(    #B
            -1, self.vocab_size    #B
        )    #B
        # Shape: (batch * max_length,)
        target_flat = target.reshape(-1)
        
        recon_loss = F.cross_entropy(    #C
            recon_flat,
            target_flat,
            ignore_index=0  # Don't compute loss on padding tokens
        )
        return {'loss': recon_loss, 'recon_loss': recon_loss}

Inspect how token predictions and targets are reshaped for cross-entropy, excluding padding. section 10.2.3

Good reconstruction alone does not guarantee useful molecule generation. A basic autoencoder can reconstruct molecules it has seen while learning a discontinuous latent space that fails to generate valid new molecules. A variational autoencoder instead maps inputs to distributions and regularizes the latent space to support smoother exploration. section 10.5

Evaluate a molecular autoencoder on more than reconstruction: check whether nearby latent points correspond to similar structures and whether sampled molecules are valid and diverse. MOSES provides standardized molecular data, splits, metrics, and baselines for such comparisons. section 10.2.4 section 10.2.1

How Build AI Drug Discovery Pipelines covers it

How deep it goes
Develops tokenization, embeddings, encoder-decoder architecture, reconstruction loss, and benchmark evaluation before extending the model to a variational autoencoder.
Examples
Uses SMILES reconstruction and the MOSES molecular generation benchmark to illustrate and evaluate the model.
What you can do after
Build it from scratch with PyTorch for molecular reconstruction and evaluate reconstruction, latent-space quality, and generation.
How it connects
Connects latent spaces to molecular optimization, variational autoencoders, and recurrent neural networks.

Also explained in

Compare how 15 books explain autoencoder →

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Build AI Drug Discovery Pipelines. The section links point to the parts of the book it draws on.