liveBook · Manning

Reconstruction loss

As explained in Build AI Drug Discovery Pipelines

A reconstruction loss quantifies the discrepancy between an input and the output reconstructed from its encoded representation.

Reconstruction loss measures the difference between an input and the output recreated from its encoded representation. In an autoencoder, an encoder compresses the input into a latent vector, and a decoder uses that vector to reconstruct the input. Minimizing the loss gives the network a training signal for retaining useful information through the compressed representation. section 10.2.2

Figure 10.2 Core components of an autoencoder neural network applied to molecular structures. The process flows from left to right: an input molecule (X) is fed into an encoder network that compresses the molecular representation into a low-dimensional latent vector. This compressed representation captures the most essential features of the molecule in a compact form. The decoder network then attempts to reconstruct the original molecule (X') from this latent vector. The reconstruction loss (shown by the dashed red arrow) measures the difference between the input molecule and the reconstructed output, providing the training signal that forces the network to learn meaningful molecular representations. The information bottleneck created by the latent vector ensures the autoencoder must learn to encode only the most important molecular features necessary for accurate reconstruction.

Follow a molecule through the encoder and decoder; the loss compares the reconstructed output with the original input. section 10.2.2

For molecular design, a SMILES string represents a molecule as a sequence of characters. The sequence is converted to token IDs before encoding. The decoder produces scores for each vocabulary token at each sequence position; cross-entropy compares those scores with the target token IDs. Padding positions are ignored so they do not contribute to the loss. section 10.2.2 section 10.2.3

Listing 10.5 Understanding the Loss Function
    def loss_function(self, recon, target, z=None):
        target = self._pad_or_truncate(target)    #A
        
        # Shape: (batch * max_length, vocab_size)
        recon_flat = recon.reshape(    #B
            -1, self.vocab_size    #B
        )    #B
        # Shape: (batch * max_length,)
        target_flat = target.reshape(-1)
        
        recon_loss = F.cross_entropy(    #C
            recon_flat,
            target_flat,
            ignore_index=0  # Don't compute loss on padding tokens
        )
        return {'loss': recon_loss, 'recon_loss': recon_loss}

The PyTorch loss function reshapes decoder scores and target IDs, then applies cross-entropy with padding index 0 ignored. section 10.2.3

Accurate reconstruction alone does not guarantee useful generation of new molecules. A basic autoencoder can reconstruct familiar molecules well while leaving a discontinuous latent space that is difficult to explore. A variational autoencoder combines reconstruction loss with a KL divergence term to regularize that space for sampling. section 10.5

How Build AI Drug Discovery Pipelines covers it

How deep it goes
Explains token-level cross-entropy, tensor reshaping, and padding exclusion in a PyTorch loss function.
Examples
Uses molecular SMILES sequences and decoder token scores to illustrate reconstruction.
What you can do after
Apply it via PyTorch cross-entropy to decoder scores and target token IDs while ignoring padding.
How it connects
Connects autoencoders, latent space, cross-entropy, variational autoencoders, and KL divergence.

Also explained in

Compare how 11 books explain reconstruction loss →

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Build AI Drug Discovery Pipelines. The section links point to the parts of the book it draws on.