liveBook · Manning

Logit

As explained in Build AI Drug Discovery Pipelines

A logit is an unnormalized real-valued score for a possible output token before conversion to probabilities.

A logit is an unnormalized score for a possible output. In a molecule-generating autoencoder, each character position in a SMILES string has one logit per vocabulary token. These scores are not yet probabilities. section 10.2.3

The decoder expands a latent vector into a tensor shaped (batch, max_length, vocab_size). For each molecule and sequence position, the last dimension holds the scores for all possible tokens. section 10.2.3

Listing 10.4 Basic Autoencoder Architecture: Decoder
        decoder_layers = []
        input_dim = latent_dim    #A
        
        for hidden_dim in reversed(hidden_dims):    #B
            decoder_layers.extend([
                nn.Linear(input_dim, hidden_dim),
                nn.ReLU(),
                nn.BatchNorm1d(hidden_dim),
                nn.Dropout(0.1)
            ])
            input_dim = hidden_dim
        
        decoder_layers.extend([    #C
            nn.Linear(input_dim, vocab_size * max_length),
            nn.Unflatten(1, (max_length, vocab_size))    #D
        ])
        self.decoder = nn.Sequential(*decoder_layers)
        self._initialize_weights()    #E

The final linear layer produces max_length × vocab_size scores per molecule, then unflattens them into position-by-vocabulary predictions. section 10.2.3

During training, cross-entropy loss compares the logits with the true token indices. Padding positions are ignored, so the model is not penalized for predictions where there is no molecular character to reconstruct. section 10.2.3

Listing 10.5 Understanding the Loss Function
    def loss_function(self, recon, target, z=None):
        target = self._pad_or_truncate(target)    #A
        
        # Shape: (batch * max_length, vocab_size)
        recon_flat = recon.reshape(    #B
            -1, self.vocab_size    #B
        )    #B
        # Shape: (batch * max_length,)
        target_flat = target.reshape(-1)
        
        recon_loss = F.cross_entropy(    #C
            recon_flat,
            target_flat,
            ignore_index=0  # Don't compute loss on padding tokens
        )
        return {'loss': recon_loss, 'recon_loss': recon_loss}

The loss function reshapes the predictions and targets, then passes them to PyTorch cross-entropy with ignore_index=0. section 10.2.3

Binary log-odds

In logistic regression for CYP3A4 inhibition, the score before the sigmoid is the log-odds of inhibition. If a feature has weight 0.45, increasing it by one unit multiplies the predicted odds by exp(0.45), or about 1.57, with other features fixed. That is a 57% increase in odds, not in probability. section 5.1.3

How Build AI Drug Discovery Pipelines covers it

How deep it goes
Explains decoder output shapes and cross-entropy training, and relates binary logistic scores to odds and feature weights.
Examples
Uses a SMILES decoder and a CYP3A4 inhibitor feature with weight 0.45.
What you can do after
Apply it via PyTorch cross-entropy to train a decoder against token indices while ignoring padding.
How it connects
Connects logits to autoencoders, cross-entropy loss, logistic regression, sigmoid, and log-odds.

Also explained in

Compare how 24 books explain logit →

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Build AI Drug Discovery Pipelines. The section links point to the parts of the book it draws on.