A logit is an unnormalized score for a possible output. In a molecule-generating autoencoder, each character position in a SMILES string has one logit per vocabulary token. These scores are not yet probabilities. section 10.2.3
The decoder expands a latent vector into a tensor shaped (batch, max_length, vocab_size). For each molecule and sequence position, the last dimension holds the scores for all possible tokens. section 10.2.3
Listing 10.4 Basic Autoencoder Architecture: Decoder
decoder_layers = []
input_dim = latent_dim #A
for hidden_dim in reversed(hidden_dims): #B
decoder_layers.extend([
nn.Linear(input_dim, hidden_dim),
nn.ReLU(),
nn.BatchNorm1d(hidden_dim),
nn.Dropout(0.1)
])
input_dim = hidden_dim
decoder_layers.extend([ #C
nn.Linear(input_dim, vocab_size * max_length),
nn.Unflatten(1, (max_length, vocab_size)) #D
])
self.decoder = nn.Sequential(*decoder_layers)
self._initialize_weights() #EThe final linear layer produces max_length × vocab_size scores per molecule, then unflattens them into position-by-vocabulary predictions. section 10.2.3
During training, cross-entropy loss compares the logits with the true token indices. Padding positions are ignored, so the model is not penalized for predictions where there is no molecular character to reconstruct. section 10.2.3
Listing 10.5 Understanding the Loss Function
def loss_function(self, recon, target, z=None):
target = self._pad_or_truncate(target) #A
# Shape: (batch * max_length, vocab_size)
recon_flat = recon.reshape( #B
-1, self.vocab_size #B
) #B
# Shape: (batch * max_length,)
target_flat = target.reshape(-1)
recon_loss = F.cross_entropy( #C
recon_flat,
target_flat,
ignore_index=0 # Don't compute loss on padding tokens
)
return {'loss': recon_loss, 'recon_loss': recon_loss}The loss function reshapes the predictions and targets, then passes them to PyTorch cross-entropy with ignore_index=0. section 10.2.3
Binary log-odds
In logistic regression for CYP3A4 inhibition, the score before the sigmoid is the log-odds of inhibition. If a feature has weight 0.45, increasing it by one unit multiplies the predicted odds by exp(0.45), or about 1.57, with other features fixed. That is a 57% increase in odds, not in probability. section 5.1.3























