Cross entropy loss measures the discrepancy between target and predicted probabilities. It gives a training algorithm a numerical penalty for assigning low probability to the correct class; smaller loss means a better prediction. In drug discovery, it can train a classifier to identify compounds that inhibit a cytochrome P450 enzyme. section 5.1.1
For binary classification, the target y is 0 or 1 and the prediction ŷ is the probability of class 1. Binary cross entropy combines the penalties for both possible classes. For a true class of 1, a predicted probability of 0.3 produces a loss of about 1.20, while 0.8 produces about 0.22. Training aggregates losses across examples and uses gradient descent to update the model. section 5.1.1
The binary cross-entropy equation shows how the target selects the penalty for the predicted class probability. section 5.1.1
For more than two mutually exclusive classes, softmax regression predicts a probability distribution across classes and minimizes multiclass cross entropy. This setup is unsuitable if a molecule can inhibit none or several of the enzymes: choosing one class forces exactly one answer. section 5.4.1
Classifying antibody sequences
A classifier can distinguish HIV-1 from SARS-CoV-2 antibody sequences using integer labels 0 and 1. PyTorch’s CrossEntropyLoss takes class logits and labels, applying softmax and negative log-likelihood internally in a numerically stable way. Evaluate the resulting classifier with metrics such as accuracy, precision, recall, F1 and area under the ROC curve, rather than relying on loss alone. section 12.3.4
loss_fn = nn.CrossEntropyLoss() logits = model(batch) # (B, num_classes) loss = loss_fn(logits, batch["label"]) # scalar
This PyTorch example passes a batch of class logits and integer labels to CrossEntropyLoss. section 12.3.4
Reconstructing molecular strings
An autoencoder reconstructing SMILES strings predicts vocabulary scores at each sequence position. Cross entropy compares those scores with the original token indices. Set ignore_index=0 when zero represents padding, so padded positions do not contribute to the loss. section 10.2.3
Listing 10.5 Understanding the Loss Function
def loss_function(self, recon, target, z=None):
target = self._pad_or_truncate(target) #A
# Shape: (batch * max_length, vocab_size)
recon_flat = recon.reshape( #B
-1, self.vocab_size #B
) #B
# Shape: (batch * max_length,)
target_flat = target.reshape(-1)
recon_loss = F.cross_entropy( #C
recon_flat,
target_flat,
ignore_index=0 # Don't compute loss on padding tokens
)
return {'loss': recon_loss, 'recon_loss': recon_loss}The reconstruction loss reshapes predictions to (batch × sequence length, vocabulary size) and targets to a matching one-dimensional array before calling F.cross_entropy. section 10.2.3

























