Cross-entropy loss measures how wrong a model’s predicted probability distribution is for a target outcome. It returns one error value: smaller is better, while assigning low probability to the correct outcome produces a larger penalty. section 3.1.2
Sentiment classification
For a sentiment model, the correct label is “positive,” but the model assigns it 30% softmax probability. Cross-entropy penalizes this uncertainty and signals that the model should become more confident. section 3.1.2
During fine-tuning, cross-entropy supports the training loop by:
- Converting model outputs and targets into a single loss value. section 3.1.2
- Providing the error signal that gradient descent uses to adjust parameters toward lower-loss configurations. section 3.1.2
Listing 3.1 Basic steps of the training process
for inputs, targets in dataloader: #A
outputs = model(inputs) #B
loss = loss_fn(outputs, targets) #C
optimizer.zero_grad() #D
loss.backward() #E
optimizer.step() #F
if step % 10 == 0: #G
print(f"Step {step}, Loss: {loss.item():.4f}")Shows where the loss function fits between model inference, target comparison, backpropagation, and optimizer updates. section 3.1.2

























