liveBook · Manning

Task loss

As explained in Rearchitecting LLMs

A task loss is an optimization objective that measures disagreement between model predictions and hard labels for a specified task.

Task loss measures disagreement between a model’s predictions and hard labels for a specific task. In next-token prediction, cross-entropy trains the student directly on the correct tokens from the dataset. section 6.3.1 section 6.6

In knowledge distillation, task loss can anchor the student to the dataset while logit loss encourages it to imitate the teacher’s probability distribution. A compound objective can also include hidden loss to align intermediate representations. section 6.3.1 section 6.8

shift_logits = student_logits[..., :-1, :].contiguous() #A
shift_labels = labels[..., 1:].contiguous() #B
loss_task = F.cross_entropy(
    shift_logits.view(-1, shift_logits.size(-1)), #C
    shift_labels.view(-1),
    ignore_index=-100 #D
)

The next-token implementation shifts student logits and labels before applying cross-entropy. It drops the last logit and first label, flattens the remaining positions, and ignores labels marked -100. section 6.3.1

Listing 6.6 General structure of the compound loss function
def compute_compound_loss(
    student_logits,
    teacher_logits,
    student_hiddens,
    teacher_hiddens,
    labels,
    layer_map,
    alpha=0.4,
    beta=0.4,
    gamma=0.2,
    temperature=2.0
):
 
    loss_task = ...
    loss_logits = ...
    loss_hidden = ...
 
    total_loss= alpha * loss_task + beta * loss_logits + gamma * loss_hidden
 
    loss_dict = {
         'total': total_loss.item(),
         'task': loss_task.item(),
         'logits': loss_logits.item(),
         'hidden': loss_hidden.item()
     }
 
     return total_loss, loss_dict
 

The compound-loss structure weights task, logit, and hidden losses with alpha, beta, and gamma. It returns the total loss and separate scalar values for monitoring. section 6.3.1

A baseline for generative models assigns weights of 0.4 to task loss, 0.4 to logit loss, and 0.2 to hidden loss. Weights commonly sum to 1.0 for interpretability, although this is not mandatory; changing their combined scale also changes gradient magnitude. Adjust the balance to the objective rather than assuming hard labels are always necessary: reported Ministral experiments found that a pure distillation objective outperformed weighted combinations with next-token prediction loss. section 6.3.1 section 6.6

How Rearchitecting LLMs covers it

How deep it goes
Works through shifted cross-entropy and its weighting within a compound loss.
Examples
Uses shifted token predictions and labels, plus Minitron and Ministral distillation comparisons.
What you can do after
Apply it via PyTorch cross-entropy in a weighted distillation objective.
How it connects
Connects task loss to knowledge distillation, cross-entropy, logit loss, hidden loss, and next-token prediction.

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Rearchitecting LLMs. The section links point to the parts of the book it draws on.