Next-token prediction trains a language model to predict the next token from the preceding sequence. It produces fluent, coherent text by learning statistical patterns, but does not explicitly optimize for safety, helpfulness, or appropriateness. section 10.1
Fluent is not always better
For a mental-health query, “Consider talking to a professional” is plausible, but a response that acknowledges distress and gives clear emergency guidance may be preferable. Predicting likely text does not itself teach that preference. section 10.1
In supervised fine-tuning, the model still predicts response tokens one by one, conditioned on the prompt and earlier response tokens. The loss ignores prompt and padding tokens; an <EOS> target teaches the model where to stop. section 10.5
Figure 10.4 Supervised Fine-Tuning with label masking.

Label masking distinguishes prompt inputs from response tokens used for training. section 10.5












