A sentiment score measures how closely a model’s response aligns with the expected output during instruction-based fine-tuning. The score guides parameter updates: desirable responses prompt minimal changes, while undesirable responses prompt larger adjustments. section 12.2.2
The fine-tuning loop applies the score through these steps:
- Combine instruction and input fields into a prompt. section 12.2.2
- Send the prompt to the model and compare its response with the stored output. section 12.2.2
- Use the resulting score to adjust model parameters across dataset entries and repeated epochs. section 12.2.2
Figure 12.2 A visualization of what happens during fine-tuning
A process visualization clarifies where response comparison and sentiment scoring occur within the fine-tuning loop. section 12.2.2
