liveBook · Manning

Chosen response

As explained in LLM Customization and Fine-Tuning

The candidate response in a preference pair that a human or automated system prefers.

A chosen response is the preferred answer in a preference pair: the model is trained to favor it over a rejected response to the same prompt. Preferences can come from human judgments or automated systems. section 8.2 section 8.2.5

A preference record contains:

  • A prompt, represented as the conversation leading up to the user’s question. section 8.2.1
  • A chosen assistant response and a rejected assistant response to that prompt. section 8.2.1

Enterprise support preference

For an IT installation question, prefer the answer that follows the organization’s format and directs users to its approved Software Center. A fluent answer directing users to a public download can be rejected, even if technically correct. section 8.2.1

Figure 8.3 The DPO training mechanism: raise the chosen response, lower the rejected one, stay near the frozen reference.

The diagram shows how DPO raises the chosen response’s probability while lowering the rejected response’s and constraining drift from a frozen reference. section 8.2.5

How LLM Customization and Fine-Tuning covers it

How deep it goes
Defines its role in a preference pair and traces how DPO raises its probability relative to rejected output and a frozen reference.
Examples
Contrasts an IT support answer directing users to the approved Software Center with a technically correct public-download answer.
What you can do after
Build a TRL-style preference record by pairing a prompt with preferred and dispreferred assistant messages.
How it connects
Connects chosen responses to rejected responses, preference pairs, DPO, and frozen reference models.

Related concepts

This explainer was generated by Manning's knowledge graph from the text of LLM Customization and Fine-Tuning. The section links point to the parts of the book it draws on.