Preference

As explained in Deep Learning with Python, Third Edition

The relative ranking of candidate responses by people according to what is more helpful or desirable.

How Deep Learning with Python, Third Edition covers it

How deep it goes

Defines ranking-based data collection and reward modeling, then sketches score-based filtering without deriving the training loss.

Examples

Uses ranked prompt responses and a generate-score-filter-fit loop as concrete illustrations.

What it assumes

Assumes familiarity with supervised fine-tuning while establishing why handwritten output alone is insufficient.

What you can do after

Evaluate how human rankings can supervise a reward model and guide response selection.

How it connects

Connects supervised fine-tuning, reinforcement learning from human feedback, reward models, and Transformers.

liveBookmanning.com

Read Deep Learning with Python, Third Edition on liveBook Get the book at manning.com

Also explained in

• 

  liveBookmanning.com
  Reinforcement Learning from Human FeedbackA human preference is a person's comparative judgment about the desirability or quality of model behavior or outputs.
• 

  liveBookmanning.com
  Bayesian Optimization in ActionA user preference is a person's relative desirability judgment over products, experiences, or other alternatives.

Compare how 3 books explain preference →

Related concepts

• reinforcement learning with human feedbackused by · Reinforcement learning with human feedback is a model-training approach that uses human evaluations or preferences to guide optimization of model behavior.
• reward modelused by · A reward model is a learned model that assigns scores to outputs, actions, or outcomes to represent their desirability according to preferences or evaluation criteria.
• recommender systemused by · A recommender system is software that predicts, selects, or ranks items, content, or actions for users based on available information about users, items, or context.
• experimentationmeasured by · Experimentation is the deliberate testing of an idea, intervention, or alternative by observing outcomes to gather evidence or compare effects.
• user profilepart of · A user profile is a collection of information associated with a user or account, such as identifying attributes, preferences, settings, or activity.

This explainer was generated by Manning's knowledge graph from the text of Deep Learning with Python, Third Edition. The section links point to the parts of the book it draws on.