Zero-shot learning

Q: How deeply does Reinforcement Learning from Human Feedback cover zero-shot learning?

Defines the evaluation setup and motivation, but stops short of detailing implementation beyond sampling and exact-match terminology.

As explained in Reinforcement Learning from Human Feedback

Zero-shot evaluation measures a model on questions or tasks without providing task-specific examples in the prompt.

Zero-shot evaluation tests a model on questions or tasks without task-specific examples in the prompt. It measures whether instruction-tuned models can generalize beyond the tasks used during training, making evaluation closer to open-ended use than example-conditioned testing. section 16.1.3

A zero-shot question

A prompt can ask, “What is the capital of France?” and leave the assistant response field empty. The model must answer without demonstrations or task-specific examples. section 16.1.3

FLAN demonstrated that models fine-tuned on specific tasks could answer unseen zero-shot questions; T0 reported similar results. Zero-shot evaluation therefore marks an important transition toward instruction fine-tuning, later RLHF, and generation-based assessment. section 16.1.3

User: "What is the capital of France?"
Assistant:

Inspect a minimal zero-shot prompt: it specifies a question but supplies no worked examples for the requested task. section 16.1.3

How Reinforcement Learning from Human Feedback covers it

Cover of Reinforcement Learning from Human Feedback

liveBook manning.com

Read Reinforcement Learning from Human Feedback on liveBook Get the book at manning.com

Also explained in

liveBook manning.com

Introduction to Generative AI, Second EditionZero-shot learning is performing a task without task-specific labeled examples or demonstrations.
liveBook manning.com

Starting Data Analytics with Generative AI and PythonA zero-shot learning model performs a task or makes predictions for categories without task-specific examples for those categories during inference.
liveBook manning.com

Sutskever's ListA zero-shot setting evaluates a model on a task or query without providing task-specific examples in the input.
liveBook manning.com

LLMs in ProductionA learning or inference setting in which a model performs a task without being given task-specific examples.
liveBook manning.com

Knowledge Graphs and LLMs in ActionZero-shot learning is performing a task without task-specific examples, relying on a task description and pretrained knowledge.
liveBook manning.com

Generative AI in ActionZero-shot learning is the ability to perform a task or recognize classes not directly represented in task-specific training examples by relying on broader learned associations or descriptions.
liveBook manning.com

Hugging Face in ActionZero-shot classification is the assignment of labels to inputs without task-specific labeled examples during model adaptation or inference.
liveBook manning.com

Transformers in ActionZero-shot learning is the ability to perform a task or recognize classes without task-specific examples during adaptation or inference.
liveBook manning.com

Build a Large Language Model (From Scratch)A learning setup where a model handles a task without having seen any task-specific examples beforehand.

Compare how 10 books explain zero-shot learning →

Related concepts

few-shot learningcontrasts with · Few-shot learning is the adaptation or performance of a model on a task using only a small number of task-specific examples.
prompt engineeringused by · Prompt engineering is the deliberate design and refinement of instructions, context, examples, or other inputs to guide an AI model’s behavior or outputs.
one-shot learningcontrasts with · One-shot learning is a machine-learning setting in which a model generalizes to recognize or perform a task from a single labeled or demonstrated example.
pretraininguses · Pretraining is the initial training of a model on broad data to learn general patterns or capabilities before adaptation to specific tasks or domains.
DetectGPTincludes · DetectGPT is a zero-shot method for detecting machine-generated text by analyzing changes in language-model likelihoods under perturbations.
NLIuses · Natural language inference is the task of determining whether a hypothesis is entailed by, contradicts, or is neutral with respect to a premise.

This explainer was generated by Manning's knowledge graph from the text of Reinforcement Learning from Human Feedback. The section links point to the parts of the book it draws on.