Zero-shot learning

As explained in Build a Large Language Model (From Scratch)

A learning setup where a model handles a task without having seen any task-specific examples beforehand.

Zero-shot learning lets a model generalize to a completely unseen task without prior task-specific examples. GPT-like language models perform this through text completion, using the input to specify the task without retraining, fine-tuning, or architectural changes. section 1.4

Figure 1.6 In addition to text completion, GPT-like LLMs can solve various tasks based on their inputs without needing retraining, fine-tuning, or task-specific model architecture changes. Sometimes it is helpful to provide examples of the target within the input, which is known as a few-shot setting. However, GPT-like LLMs are also capable of carrying out tasks without a specific example, which is called zero-shot setting.

Contrasts zero-shot prompting with few-shot prompting: both express the target through input, but zero-shot prompting supplies no task-specific example. section 1.4

GPT-like models apply zero-shot learning to generative tasks such as:

• machine translation section 1.4
• text summarization section 1.4
• fiction writing and computer-code generation section 1.4

How Build a Large Language Model (From Scratch) covers it

How deep it goes

Defines generalization to unseen tasks and contrasts few-shot prompting, but does not explain internal mechanisms.

Examples

Uses translation, summarization, fiction writing, and computer-code generation as examples of GPT capabilities.

What you can do after

Evaluate how GPT-like models can perform tasks from instructions alone, without retraining or task-specific examples.

How it connects

Connects zero-shot learning with GPT, text completion, transformer architecture, and few-shot learning.

liveBookmanning.com

Read Build a Large Language Model (From Scratch) on liveBook Get the book at manning.com

Also explained in

• 

  liveBookmanning.com
  Introduction to Generative AI, Second EditionZero-shot learning is performing a task without task-specific labeled examples or demonstrations.
• 

  liveBookmanning.com
  Starting Data Analytics with Generative AI and PythonA zero-shot learning model performs a task or makes predictions for categories without task-specific examples for those categories during inference.
• 

  liveBookmanning.com
  Sutskever's ListA zero-shot setting evaluates a model on a task or query without providing task-specific examples in the input.
• 

  liveBookmanning.com
  LLMs in ProductionA learning or inference setting in which a model performs a task without being given task-specific examples.
• 

  liveBookmanning.com
  Knowledge Graphs and LLMs in ActionZero-shot learning is performing a task without task-specific examples, relying on a task description and pretrained knowledge.
• 

  liveBookmanning.com
  Generative AI in ActionZero-shot learning is the ability to perform a task or recognize classes not directly represented in task-specific training examples by relying on broader learned associations or descriptions.
• 

  liveBookmanning.com
  Hugging Face in ActionZero-shot classification is the assignment of labels to inputs without task-specific labeled examples during model adaptation or inference.
• 

  liveBookmanning.com
  Transformers in ActionZero-shot learning is the ability to perform a task or recognize classes without task-specific examples during adaptation or inference.
• 

  liveBookmanning.com
  Reinforcement Learning from Human FeedbackZero-shot evaluation measures a model on questions or tasks without providing task-specific examples in the prompt.

Compare how 10 books explain zero-shot learning →

Related concepts

• few-shot learningcontrasts with · Few-shot learning is the adaptation or performance of a model on a task using only a small number of task-specific examples.
• prompt engineeringused by · Prompt engineering is the deliberate design and refinement of instructions, context, examples, or other inputs to guide an AI model’s behavior or outputs.
• one-shot learningcontrasts with · One-shot learning is a machine-learning setting in which a model generalizes to recognize or perform a task from a single labeled or demonstrated example.
• pretraininguses · Pretraining is the initial training of a model on broad data to learn general patterns or capabilities before adaptation to specific tasks or domains.
• DetectGPTincludes · DetectGPT is a zero-shot method for detecting machine-generated text by analyzing changes in language-model likelihoods under perturbations.
• NLIuses · Natural language inference is the task of determining whether a hypothesis is entailed by, contradicts, or is neutral with respect to a premise.

This explainer was generated by Manning's knowledge graph from the text of Build a Large Language Model (From Scratch). The section links point to the parts of the book it draws on.