Few-shot learning

As explained in Build a Large Language Model (From Scratch)

The ability to learn a new task from only a few examples rather than extensive training data.

Few-shot learning lets a GPT-like language model perform a new task from a minimal number of examples supplied in the input, without retraining or task-specific architecture changes. section 1.4

Few-shot learning fits into the LLM workflow as follows:

• A pretrained model performs text completion and uses example inputs to infer the target task. section 1.3
• Examples are placed directly in the prompt, unlike fine-tuning, which trains further on labeled data. section 1.4 section 1.3
• Zero-shot learning omits task-specific examples, while few-shot learning includes a small number of them. section 1.4

Figure 1.6 In addition to text completion, GPT-like LLMs can solve various tasks based on their inputs without needing retraining, fine-tuning, or task-specific model architecture changes. Sometimes it is helpful to provide examples of the target within the input, which is known as a few-shot setting. However, GPT-like LLMs are also capable of carrying out tasks without a specific example, which is called zero-shot setting.

Compare zero-shot and few-shot prompting, including how target examples appear in the input without model retraining. section 1.4

How Build a Large Language Model (From Scratch) covers it

How deep it goes

Defines few-shot and contrasts it with zero-shot learning, but stops short of prompt construction or implementation details.

Examples

Uses target examples embedded in an input prompt and contrasts them with example-free zero-shot inputs.

What it assumes

Assumes familiarity with pretrained LLMs and text completion while distinguishing few-shot learning from fine-tuning.

What you can do after

Apply it via GPT-like LLM inputs that provide a few task examples without retraining or changing model architecture.

How it connects

Connects few-shot learning with GPT, text completion, zero-shot learning, pretraining, fine-tuning, and transformer architecture.

liveBookmanning.com

Read Build a Large Language Model (From Scratch) on liveBook Get the book at manning.com

Also explained in

• 

  liveBookmanning.com
  Software Testing with Generative AIFew-shot prompting is a prompting technique that supplies a small number of explicit examples to clarify instructions and guide a generative model's output.
• 

  liveBookmanning.com
  Natural Language Processing in Action, Second EditionFew-shot learning is adapting a model to perform a task from a small number of examples supplied at inference time, often in the prompt.
• 

  liveBookmanning.com
  AI Agents and ApplicationsPrompt examples are illustrative instances supplied in a prompt to guide a language model toward a desired task behavior or output pattern.
• 

  liveBookmanning.com
  Effective Conversational AIFew-shot prompting is the practice of providing a language model with a small number of example demonstrations in a prompt to guide its response.
• 

  liveBookmanning.com
  LLMs in ProductionFew-shot learning is the adaptation or prediction of a model from only a small number of labeled examples.
• 

  liveBookmanning.com
  Introduction to Generative AI, Second EditionFew-shot prompting is a technique that guides a model by providing a small number of task-specific examples in the input.
• 

  liveBookmanning.com
  Build AI-Enhanced Web AppsFew-shot learning is a technique in which a model performs or adapts to a task by inferring the desired behavior from a small number of examples.
• 

  liveBookmanning.com
  Generative AI in ActionFew-shot learning is the adaptation of a model to a task by providing a small number of task-specific examples at use time.
• 

  liveBookmanning.com
  The Art of AI Product DevelopmentFew-shot prompting is prompting a model with a small number of input-output examples to guide its behavior on a new input.
• 

  liveBookmanning.com
  Data Analysis with LLMsFew-shot learning is an approach in which a model infers a task from a small number of examples specifying inputs and desired outputs.
• 

  liveBookmanning.com
  Time Series Forecasting Using Foundation ModelsA few-shot prompt is a model input that includes a small number of examples to demonstrate the desired task or output behavior.
• 

  liveBookmanning.com
  The Complete Obsolete Guide to Generative AIFew-shot learning is adapting a model to a task by conditioning it on a small number of task-specific examples or demonstrations.
• 

  liveBookmanning.com
  Transformers in ActionFew-shot learning is the adaptation of a model to a new task using only a small number of task-specific examples.
• 

  liveBookmanning.com
  Learn Generative AI with PyTorchA prompting approach that provides exactly one example to condition a large language model on the task.
• 

  liveBookmanning.com
  AI for Everyday ITFew-shot prompting is a technique that supplies an AI model with a small number of examples in the prompt to guide its behavior on a task.
• 

  liveBookmanning.com
  Sutskever's ListA learning or prompting approach in which a model performs a task from a small number of examples supplied during evaluation or adaptation.
• 

  liveBookmanning.com
  Deep Learning with R, Third EditionFew-shot learning is adapting or directing a model to perform a task from only a small number of examples.

Compare how 18 books explain few-shot learning →

Related concepts

• in-context learningkind of · In-context learning is a model’s adaptation of its behavior to examples, instructions, or other information in its input context without updating its parameters.
• prompt engineeringkind of · Prompt engineering is the deliberate design and refinement of instructions, context, examples, or other inputs to guide an AI model’s behavior or outputs.
• language modeluses · A language model is a computational model that represents patterns in linguistic tokens to estimate sequence probabilities and predict or generate language from context.
• promptuses · An input, typically containing instructions or context, supplied to an artificial-intelligence model to guide or elicit its output.
• zero-shot promptingcontrasts with · Zero-shot prompting asks a model to perform a task through instructions or a task description without providing task-specific examples or demonstrations.
• zero-shot learningcontrasts with · Zero-shot learning is a setting in which a model performs a task or predicts categories without task-specific examples during training, adaptation, or inference, relying on learned knowledge and available task information.

This explainer was generated by Manning's knowledge graph from the text of Build a Large Language Model (From Scratch). The section links point to the parts of the book it draws on.