liveBook · Manning

In-context learning

As explained in LLM Customization and Fine-Tuning

A technique in which a model uses examples and instructions in its prompt to perform a task without updating its weights.

In-context learning (ICL) conditions a model on instructions and examples in its prompt so it follows an input-output pattern without updating its weights. It is a low-overhead first option when a task may not need fine-tuning: the model selects a pattern it already learned, while the application pays to send the context on each request. section 4.0 section 4.1

Figure 4.1 The in-context learning ladder: zero-shot, one-shot, few-shot, many-shot, and RAG sit on a single rung of the adaptation continuum, with engineering cost climbing gradually within the rung before chapter 5's parameter-update techniques take over.

The ladder places zero-shot through many-shot prompting and retrieval-augmented generation on the no-weight-update rung, with engineering cost increasing across approaches. section 4.1

The practical question is whether a prompt meets the task’s quality bar at an acceptable operating cost. Clear instructions and a few representative examples can be tried in hours rather than a training cycle. A response schema can also enforce a required output shape without a weight update. section 1.6.2 section 2.1

A practical evaluation sequence:

  • Establish a baseline and compare it with the system or human process being replaced, including the frequency of human agreement. section 2.1
  • Add hand-picked examples with correct labels, keeping them separate from test data. Start with a few rather than assuming that more examples are always better. section 1.6.2 section 2.1
  • Score prompt variants on the same held-out tickets. Grow the evaluation set to a few hundred when a small difference could decide the next investment. section 2.1
  • Compare the remaining errors with their downstream cost. A soft routing hint may tolerate accuracy that would make human re-triage too expensive. section 2.1

Ticket classification

A twelve-category help-desk classifier used Qwen3-4B-Instruct-2507 with eight hand-picked examples, weighted toward high-volume categories. After four short rounds addressing observed errors, it reached 82% top-1 accuracy on 100 labeled tickets, without a training pipeline or a separate checkpoint to host. section 4.0

ICL has low fixed cost but recurring token cost and latency: the instructions, examples, and query travel with every request. In a 50-shot prompt, input tokens can dominate output tokens by an order of magnitude. A self-hosted fine-tuned model instead incurs training and hosting costs that can be amortized across requests, with shorter prompts per call; volume therefore affects the break-even decision. section 4.7.1 section 2.1

Prompting alone cannot teach knowledge the model never saw during pre-training or reliably enforce behavior the base model resists. For changing knowledge or clickable citations, retrieval-augmented generation adds relevant document chunks to the prompt without retraining. If a carefully evaluated prompt still misses the required behavior, a weight-update technique such as LoRA may be worth considering. section 1.6.2 section 2.1

How LLM Customization and Fine-Tuning covers it

How deep it goes
Explains prompt design, held-out evaluation, cost tradeoffs, and escalation criteria, with a worked ticket-classification workflow.
Examples
Uses eight labeled examples and 100 held-out tickets to evaluate a twelve-category help-desk classifier.
What you can do after
Build a few-shot classifier and evaluate whether its accuracy and operating cost justify deployment or further adaptation.
How it connects
Connects prompt engineering, few-shot prompting, retrieval-augmented generation, response schemas, and fine-tuning.

Also explained in

Compare how 3 books explain in-context learning →

Related concepts

This explainer was generated by Manning's knowledge graph from the text of LLM Customization and Fine-Tuning. The section links point to the parts of the book it draws on.