liveBook · Manning

Prompt engineering

As explained in LLM Customization and Fine-Tuning

Steering a model's behavior through its input instructions or examples without updating its weights.

Prompt engineering steers a model through instructions, examples, and output requirements without changing its weights. It is a fast first step for tasks the model can already handle: development may take hours rather than weeks, but examples add input tokens to every request. section 1.6.2 section 4.0

Build and test a baseline before committing to training:

  • Write a clear instruction and provide the context the task requires. Add a few representative input-output examples to demonstrate the desired pattern. section 1.6.2 section 5.1.1
  • Specify the output shape. When format is the only problem, a response schema enforced at decode time can produce valid JSON or a fixed template without training. section 1.6.2 section 5.1.8
  • Measure accuracy on held-out examples and check run-to-run variability. Fifty examples can reveal a catastrophic gap; distinguishing differences of a few points typically needs 200 to 500. section 1.7 section 4.0
  • Inspect errors and revise the prompt, then compare alternatives on the same held-out set. Escalate when the measured quality gap persists, rather than assuming fine-tuning is necessary. section 4.0 section 1.7 section 4.9

Routing support tickets

For twelve-category ticket routing, an eight-shot prompt on Qwen3-4B-Instruct-2507 reached 82% top-1 accuracy on 100 labeled tickets after four rounds of error-driven revisions. The examples were weighted toward high-volume categories; no training pipeline or new checkpoint was required. section 4.0

Prompting cannot supply missing internal knowledge unless that knowledge is included in the request, and it may not reliably enforce behavior the base model resists. Use retrieval-augmented generation for changing documents or verifiable citations; consider fine-tuning when simpler approaches leave a measurable behavioral gap. section 1.6.2 section 5.1.8 section 6.13

Low setup cost does not guarantee low operating cost. In an illustrative eight-shot prompt, about 1,800 input tokens and 50 output tokens cost roughly 0.002X per request, where X is the input price per million tokens and output costs 4X. At high volume, compare that recurring cost with training and hosting; also test whether fewer examples preserve quality. section 4.7.3

Table 4.6 ICL vs. fine-tuning crossover at four volumes (illustrative; units of X per million tokens)
Daily volume
Daily ICL spend
LoRA training one-off
LoRA daily hosting
Break-even
100 req/day
~0.2 X
~200 X
~10 X
Never (daily hosting alone exceeds API spend)
10K req/day
~20 X
~200 X
~10 X
About 3 weeks
100K req/day
~200 X
~200 X
~10 X
1 to 2 days
1M req/day
~2,000 X
~1,000 X (full SFT)
~100 X
Under 1 day

Compare illustrative prompting and fine-tuning costs across request volumes; recalculate for actual prices and caching. section 4.7.3

How LLM Customization and Fine-Tuning covers it

How deep it goes
Works through prompt design, held-out validation, variability checks, and volume-based cost comparisons.
Examples
Uses an eight-shot support-ticket classifier and a comparison of format prompting with an IT-support LoRA adapter.
What you can do after
Practice designing and evaluating a few-shot prompt, then compare its quality and recurring cost with alternatives.
How it connects
Connects in-context learning and structured output with retrieval-augmented generation, LoRA, and supervised fine-tuning.

Also explained in

Compare how 51 books explain prompt engineering →

Related concepts

This explainer was generated by Manning's knowledge graph from the text of LLM Customization and Fine-Tuning. The section links point to the parts of the book it draws on.