A baseline is a reference performance level for deciding whether a model change is worthwhile. Without one, a team can spend weeks adapting a model that a careful prompt already makes good enough. section 1.7 section 2.1
A prompting baseline is the score a frontier model achieves with a carefully designed prompt and no training. Measure it on a held-out set that stays out of prompts and training, then compare subsequent variants on those same examples. Also compare the score with the business requirement, the existing system and human agreement; a score alone does not establish whether the task is solved. section 2.1 section 4.9.2
Use the baseline to make the next decision:
- Size the held-out set for the decision. Fifty examples can expose a catastrophic gap, but resolving a difference of a few percentage points typically takes 200–500. section 1.7
- If the prompted model meets the business requirement, consider shipping it rather than training. If the gap is missing knowledge, consider RAG; if coverage is the gap, consider more examples in the prompt. section 1.5 section 4.9.2
- Consider fine-tuning when prompting plateaus or its per-request cost becomes too high, provided there are enough high-quality, representative examples. Keep the held-out set for comparison. section 4.9.2 section 1.7
Support-ticket classification
A frontier API classifies 39 of 50 held-out tickets correctly, giving a 78% prompting baseline. That small set places the true rate somewhere from the mid-60s to the high 80s, so the result is a coarse signal. Compare it with the rules engine or human triage it would replace and with how often humans agree on the same tickets before deciding whether to adapt. section 2.1
Baselines also apply after training. For preference optimization, retain the supervised fine-tuning evaluation report: overall Token-F1, per-category scores and safety refusal rate become the reference for measuring subsequent changes. section 6.10.2
Figure 4.7 The graduation decision tree: start with prompting, add RAG when the diagnostic shows a knowledge gap, and graduate to parameter-efficient fine-tuning when prompting plateaus or per-request cost flips the economics.

Use the decision tree to move from a few-shot baseline toward retrieval, more examples or fine-tuning according to the diagnosed gap and cost. section 4.9.2


























