Accuracy is the proportion of evaluated predictions or answers that are correct. It provides a common measure for deciding whether a prompt or adapted model improves a task, but the result depends on the test set and how correctness is scored. section 1.7 section 4.5.8
Establish a prompting baseline before training, then compare techniques on the same held-out set. Size the set for the decision: 50 examples can expose a large gap, while resolving a difference of a few points typically takes 200–500. Near 90% accuracy, a 50-example set has an approximate 95% confidence interval of ±8 percentage points. section 1.7
Ticket routing baseline
A twelve-category support-ticket classifier used eight examples in its prompt and scored 82% top-1 accuracy on 100 labeled tickets after four rounds of error-driven revisions. Top-1 accuracy counts a ticket as correct when the model’s first category choice is correct. The baseline avoided a training pipeline, though prompt examples add inference-token costs on every request. section 4.0
For fine-tuning, more training examples do not yield proportional accuracy gains. In one Llama-3-8B LoRA extraction experiment, accuracy rose from 70% to 88% with 200 examples, then increased gradually before flattening around 6,500 samples. Early examples should therefore be clean, representative, and diverse; later collection and labeling costs must be weighed against smaller marginal gains. section 3.3.2
Figure 3.3 Accuracy versus number of training examples

Inspect the accuracy curve to compare early gains with the later plateau. section 3.3.2
For retrieval-augmented generation (RAG), measure end-to-end accuracy alongside retrieval hit rate and answer faithfulness. Score representative held-out queries with exact match for structured outputs or an LLM judge for open-ended answers. A practical target is accuracy within five percentage points of Hit@k: if retrieval hits 80% and answers are correct 75% of the time, synthesis is doing its job. A larger gap suggests the model is not using available retrieved context effectively. section 4.5.8 section 4.10
Choose the correctness measure for the output. Token-F1 can reward shared vocabulary even when an answer adds an incorrect instruction, and penalize a useful paraphrase. For distillation, an LLM judge can supplement checkpoint metrics by assessing correctness, completeness, and coherence on the final candidate. section 7.2.3
Catalog tagging
Lowe’s fine-tuned an OpenAI model on internal product data and reported a 20% improvement in product-tagging accuracy. Better tags address missed search matches and manual catalog QA, although the published case did not provide a payback period. section 5.7.1
















































