Accuracy describes how often evaluated outputs are correct, or how closely they meet a specified reference or criterion. For a chatbot, translation system, or summarizer, the criterion must reflect the task: a single accuracy figure cannot capture every aspect of generated text. section 12.1 section 12.0
Make an accuracy assessment useful by pairing it with a clear evaluation plan:
- Test the model on its intended tasks, using unambiguous, fair prompts and recorded criteria and results. section 12.1
- Compare an adapted model with a baseline, and combine automated measures with manual review and user feedback. section 9.3.2
- Monitor deployed performance and the quality of data sources; revisit the assessment as conditions change. section 11.1 section 12.1
Translation after fine-tuning
Compare a fine-tuned translation model with its pre-fine-tuning version or another translation model. Assess translation accuracy alongside fluency to see whether adaptation improved the intended task rather than only one measure. section 9.3.2
Accuracy is not a substitute for other checks. A content filter, for example, also needs precision, recall, false-positive and false-negative rates, user feedback, and bias checks. Choose measures that reveal the errors that matter for the application. section 13.5.3
