Conversational performance measures how well an LLM communicates in dialogue with human-like, context-aware, coherent responses that stay consistent and adapt to topic and style changes. section 7.7
It is treated as a practical evaluation target for instruction-fine-tuned chatbots, alongside multiple-choice benchmarks, human preference comparisons, and automated LLM-based judging. section 7.7
The treatment uses conversational performance to focus evaluation on:
- relevant and coherent responses. section 7.7
- consistency across turns and topics. section 7.7
- automatic scoring with another LLM when human review is too slow. section 7.7
Held-out response evaluation
A test set is expanded into model responses, saved to JSON, and then compared against correct outputs for later automated evaluation of conversational quality.
