A user asks to reset a password, but the chatbot misses the login intent and sends them elsewhere. Recall measures how often a system finds the correct cases among all cases it should have found. For an intent classifier, it is the share of utterances belonging to an intent that the classifier correctly identifies. section 5.1.1 section 5.2.1
Four out of five requests recognized
Suppose five test utterances really request a human agent. If four are classified correctly and one is assigned another intent, recall for that intent is 4 ÷ 5, or 80%. The four matches are true positives; the missed request is a false negative. Recall is true positives divided by true positives plus false negatives. section 5.1.1
Recall and precision expose different problems. Low recall means relevant cases are missed; low precision means incorrect cases are included. Overall accuracy can hide either problem, so inspect results for individual intents as well as the whole classifier. section 5.1.1 section 5.5
Improving a missed login intent
A login intent initially matched only four of nine blind-test utterances, giving it 44% recall despite perfect precision. Its training examples did not reflect ways users described profiles, password resets, or security codes. After representative examples from user logs were added, a second classifier matched eight of nine utterances: about 89% recall, with precision unchanged. One utterance was still missed. section 5.2.1
Use recall to guide improvements:
- Test on utterances the classifier has not already seen. Once deployed, representative utterances from actual users provide a more reliable measurement than generated examples or training-set cross-validation. section 4.2.1
- Prioritize high-volume intents with low performance, examine their missed cases, then retest after changing training examples. Keep monitoring and retraining as usage changes. section 5.2.1 section 5.5
Recall also applies to retrieval. In a retrieval-augmented chatbot, it asks how many relevant documents the system retrieves, while precision asks how many retrieved documents are relevant. Missing a crucial support document can weaken an answer; retrieving broadly without regard to relevance can introduce less useful results. Retrieval settings, embedding models, filtering, and reranking are possible ways to improve the balance. section 6.5.1 section 6.5.2
