Negative testing evaluates whether a system correctly rejects, declines, or otherwise handles inputs that it should not answer or process.
A conversational system can fail by answering confidently when a user asks something outside its purpose. Negative testing checks whether it recognizes such cases and avoids attempting an inappropriate answer. section 4.4.2
Candidate negative-test utterances include:
Intelligible questions entirely unrelated to the bot’s domain, such as pop-culture trivia sent to an electric-utility account assistant. section 4.4.2
Ambiguous phrases such as “account,” “driver license,” “that one,” or “2,” when no single intent can be assigned. section 4.4.2
Unrecognized topics that may be routed to an LLM, while remaining outside the ordinary classifier accuracy test set. section 4.4.2
How Effective Conversational AI covers it
How deep it goes
Distinguishes off-topic and ambiguous utterances from ordinary classifier tests, but does not assign them golden intents.
Examples
Uses utility-account questions, pop-culture trivia, and ambiguous utterances such as “account,” “that one,” and “2.”
What it assumes
Assumes a defined domain, bot purpose, and classifier test set against which unsuitable utterances can be separated.
What you can do after
Diagnose whether a bot appropriately rejects, defers, or otherwise handles off-topic and unclear user utterances.
How it connects
Connects domain boundaries with classifiers, intents, and LLM handling of unrecognized topics.
testuses · A test is a procedure or check that evaluates behavior using specified inputs or evidence against expected results or criteria.
This explainer was generated by Manning's knowledge graph from the text of Effective Conversational AI. The section links point to the parts of the book it draws on.
