A small change is a narrowly scoped modification to a conversational AI system. It limits the change’s blast zone, makes its effects easier to evaluate, and creates a new learning opportunity. Because a change can help, do little, or make performance worse, predictable scope matters. section 1.3.2
Use a small-change cycle to improve a system without losing track of causes:
- Measure a baseline and connect the problem to a meaningful business or completion metric. section 1.3.2
- Identify one broken, weak, or non-optimal behavior, such as excessive transfers to agents. section 1.3.2
- Implement one focused remedy, such as updating the worst-performing intent or copyediting one dialogue step. section 1.3.2
- Deploy, record the effect on the original problem, and undo the change if it performs poorly. section 1.3.2
One intent instead of every intent
If users frequently say the bot does not understand them, retrain the single worst-performing intent rather than retraining all intents. If a process has low completion, change one step rather than rearranging the entire flow. A result that declines is then easier to trace to its likely cause. section 1.3.2 section 5.1.2
Small scope does not guarantee small effects. Classifier changes can affect multiple intents, and generative AI can produce substantially different results after minor prompt-format changes. Test each change and retain a tolerance for rework. section 5.1.2 section 10.1.1
Figure 1.11 Making many small changes—like retraining one intent at a time—has a smaller “blast zone” for each change, bringing quicker value and more learning.
Compare the smaller blast zone, quicker value, and greater learning associated with changing one intent at a time. section 1.3.2
