A feedback loop turns evidence from deployed AI interactions into a change in system behavior. Collecting ratings is only the first step: the useful loop identifies a failure, changes how the system handles it, and checks whether the result improves user and business outcomes. section 10.4.1 section 10.4.3
Make the loop actionable:
- Offer optional thumbs-up/down, star ratings, and comments. Different formats trade user effort for detail; required ratings can worsen the experience and bias the data. section 10.4.1
- Track the feedback rate as well as ratings. Falling participation can warn of disengagement before satisfaction scores decline. section 10.4.1
- Check whether users complete tasks and whether downstream support tickets fall. A favorable rating alone does not establish that a problem was resolved. section 10.4.3
Close the loop on billing failures
When 23% of billing questions led to escalations, analysis found that simple billing examples had left the model poorly equipped for prorations and enterprise discounts. Specialized prompts detected complex questions and routed them to human agents with context. User-initiated escalations fell to 8%, while satisfaction rose from 3.2 to 4.1 out of 5. section 10.4.3
A loop can also reinforce harm. Fine-tuning on human-agent conversations that contain unequal service can reproduce and amplify that inequality; later AI interactions can then become biased training data. Treat feedback used for model improvement as something to examine, not automatically as a trustworthy target. section 11.2.1
Figure 10.5 Multimodal user feedback system showing three complementary mechanisms with varying participation rates and data richness. The declining feedback-rate trend serves as an early warning of user disengagement, typically preceding drops in overall satisfaction by two or three weeks.
Compare the effort and detail of the three voluntary feedback options, then inspect how a declining participation rate can precede lower satisfaction. section 10.4.1
