Reliability is the degree to which a system consistently produces dependable results. For an LLM-based tool, that means giving accurate answers across expected and unexpected inputs without creating costly surprises. section 8.1 section 1.9
Reliability matters because:
- Unexpected inputs can produce incorrect or otherwise unexpected output. section 8.1
- Error costs and liability can outweigh the apparent convenience of automation. section 8.1
- Hidden remediation work can make a seemingly simple chatbot process more complex. section 8.1
Customer support chatbot
Replacing FAQs, email tickets, or phone support with an LLM chatbot may appear streamlined, but incorrect answers can reduce customer retention. A trial deployment helps evaluate the consequences before replacing support wholesale. section 8.1
Figure 8.1 When looking at the process diagram, it would seem like replacing FAQs, email tickets, and support numbers could be simplified and streamlined with an LLM-based chatbot. However, the folly of this view is that the process is incomplete. The potential errors and remediation processes required to ensure an LLM will perform accurately are hidden and create more complexity.
Use the process diagram to see how chatbot simplicity hides accuracy checks, error handling, and remediation work. section 8.1
