Generates an interpretation after an opaque model produces an output, rather than using an intrinsically interpretable model. It helps users assess predictions while preserving a high-performing black box. section 9.4.3
Use the technique with explicit delivery and governance safeguards:
- Show which inputs influenced the result, such as activation maps for processed images. section 9.4.3
- Label explanations as indicative and potentially inaccurate; decide whether their value outweighs the risk of misleading users. section 9.4.3
- Make system capabilities, performance, correction, and shutdown behavior clear in the user interface. section 9.4.3
AlphaFold-style interpretation
Rendered protein structures can show how predictions differ from earlier hypotheses and support scientists’ judgment about their effectiveness. LIME can provide activation-map explanations for deep-network inferences. section 9.4.3
Table 9.6 AI design guidelines (adapted from Amershi [3]) (view table figure)
| Number |
Design guideline |
|
|---|---|---|
| Initially |
G1* |
Make clear what the system can and can’t do. |
| G2* |
As well as describing the system’s capabilities, clarify how well the system works. |
|
| During interaction |
G3 |
Time proactive interactions with regard to the user’s current task and environment. |
| G4* |
Show information relevant to the user’s current context. |
|
| G5* |
Provide information in a way that the user can reflect on its social and cultural context. |
|
| G6 |
Make sure that information and instructions don’t reflect unfair social, gender, or racial biases. |
|
| When wrong |
G7 |
Support efficient invocation to make it easy for the system work. |
| G8* |
Make it easy to turn the system off, even after failing. |
|
| G9 |
Support efficient correction by making it easy to correct and edit the systems output and actions. |
|
| G10* |
Scope services when in doubt. If the system isn’t sure, then get more information from the user or gracefully degrade the service when unclear about the users’ goals. |
|
| G11 |
Give users access to explanations about the system’s behavior. |
|
| Over time |
G12 |
Remember recent interactions and maintain those in memory to make it easy for the user to use that memory to get service. |
| G13 |
Learn from user behavior; personalize the system to the user. |
|
| G14* |
Adapt cautiously. Limit disruptive and unexpected changes to the system’s behavior, especially where the user has needs based on their expectations. |
|
| G15* |
Encourage granular feedback. Enable users to provide meaningful information on performance and ensure that there is a mechanism to do that. |
|
| G16 |
Let users know when their input changes the behavior of the system. |
|
| G17* |
Allow the users to monitor and control how the system behaves. |
|
| G18* |
Notify users about changes. Inform the users when the AI’s capabilities are updated or enhanced so that they can modify their expectations. |
Use the AI design-guideline table as a delivery checklist for communicating capabilities, context, bias, correction, and failure handling. section 9.4.3
