liveBook · Manning

AI agent

As explained in Delivering Machine Learning Projects: From design to deployment

An AI system is a computational system that performs tasks associated with artificial intelligence, potentially incorporating machine learning models.

An AI agent is a computational system that performs tasks associated with artificial intelligence and may incorporate machine-learning models. Treat deployment as a system-design problem, not merely a model choice: users need to understand capabilities, limitations, performance, and failure behavior. section 9.4.3

Design and delivery priorities include:

  • Make the system’s capabilities and limits visible before use. section 9.4.3
  • Communicate validated performance and relevant context during interaction. section 9.4.3
  • Provide correction, editing, and an accessible way to turn the system off after failure. section 9.4.3

Operational failure

An undisclosed aircraft system activated unexpectedly after sensor damage, causing pilots to encounter behavior they did not understand or know how to handle. A smart-building system illustrates the alternative: signage and public performance information establish awareness and expectations. section 9.4.3

Table 9.6 AI design guidelines (adapted from Amershi [3]) (view table figure)

When to use

Number

Design guideline

Initially

G1*

Make clear what the system can and can’t do.

G2*

As well as describing the system’s capabilities, clarify how well the system works.

During interaction

G3

Time proactive interactions with regard to the user’s current task and environment.

G4*

Show information relevant to the user’s current context.

G5*

Provide information in a way that the user can reflect on its social and cultural context.

G6

Make sure that information and instructions don’t reflect unfair social, gender, or racial biases.

When wrong

G7

Support efficient invocation to make it easy for the system work.

G8*

Make it easy to turn the system off, even after failing.

G9

Support efficient correction by making it easy to correct and edit the systems output and actions.

G10*

Scope services when in doubt. If the system isn’t sure, then get more information from the user or gracefully degrade the service when unclear about the users’ goals.

G11

Give users access to explanations about the system’s behavior.

Over time

G12

Remember recent interactions and maintain those in memory to make it easy for the user to use that memory to get service.

G13

Learn from user behavior; personalize the system to the user.

G14*

Adapt cautiously. Limit disruptive and unexpected changes to the system’s behavior, especially where the user has needs based on their expectations.

G15*

Encourage granular feedback. Enable users to provide meaningful information on performance and ensure that there is a mechanism to do that.

G16

Let users know when their input changes the behavior of the system.

G17*

Allow the users to monitor and control how the system behaves.

G18*

Notify users about changes. Inform the users when the AI’s capabilities are updated or enhanced so that they can modify their expectations.

Use the guideline table as a design checklist spanning initial disclosure, interaction, contextual fairness, correction, and shutdown. section 9.4.3

How Delivering Machine Learning Projects: From design to deployment covers it

How deep it goes
Defines design requirements and failure risks but stops short of model algorithms, training, or implementation internals.
Examples
Uses aircraft automation failure and smart-building signage as concrete design and governance examples.
What it assumes
Assumes familiarity with AI systems, machine learning projects, and user interface design.
What you can do after
Evaluate/understand an AI agent’s user-facing risks and translate them into disclosure, correction, and shutdown controls.
How it connects
Connects AI agents with machine learning models, user interface design, field testing, validation, and human control.
Cover of Delivering Machine Learning Projects: From design to deployment
liveBookmanning.com
Read Delivering Machine Learning Projects: From design to deployment on liveBook Get the book at manning.com

Also explained in

Compare how 33 books explain AI agent →

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Delivering Machine Learning Projects: From design to deployment. The section links point to the parts of the book it draws on.