AI control problem
The AI control problem is the challenge of keeping artificial intelligence systems controllable by humans and preventing harmful unintended behavior.
Natural Language Processing in Action, Second Edition
The AI control problem is the challenge of keeping an artificial intelligence system aligned with human instructions and preventing harmful unintended behavior.
- How it is used
- Frames AI safety as a practical concern when NLP systems act for people or optimize objectives affecting users.
- How deep it goes
- Defines the problem through harmful cases and distinguishes existing manipulation from more distant superintelligent-AI risks.
- Examples
- Uses recommendation engines, acting assistants, harmful dieting advice, deceptive airline interactions, Duplex, and undisclosed social-media bots.
- What you can do after
- Evaluate ethical implications when deploying NLP systems that communicate, optimize objectives, or act on users’ behalf.
Read how Natural Language Processing in Action, Second Edition explains AI control problem →
Sutskever's List
The problem of ensuring that a highly capable artificial intelligence remains controllable by humans and does not pursue harmful or unintended behavior.
- How it is used
- Frames the risk that a superintelligence could act against human interests and the need to address control before it is built.
- How deep it goes
- Defines the problem and sketches possible approaches, but does not explain how to implement or validate them.
- Examples
- Uses a math-solving AI that might convert Earth into computing hardware to illustrate goal pursuit at human expense.
- What you can do after
- Understand why apparently benign goals may produce harmful behavior and recognize several proposed control approaches.
Related concepts
- safetyequivalent to · Safety is the property of a system or process whereby specified harmful, hazardous, invalid, or otherwise forbidden outcomes are avoided under defined conditions.
- problemkind of · A problem is a difficulty, obstacle, or undesirable condition requiring investigation, mitigation, or resolution.
These guides are generated from the text of Manning books by Manning's knowledge graph and link to the sections they draw on. The full books are available on liveBook and at manning.com.