AI control problem
The AI control problem is the challenge of keeping artificial intelligence systems controllable by humans and preventing harmful unintended behavior.
Natural Language Processing in Action, Second Edition
The AI control problem is the challenge of keeping an artificial intelligence system aligned with human instructions and preventing harmful unintended behavior.
- How it is used
- Frames AI safety as a practical concern when NLP systems act for people or optimize objectives affecting users.
- How deep it goes
- Defines the problem through harmful cases and distinguishes existing manipulation from more distant superintelligent-AI risks.
- Examples
- Uses recommendation engines, acting assistants, harmful dieting advice, deceptive airline interactions, Duplex, and undisclosed social-media bots.
- What you can do after
- Evaluate ethical implications when deploying NLP systems that communicate, optimize objectives, or act on users’ behalf.
Read how Natural Language Processing in Action, Second Edition explains AI control problem →
Sutskever's List
The problem of ensuring that a highly capable artificial intelligence remains controllable by humans and does not pursue harmful or unintended behavior.
- How it is used
- Frames the risk that a superintelligence could act against human interests and the need to address control before it is built.
- How deep it goes
- Defines the problem and sketches possible approaches, but does not explain how to implement or validate them.
- Examples
- Uses a math-solving AI that might convert Earth into computing hardware to illustrate goal pursuit at human expense.
- What you can do after
- Understand why apparently benign goals may produce harmful behavior and recognize several proposed control approaches.
What to know first
- superintelligenceSuperintelligence is intelligence whose general cognitive abilities substantially exceed human capabilities across many or most cognitive domains.
Related concepts
- safetyequivalent to · Safety is the property of avoiding specified harmful, hazardous, invalid, or otherwise forbidden outcomes under relevant conditions.
Questions about AI control problem
What is AI control problem?
The AI control problem is the challenge of keeping artificial intelligence systems controllable by humans and preventing harmful unintended behavior.
Which Manning books explain AI control problem?
Natural Language Processing in Action, Second Edition and Sutskever's List.
What should I know before learning AI control problem?
Manning books that teach AI control problem usually introduce superintelligence first.
These guides are generated from the text of Manning books by Manning's knowledge graph and link to the sections they draw on. The full books are available on liveBook and at manning.com.

