Safety
Safety is the property of a system or process whereby specified harmful, hazardous, invalid, or otherwise forbidden outcomes are avoided under defined conditions.
Sutskever's List
AGI safety is the field concerned with preventing artificial general intelligence from causing unintended, harmful, or uncontrolled outcomes.
- How it is used
- Frames both present-day AI harms and precautionary decisions about more capable systems.
- How deep it goes
- Defines unsafe systems through possible sources of harm and works through incidents and decisions, but gives no technical safety procedure.
- Examples
- Uses an airline chatbot’s fare advice, an automated-driving failure, GPT-2’s release debate, and the Altman firing.
- What you can do after
- Evaluate/understand it by distinguishing current harms from speculative risks and recognizing the costs of preemptive safety decisions.
Rust Servers, Services, and Apps
The property of software or a programming language preventing classes of invalid or hazardous behavior.
- How it is used
- Frames program safety as a benefit of Rust for application backends and software infrastructure.
- How deep it goes
- Distinguishes three safety aspects and explains compile-time type checking, while deferring detailed memory-safety analysis.
- Examples
- Contrasts manual memory management in C and C++ with garbage-collected languages including Java, C#, Python, Ruby, and Go.
- What you can do after
- Understand how compile-time type checking helps prevent a class of runtime errors.
Think Distributed Systems
A system guarantee that a specified bad event or condition will never occur.
- How it is used
- Uses safety properties to specify forbidden system behaviors and judge correctness in distributed transactions, failure handling, and consensus.
- How deep it goes
- Defines safety formally, models reachable states, and applies it to transactions and consensus, while separating it from liveness and system-model limits.
- Examples
- Illustrates safety with a two-participant transaction, a state variable constrained to 1 or 2, coordinator failure, and consensus decision rules.
- What you can do after
- Evaluate distributed-system guarantees by identifying forbidden states, conflicting decisions, failure responses, and the boundary between safety and liveness.
Introduction to Generative AI, Second Edition
Harmlessness is the property of an AI system's behavior that avoids causing or enabling harm.
- How it is used
- Frames AI behavior around preventing unwanted outputs and managing harms in language-model and agent deployments.
- How deep it goes
- Explains safeguards, filtering, oversight, and trade-offs, but stops short of implementation detail or a complete theory of advanced-agent safety.
- Examples
- Uses toxic or sexist outputs, response blocking, toxicity classification, and supervised agents making safety-sensitive decisions.
- What you can do after
- Evaluate safety trade-offs between useful responses, harmful outputs, filtering errors, and human control.
Read how Introduction to Generative AI, Second Edition explains safety →
Generative AI in Action
AI safety concerns the prevention and mitigation of harmful, unreliable, or uncontrollable behavior by AI systems.
- How it is used
- Frames safety as a deployment concern requiring validation, privacy protection, fairness, and proactive prevention of misuse.
- How deep it goes
- Organizes safety into four categories and identifies concrete harms, but does not develop detailed implementation procedures or mathematical methods.
- Examples
- Uses RAG grounding, controlled enterprise data, source attribution, privacy compliance, deepfakes, cyberattacks, misinformation, and incorrect information.
- What you can do after
- Evaluate/understand AI risks and apply grounded data, validation, fairness analysis, and misuse controls to enterprise deployments.
Natural Language Processing in Action, Second Edition
AI safety is the field concerned with preventing artificial intelligence systems from causing harmful outcomes and maintaining reliable human control, including over long-term risks.
- How it is used
- Frames harmful autonomous behavior as a control problem requiring attention to both current failures and future existential risks.
- How deep it goes
- Defines the safety–ethics distinction and illustrates harms, but stops short of presenting technical mitigation methods.
- Examples
- Uses harmful dieting advice, deceptive airline interactions, attention-driven recommendations, geopolitical conflict, and human disempowerment as risk examples.
- What you can do after
- Evaluate AI harms by distinguishing immediate ethical failures from long-term control and existential-safety concerns.
Read how Natural Language Processing in Action, Second Edition explains safety →
Rust in Action: Systems programming concepts and techniques
Software safety is the property of software behavior being constrained to avoid invalid, harmful, or erroneous outcomes.
- How it is used
- Frames Rust’s central promise: preventing invalid behavior while retaining control and systems-level performance.
- How deep it goes
- Explains compile-time memory safety, concurrency errors, and zero-cost checks, but does not detail the underlying checking mechanisms.
- Examples
- Uses invalid data access, insecure IoT input parsing, infrequently updated firmware, and multiple integer representations.
- What you can do after
- Evaluate how Rust’s compile-time guarantees protect memory and concurrency without requiring runtime safety costs.
Read how Rust in Action: Systems programming concepts and techniques explains safety →
The Art of Code
A safety guarantee is an assurance that specified invalid or unsafe program conditions are prevented or detected.
- How it is used
- Evaluates code by how early it exposes missing cases and invalid assumptions, preferably through compiler support.
- How deep it goes
- Compares runtime-risk reduction and compiler-checked exhaustiveness, stopping short of treating safety as an isolated quality.
- Examples
- Contrasts Kotlin if/else promotional-message selection with Haskell algebraic data types and pattern matching.
- What you can do after
- Evaluate alternative implementations by their handling of invalid assumptions, missing variants, casts, and compiler-checkable cases.
Related concepts
- state machine safetyincludes · State machine safety is the property that no two processes apply different log entries at the same index to their state machines.
- livenesscontrasts with · Liveness is a correctness property requiring that a system eventually makes specified progress or produces a desired event or condition.
- securitycontrasts with · The protection of systems, information, resources, and operations against unauthorized access, use, disclosure, alteration, disruption, or destruction.
These guides are generated from the text of Manning books by Manning's knowledge graph and link to the sections they draw on. The full books are available on liveBook and at manning.com.