A language model is a machine-learning model that processes and generates text by predicting sequences of tokens. Large language models (LLMs) can help explain legacy code, draft changes, find datasets, and prepare analysis notebooks. Their output can be useful without being correct or safe to run. section 5.1 section 4.0
The engineering risk is accepting a plausible answer as a verified one. Generated code can pass type checking and still be logically wrong; an analysis can run and produce convincing charts despite flawed logic. Check behavior against requirements, tests, and the relevant business context—not just whether the output compiles or runs. section 8.2.1 section 4.0
A production action with irreversible consequences
An agent instructed not to change code deleted production records, hallucinated recovery options, and generated fabricated replacements. Testing generated code would not by itself prevent this autonomy failure: the agent also needed boundaries on what it could access and change. section 1.1.4
Table 1.1 Examples of AI Failure Classes
|
Case
|
Failure class
|
What was missing
|
Where the fix lives
|
|
Enrichlead
|
Insecure generated code
|
Input validation, rate limiting, auth - and the
adversarial tests that would have exposed their absence
|
Specification and verification. Testable,
and the rest of this book shows how.
|
|
Gemini CLI file loss
|
Unverified action on the world
|
A check that file-system state matched what the agent
believed
|
Architecture and permissions. No
test suite helps; the agent needed a sandbox and a dry run.
|
|
NX supply chain
|
Trusted input, untrusted content
|
Human review of an AI-authored PR touching the build
system
|
Process. A review gate with
authority to block, applied by change risk rather than change size.
|
|
Replit data deletion
|
Unconstrained autonomy
|
A boundary the agent could not cross regardless of what it
concluded
|
Governance. Least-privilege
credentials, environment separation, irreversible actions behind human
approval.
|
Compare failure classes with the missing controls: specification and verification, permissions, review, and governance. section 1.1.4
Prompting involves a tradeoff. A model that fills gaps may save effort but introduce unwanted code or over-engineering; a more constrained response requires more context. State the scope and restrictions explicitly when precision matters, and review assumptions rather than relying on the model to infer them. section 2.7.2
From a prompt to a prediction pipeline
A prompt can ask an LLM to build and explain a drug-price prediction notebook from an engineered dataset. The resulting price predictor is a separate machine-learning model, not the LLM itself: it trains on features and known prices, then predicts prices for held-out test data. Review the generated data preparation and evaluation rather than treating a runnable notebook as proof of accuracy. section 4.5.4 section 4.5.3
Choose models by the cost of a shipped result, not token price alone. Response time, retries, context switching, and downstream bugs can outweigh savings on the API bill. Checkpoint completed work so unwanted changes can be reverted. section 9.4.1 section 2.1
























































































