8 Evaluation is king: how to trust code you didn’t write
This chapter covers
- The trust deficit in AI-generated code
- Why AI code needs different evaluation than human code
- The hierarchy of confidence
- Practical techniques at each level
- The shift from code author to trust engineer
Michael Feathers in “Working Effectively With Legacy Code” defined legacy code as "legacy code is simply code without tests". Not old code. Not bad code. Code nobody can change safely, because nobody can tell whether a change made things better or worse
By that definition, AI generates legacy code. Not eventually, not after it rots: at the moment of generation. Even if it arrives with one-hundred percent code coverage, there is no author to interrogate, and nobody who has formed an opinion about what it should do.
There's an old joke in software engineering: "It works on my machine." For decades we lived in the gap between development and production, the constant tension between "I tested it" and "it actually works." AI-assisted development has made this joke obsolete, not because we solved the problem, but because we made it worse.
Now the phrase is: "The AI said it works."
8.1 Evaluation matters more than generation
8.1.1 Code is now produced faster than confidence
8.1.2 Scale of generation creates scale of potential debt
8.1.3 "Works for me" means nothing now
8.1.4 How the community measures "good AI code" (and why it's misleading)
8.1.5 The benchmark-reality gap
8.1.6 Where evaluation actually lives
8.2 Always-on verification
8.2.1 Compilation and type checking: a signal, not proof
8.2.2 Static analysis: linters, rule engines, and SAST
8.2.3 AI code review: rules, checklists, and heuristics
8.2.4 Dependency, license, and security scanning
8.2.5 Diff-based sanity checks: what actually changed?
8.2.6 The limits of always-on gates
8.3 Tests as contracts, not as coverage
8.3.1 Manual testing: still necessary, but different
8.3.2 Unit tests: the foundation that can betray you
8.3.3 TDD with AI and test-driven prompting
8.3.4 Code coverage: why AI can game it
8.3.5 Snapshot and golden testing: where they help and where they rot
8.3.6 Approval testing and visual regression
8.3.7 Key shift: tests describe intent, not implementation
8.4 Tests that break AI-generated code
8.4.1 Property-based testing: from examples to universals
8.4.2 Invariants, metamorphic relations, and conservation laws
8.4.3 Fuzzing: when random isn't random enough