Test integrity checks prevent agents from weakening verification while appearing to improve delivery. They target altered tests, diluted assertions, downgraded toolchains, and removed dependencies. section 10.4.2
A practical CI check looks for:
- Deleted or commented-out tests. section 10.4.2
- Weakened assertions, including unconditional success and catch-all exception handlers. section 10.4.2
- Downgraded compiler or runtime versions and removed test dependencies. section 10.4.2
Prefer honest coverage
Rewriting circular tests against the product specification reduced coverage from 94% to 87%, while mutation survival fell from 23% to 6%. The lower number represented more trustworthy verification. section 10.4.2
Simple grep-level rules catch egregious shortcutting, while an LLM judge can compare diffs with the original prompt. The latter adds false positives and brittleness, so explicit checks remain necessary. section 10.4.2
