liveBook · Manning

Mechanical verification

As explained in Vibe Engineering

Verification performed by automated checkers against explicit artifacts such as types, tests, contracts, and schemas.

Mechanical verification uses automated checkers against explicit artifacts such as types, tests, contracts, and schemas. It can make verification cheaper as code generation accelerates, but a passing check only establishes what the artifact actually specifies. section 1.5

A green result is not proof that code does what the business meant. If the same model generates both code and tests, it can miss the same edge case twice. A model reviewing another model’s changes can widen the review surface, but is not an independent second opinion. section 1.5 section 5.3.2

Refund-policy boundaries

An executable refund specification can check that a request within 30 days is approved, one after 30 days is rejected, and partial refunds cannot exceed the original order amount. These checks are useful only if the stated rules capture the intended policy. section 1.5

Listing 5.4 Pre-commit hook example
#!/bin/bash
# .git/hooks/pre-commit
set -euo pipefail
 
DIFF=$(git diff --cached)
[ -z "$DIFF" ] && exit 0
 
# Deterministic checks block. This is the real gate.
gitleaks protect --staged --redact || exit 1
 
# The model advises. It cannot block on its own.
VERDICT=$(printf '%s' "$DIFF" | claude -p "Review this diff against our
conventions: error handling, test coverage for new public functions,
naming. Reply with exactly one line of JSON and nothing else:
{\"verdict\":\"clean\"} or {\"verdict\":\"concerns\",\"notes\":\"...\"}")
 
if [ "$(printf '%s' "$VERDICT" | jq -r '.verdict')" = "concerns" ]; then
    echo "AI reviewer raised concerns (advisory, not blocking):"
    printf '%s' "$VERDICT" | jq -r '.notes'
fi
exit 0

A pre-commit hook makes a deterministic secret scan the blocking gate while treating the model’s review as advisory. section 5.3.2

Put human attention into specifications and adversarial, mutation-resistant test plans rather than relying on exhaustive line-by-line review. Run spec-derived tests and policy gates consistently in CI/CD, and maintain shared verification assets with clear ownership. section 1.5

How Vibe Engineering covers it

How deep it goes
Defines mechanical checks and contrasts them with semantic verification, stopping short of treating passing checks as proof of business intent.
Examples
Uses refund-policy boundary cases and a pre-commit secret scan with advisory AI review.
What you can do after
Evaluate verification results against business intent and design test plans to counter shared model blind spots.
How it connects
Connects spec-driven development, semantic verification, CI/CD, and AI code review.

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Vibe Engineering. The section links point to the parts of the book it draws on.