liveBook · Manning

Verification

As explained in Vibe Engineering

The systematic evaluation of an implementation or output against specified requirements and expected behavior.

A green pipeline can still approve the wrong behavior. If generated code truncates a discount to 4.99 and its generated test expects 4.99, both agree—but neither verifies the required result of 5.00. Verification evaluates behavior against requirements, not merely against an implementation’s own output. section 10.4.1

For AI-generated code, verification is the work between producing an implementation and trusting it. Write an executable specification before generation where practical: tests and contracts should express human intent, including edge cases. High coverage or passing tests alone do not establish that the right behavior was specified. section 1.8 section 8.3

Build confidence through complementary gates:

  • Check locally that the code runs and roughly behaves as expected; treat this as an iteration signal, not proof. section 8.1.6
  • Review the implementation for understandable behavior, conventions, and visible problems. Run automated tests, compilation, linting, and security checks in CI. section 8.1.6
  • Test in staging for integration and performance problems, then use production feedback to observe behavior under real conditions. Find failures earlier when possible to reduce user impact. section 8.1.6

Mechanical checks can establish whether code satisfies the assertions provided. Semantic checks ask whether those assertions capture what the business actually means. An AI judging its own output can repeat the same misunderstanding, so use machine-checkable specifications and human judgment rather than self-review alone. section 1.8 section 3.6.3

A refund policy as a contract

Specify that a refund within 30 days is allowed, a later request is rejected, and partial refunds cannot exceed the original amount. Generate implementation and tests against these rules, then probe the boundary around 30 days and review assumptions about time zones and rounding. section 1.5

Feature: Refund policy
  Rule: Time-bounded, single-order refunds
    Example: Within 30 days is allowed
      Given an order "O1" purchased at 2025-08-01T10:00:00Z
      When a refund of 25.00 USD is requested at 2025-08-28T09:59:00Z
      Then the refund is approved and remaining refundable equals 75.00 USD
 
    Example: After 30 days is rejected
      Given an order "O2" purchased at 2025-08-01T10:00:00Z
      When a refund is requested at 2025-08-31T10:00:01Z
      Then the request is rejected with reason "expired"
 
    Example: Partials must not exceed the original
      Given prior refunds of 90.00 USD on a 100.00 USD order
      When a refund of 20.00 USD is requested
      Then the request is rejected with reason "exceeds-remaining"

These Given–When–Then cases make approval, expiry, and remaining-balance rules concrete enough to check. section 1.5

A parser that passes happy paths

A generated markup parser may handle isolated headers, bold text, and links while breaking on nested formatting or parentheses in link URLs. Tests should probe interactions and malformed inputs, not just demonstrate each feature separately. section 8.2

Match verification effort to risk. Cheap static checks and unit tests form a baseline; integration tests matter at system boundaries, while property testing, fuzzing, mutation testing, or formal methods may justify their cost for complex or critical behavior. A checklist is only a floor: understand failures, fixes, and what remains unverified. section 8.8.2 section 8.8.5

How Vibe Engineering covers it

How deep it goes
Develops a gated verification process, contrasts mechanical and semantic checks, and matches techniques to system criticality.
Examples
Uses refund-policy boundary cases, a markup parser, and a circular discount-rounding test to expose verification failures.
What you can do after
Evaluate generated code using specification-derived checks, layered gates, and risk-based judgment.
How it connects
Connects executable specifications, test-driven development, continuous integration, property-based testing, and mutation testing.

Also explained in

Compare how 9 books explain verification →

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Vibe Engineering. The section links point to the parts of the book it draws on.