liveBook · Manning

Semantic verification

As explained in Vibe Engineering

A verification assessment of whether a system satisfies the intended real-world business meaning and context.

A green test suite can still approve the wrong behavior. Semantic verification asks whether a system does what the business intended, not merely whether it passes mechanical checks such as types, tests, contracts, and schemas. Mechanical checks become cheaper when code generation becomes cheaper; assessing business meaning still requires judgment. section 1.5 section 1.8

Generating both code and tests with the same model can create a closed loop: shared blind spots may miss the same edge case twice. A model reviewing another model’s changes can widen the review surface, but it is not an independent second opinion. section 1.5 section 5.3.2

Feature: Refund policy
  Rule: Time-bounded, single-order refunds
    Example: Within 30 days is allowed
      Given an order "O1" purchased at 2025-08-01T10:00:00Z
      When a refund of 25.00 USD is requested at 2025-08-28T09:59:00Z
      Then the refund is approved and remaining refundable equals 75.00 USD
 
    Example: After 30 days is rejected
      Given an order "O2" purchased at 2025-08-01T10:00:00Z
      When a refund is requested at 2025-08-31T10:00:01Z
      Then the request is rejected with reason "expired"
 
    Example: Partials must not exceed the original
      Given prior refunds of 90.00 USD on a 100.00 USD order
      When a refund of 20.00 USD is requested
      Then the request is rejected with reason "exceeds-remaining"

The refund-policy examples turn business meaning into concrete checks: a time limit, an expired request, and a partial refund that exceeds the remaining amount. Their value depends on whether those rules actually express the intended policy. section 1.5

Put human attention where it scales: author and curate specifications, design adversarial and mutation-resistant test plans, and maintain shared, versioned assets such as policy packs and spec templates. Reading every generated line is not a reliable control at agent scale. Automated reviewers can help flag concerns, while deterministic checks serve as gates. section 1.5 section 5.3.2

How Vibe Engineering covers it

How deep it goes
Distinguishes verification layers and recommends specification and test-plan controls without detailing a complete verification procedure.
Examples
Uses refund-policy examples and a model-based code reviewer to illustrate checks and their limits.
What you can do after
Evaluate whether AI-assisted code reflects business intent and recognize shared blind spots in generated checks.
How it connects
Connects specifications, contracts, test plans, CI/CD, and automated code review.

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Vibe Engineering. The section links point to the parts of the book it draws on.