liveBook · Manning

Snapshot comparison

As explained in Vibe Engineering

A golden file is an approved reference-output file against which subsequent program output is compared.

Snapshot comparison stores program output as a golden baseline and fails when later output differs. It detects changes, not whether the original output was correct. section 8.3.5

A bad first output can become the standard: later tests may pass while preserving incorrect behavior. Use snapshots to catch regressions only after verifying the baseline independently. section 8.3.5

Approval testing adds explicit human review before outputs become baselines. For visual regression, reviewers inspect rendered screenshots and approve acceptable changes; stable screenshots alone do not establish good design. section 8.3.6 section 8.3.6

Treat tests as contracts that state required behavior independently of implementation. Snapshots capture implementation behavior, so pair them with tests that express requirements directly. section 8.3.7 section 8.3.7

Listing 8.7 Generate golden test set
# First run: generates snapshots
def test_parser_snapshot(snapshot):
    inputs = [
        "# Header",
        "**bold** and *italic*",
        "[link](url)",
        "Plain text",
        "## Header with **formatting**"
    ]
    for i, inp in enumerate(inputs):
        snapshot.assert_match(parse_markup(inp), f'output_{i}')

A parser test captures outputs for several markup inputs as named snapshots. section 8.3.5

How Vibe Engineering covers it

How deep it goes
Explains baseline creation, repeated comparisons, human approval, and limits; stops short of proving output correctness.
Examples
Uses markup-parser outputs and UI screenshots as candidate baselines for comparison and review.
What you can do after
Diagnose/debug with it by spotting changed parser outputs or UI screenshots, then investigating whether each difference is unintended.
How it connects
Connects snapshot comparison to regression testing, approval testing, visual regression testing, and contract-based tests.

Also explained in

Compare how 4 books explain snapshot comparison →

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Vibe Engineering. The section links point to the parts of the book it draws on.