liveBook · Manning

False positive

As explained in Spec-Driven Development

A result incorrectly judged to satisfy a condition when it does not.

A false-positive completion is a feature an agent reports as complete that later fails review or testing. It occurs when completion is accepted on the agent’s claim rather than demonstrable evidence. Imprecise success criteria can also let an incorrect solution pass. section 3.4.2 section 2.1.2

Use false positives as a signal about whether intent is verifiable:

  • Observe them qualitatively at first; a high share suggests the completion gate is relying on claims instead of proof. section 3.4.2
  • Track them alongside specification bypasses and alignment regressions to keep the specification and implementation aligned. section 3.4.2
Listing 2.1 A unit test with weak assertions in pseudocode
test "filters active users"
    # Arrange
    users = [
        { name: "John", status: "active" },
        { name: "Jane", status: "inactive" },
        { name: "Chen", status: "active" }
    ]
    # Act
    result = filterByStatus(users, "active")
    # Assert
    assert result is not empty

A test that checks only that an active-user filter returns something can pass even when its results are wrong. section 2.1.2

How Spec-Driven Development covers it

How deep it goes
Defines the metric as a share, distinguishes claims from demonstrable evidence, and begins with qualitative observation rather than tooling.
Examples
Uses an active-user filter test whose nonempty-only assertion can pass despite incorrect results.
What you can do after
Diagnose weak completion gates by tracking agent-reported features later rejected in review or testing.
How it connects
Connects specification quality, verifiable success criteria, software testing, review, and completion gates.

Also explained in

Compare how 26 books explain false positive →

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Spec-Driven Development. The section links point to the parts of the book it draws on.