liveBook · Manning

Disconfirming signal

As explained in The Architecture of Reasoning

An observation that challenges an assumption or hypothesis rather than supporting it.

A plan can look convincing while its assumptions remain untested. A disconfirming signal is an observation agreed in advance that would challenge the reasoning and require reconsidering the plan—not merely a result that falls short of success. section 6.6

Product launch

If a new product raises sales by 5% rather than the expected 10%, the team might keep iterating. But if complaints rise above the average for similar launches, that pre-agreed signal could prompt an immediate pull. section 6.6

For an AI support pilot, a decline in satisfaction, more escalations, or feedback indicating frustration could challenge the assumption that faster automated replies preserve interaction quality. Test with a limited user group, review signals within two weeks, and agree in advance when to pause expansion. section 6.7

Table 6.1 A compact reasoning scaffold for ReAct deliberation
Element
Purpose
Guiding questions
Hypothesis
Define the expected outcome and reasoning
What do we believe will happen, and why?
Key assumption
Identify what must hold true
What must be true for this to hold?
Disconfirming signal
Specify what would invalidate the reasoning
What outcome would indicate this is wrong or incomplete?
Test action
Generate a signal from reality
What can we do quickly to test this?
Observation window
Define when evaluation occurs
When and how will we evaluate the result?
Adjustment rule
Define how reasoning updates
How will we change direction if the signal appears?

Use the scaffold to connect the signal to the assumption it tests, a quick test, an observation window, and a pre-agreed adjustment. section 6.6

How The Architecture of Reasoning covers it

How deep it goes
Specify signals in advance and connect them to tests, observation windows, and adjustment rules.
Examples
Illustrate the idea with product complaints and AI support measures such as satisfaction, escalations, and customer feedback.
What you can do after
Evaluate a decision by agreeing which observations would challenge its assumptions and what response those observations trigger.
How it connects
Tie disconfirming signals to ReAct, hypotheses, assumptions, evidence, and decision adjustment.

Related concepts

This explainer was generated by Manning's knowledge graph from the text of The Architecture of Reasoning. The section links point to the parts of the book it draws on.