A plan can look convincing while its assumptions remain untested. A disconfirming signal is an observation agreed in advance that would challenge the reasoning and require reconsidering the plan—not merely a result that falls short of success. section 6.6
Product launch
If a new product raises sales by 5% rather than the expected 10%, the team might keep iterating. But if complaints rise above the average for similar launches, that pre-agreed signal could prompt an immediate pull. section 6.6
For an AI support pilot, a decline in satisfaction, more escalations, or feedback indicating frustration could challenge the assumption that faster automated replies preserve interaction quality. Test with a limited user group, review signals within two weeks, and agree in advance when to pause expansion. section 6.7
Table 6.1 A compact reasoning scaffold for ReAct deliberation
|
Element
|
Purpose
|
Guiding questions
|
|
Hypothesis
|
Define the expected outcome and reasoning
|
What do we believe will happen, and why?
|
|
Key assumption
|
Identify what must hold true
|
What must be true for this to hold?
|
|
Disconfirming signal
|
Specify what would invalidate the reasoning
|
What outcome would indicate this is wrong or incomplete?
|
|
Test action
|
Generate a signal from reality
|
What can we do quickly to test this?
|
|
Observation window
|
Define when evaluation occurs
|
When and how will we evaluate the result?
|
|
Adjustment rule
|
Define how reasoning updates
|
How will we change direction if the signal appears?
|
Use the scaffold to connect the signal to the assumption it tests, a quick test, an observation window, and a pre-agreed adjustment. section 6.6
