Evidence coverage is the extent to which decisive claims are linked to the evidence needed to support them. For an agent run, assess it alongside whether the result was accepted. Step count comes afterward: a short wrong answer and a long unsupported answer are both failures. section 5.1.5
Coverage also applies to architecture claims: it asks which parts of a claim have actually been tested. It differs from evidence class, which describes where evidence came from and which claims it can support. A test can be useful without supporting the entire claim. section 11.7
Testing a group-creation workflow
Revoking permission between stages and checking that the next call is blocked tests the connected system’s handling of state and authority. It does not prove that the intended salary group exists. That claim needs an authoritative readback of the tenant, template, and resulting group. After completion works, delayed responses, concurrent runs, and varied inputs can test whether completion and safety conditions still hold. section 11.7
Build an assessment around the claims and consequences:
- Define the accepted result and the evidence required for its decisive claims. Add evidence coverage and accepted outcome to reasoning telemetry; a trace alone records a path, not a correct business decision. section 5.1.5
- Use a representative, stratified workload, including ordinary, known-failure, edge, and high-risk cases. Separate conditions that may change the architecture rather than hiding them in one score. section 11.7
- Set evidence requirements according to consequences and uncertainty, not a universal sample count. Event frequency, tolerated error, variance, workload strata, and evaluator reliability affect the testing needed. section 11.7
- Keep quality, latency, cost, and unauthorized or duplicate effects visible separately when evaluating operation. section 11.7
