liveBook · Manning

Corroboration check pattern

As explained in Effective Behavior-Driven Development

A scenario-automation pattern that checks application logs for entries indicating potential issues, including issues that have not yet caused behavioral failures.

A scenario can pass its functional checks while the application still reports a warning or error. A corroboration check inspects application health indicators as well, so potential side effects are not overlooked. Unexpected issues prompt the team to manually verify that the application is healthy. section 19.5 section 19.7

Keep the check useful by distinguishing real health concerns from expected or unrelated log entries:

  • Check health after the scenario passes its normal functional checks. section 19.5
  • Filter warnings or errors that are expected or unrelated to the scenario’s test target. section 19.5
  • Disable issue tracking for setup steps when they exercise unrelated application areas; re-enable it for the behavior under test. section 19.5
Listing 19.11 Verify application health once the scenario has otherwise passed
*** Support/Hooks.cs ***
[Binding]
public class Hooks(AppLogContext appLogContext)    #A
{
  [AfterScenario(Order = 1)]    #B
  public void CheckAppHealth()
  {
    if (scenarioContext.ScenarioExecutionStatus == 
        ScenarioExecutionStatus.OK)    #C
    {
      appLogContext.CheckAppHealth();    #D
    }
  }
}
*** Support/AppLogContext.cs ***
public class AppLogContext
{
  private readonly ConcurrentQueue<string> logMessages   #E
      = new();
  private readonly ConcurrentQueue<string> healthIssues    #F
      = new();

  public void AddLogMessage(LogLevel logLevel,    #G
      string logMessage)
  {
    logMessages.Enqueue(logMessage);
    if (logLevel >= LogLevel.Warning)    #H
    {
      healthIssues.Enqueue(logMessage);
    }
  }
  ...
  public void CheckAppHealth()    #I
  {
    if (healthIssues.Any())
    {
      string issueText = string.Join(Environment.NewLine, healthIssues);
      throw new CorroborationCheckException($"The application log contains 
➥ warnings or errors:{Environment.NewLine}{issueText}");
    }
  }
}

See a C# after-scenario hook that checks health only after a successful scenario, with warning-level messages recorded as health issues. section 19.5

How Effective Behavior-Driven Development covers it

How deep it goes
Explains when to check health and manage irrelevant issues, with an after-scenario hook example.
Examples
Shows an after-scenario hook and log queues that record warning-level messages as health issues.
What you can do after
Diagnose potential application health issues that functional checks alone may miss.
How it connects
Connects application logs with BDD scenarios, the Context Shortcut pattern, and the Embedded Application Log pattern.

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Effective Behavior-Driven Development. The section links point to the parts of the book it draws on.