liveBook · Manning

Software testing

As explained in Build High Value Software Systems

A procedure that evaluates software behavior against expected results under specified conditions.

Software testing checks behavior under specified conditions against an expected result. Its practical purpose is to prevent rework by catching consequential defects before they reach users. More tests are not automatically better: a test that fails during a safe refactor, blocks CI without a trustworthy signal, or protects no meaningful behavior can cost more than it saves. section 5.0 section 5.1

Before adding or keeping a test, ask:

  • What customer, business, or developer value does it protect? If it fails, would the failure reveal a meaningful risk? section 5.1
  • Does it fail when behavior breaks, rather than when an implementation detail changes? section 5.1
  • Is it fast and reliable enough to run routinely? Long pipelines create waiting; intermittent failures teach developers to distrust red builds. section 5.1 section 5.4
Listing 5.1 A contrast between a wasteful test and a value-adding test
# Waste: tests implementation details
def test_payment_uses_token_bucket_limiter():
    processor = PaymentProcessor()
    processor.process(payment_request)
    assert isinstance(processor.limiter, TokenBucketRateLimiter)
    assert processor.limiter.calls == [call(rate=100, burst=10)]
 
# Value: tests behavior
def test_payment_respects_rate_limits():
    processor = PaymentProcessor()
    results = [processor.process(payment_request) for _ in range(150)]
    
    passed = sum(1 for r in results if r.success)
    rate_limited = sum(1 for r in results if r.rate_limited)
    
    assert passed <= 110  # allows burst
    assert rate_limited >= 40

The payment example contrasts an assertion about the rate limiter's type and calls with assertions about successful and rate-limited payments. The latter can continue protecting payment behavior when the limiter implementation changes. section 5.1

Preserving a legacy module's behavior

Run an unfamiliar module with realistic inputs and observe its outputs and surprising branches. Then write characterization tests for what it does today—including typical cases, discovered edge cases, and threshold boundaries. If negative inputs currently return zero, assert zero even if that rule seems odd. After a small cleanup, rerun the tests to check that the observed behavior remains intact. section 7.5

A flaky test produces different results across identical runs in an unchanged environment. Time, network latency, or shared test data can cause it. Rerunning until green does not restore a trustworthy signal; repeated false alarms make genuine failures easier to dismiss. Treat flakiness as a defect in the testing workflow, not an accepted inconvenience. section 5.4

Use coverage to find paths worth investigating, not as proof that important risks are protected. Choose test layers according to risk, keep assertions focused on business contracts, and inspect CI time, reruns, and failures during safe refactoring. The useful suite is the one that gives dependable feedback without obstructing change. section 5.9 section 5.1

How Build High Value Software Systems covers it

How deep it goes
Develops selection criteria and a legacy-module procedure; stops short of test-framework setup or implementation-heavy examples.
Examples
Contrasts payment rate-limiter assertions with behavior checks and uses a legacy module's observed edge cases to guide characterization tests.
What it assumes
Assumes familiarity with CI, refactoring, and existing modules; does not teach test tooling from zero.
What you can do after
Practice/perform risk-focused test selection, characterize existing behavior before changing it, and recognize unreliable CI feedback.
How it connects
Connects testing to CI pipelines, refactoring, characterization tests, flaky tests, coverage, and risk-based test layers.

Also explained in

Compare how 67 books explain software testing →

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Build High Value Software Systems. The section links point to the parts of the book it draws on.