liveBook · Manning

Test suite

As explained in Data-Oriented Programming in Java

A test suite is a collection of automated or executable tests that assess software against specified expectations.

A test suite is a collection of executable checks that assesses whether software meets stated expectations. Its value is not a high coverage number: a useful suite makes domain invariants, meaningful inputs, and expected outcomes clear enough to give confidence when code changes. section 11.3 section 11.6.1

Treat tests as consumers of data rather than scripts tied to a particular code path. Put relevant inputs beside assertions, generate or transform test data as needed, and avoid hard-coded fixtures whose meaning and provenance are hidden elsewhere. Randomized generation can explore rare states, but libraries are preferable once manual randomization becomes difficult to debug. section 11.8 section 11.5.7 section 12.2

A suite stays useful when it is designed around what must remain true:

  • Derive tests from formal properties and invariants, such as ordering, associativity, and monotonicity, when the domain admits them. section 7.7.3
  • Invest in data coverage and data-generation tooling instead of expanding tests merely to execute uncovered lines. section 11.6.1
  • Model invalid states out of existence where possible; a state the type system cannot express does not need a test. section 11.6.3
  • Prune overlapping integration tests as higher-level checks subsume lower-level behavior; expensive, flaky tests must earn their place. section 12.5

Data-driven file-upload integration test

Instead of uploading an unexplained CSV fixture, generate a set of rows, derive the expected US-related account IDs from those rows, serialize the input, run the service, and compare actual IDs with the derived expectation. The test remains about file upload while exposing why its result is correct; the same generator can later produce much larger data sets. section 12.2

Listing 12.13 Creating isolated dependencies for a test
@Test
void myCoolIntegrationTest () {
  // spinning up infrastructure right before we run the test
  Queue inputQueue = cloudSDK.createQueue("my-own-input-queue");
  Queue outputQueue = cloudSDK.createQueue("my-own-output-queue"); 
  Bucket bucket = cloudSDK.createBucket("bucket-for-my-test"); 

  // We construct our service using the placeholder infrastructure. 
  // (This, of course, depends on us using good Dependency 
  //  Injection practices)
  MyService service = new MyService(inputQueue, outputQueue, bucket);
  // now it’s pretty standard arrange, act, assert.
  // just that the “arrange” is done against service infrastructure 
  // that’s completely isolated to our test
  inputQueue.put(makeInputMessage());
  
  service.run()

  // then we make sure everything behaved the way we expect. 
  // for instance, the item in the input queue should have been consumed
  assertEquals(inputQueue.poll(), Optional.empty());
  // we want something in the output queue 
  assertNotEquals(outputQueue.poll(), Optional.empty());
  // and something written to our bucket 
  assertEquals(/* lookup item in bucket here */); 

  // then blow everything away 
  cloudSDK.deleteQueue(inputQueue.name());
  cloudSDK.deleteQueue(outputQueue.name());
  cloudSDK.deleteBucket(bucket.name());
}

For integration tests, create queues and storage immediately before each test and inject them into the service. Dedicated infrastructure prevents one test’s failures, leftovers, or delays from contaminating another. section 12.6

Integration suites operate against an unreliable world: networks fail, services go down, cleanups fail, and shared resources create races. Isolation, randomized resource names, namespaced cleanup, and data-focused assertions make failures more trustworthy rather than permanently noisy. section 12.6 section 12.8

How Data-Oriented Programming in Java covers it

How deep it goes
Develops data-driven, randomized, property-based, and isolated integration-suite practices, while treating coverage as an inadequate proxy for quality.
Examples
Demonstrates formal conflict-resolution properties, generated CSV uploads, large generated account sets, and isolated queues and buckets.
What it assumes
Assumes comfort reading Java tests and service code while teaching data-oriented test design and integration-test isolation practices.
What you can do after
Apply it via generated data, formal properties, and isolated infrastructure to build more trustworthy automated and integration checks.
How it connects
Connects test suites to domain modeling, type systems, randomized testing, integration testing, mocks, algebraic laws, and code coverage.

Also explained in

Compare how 22 books explain test suite →

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Data-Oriented Programming in Java. The section links point to the parts of the book it draws on.