liveBook · Manning

Property-based testing

As explained in The Coder Cafe

Property-based testing is a software testing technique that generates many inputs to verify general properties that outputs or behavior must satisfy.

Property-based testing (PBT) checks general rules about software by generating many inputs and looking for violations. A conventional test asks whether one chosen input produces one expected result; PBT asks whether a property holds across varied inputs, including edge cases a developer might not think to write by hand. section 5.1.3 section 5.2.3

A property states what must remain true as inputs change. Useful kinds include:

  • Structural properties, such as preserving a data structure’s length. section 5.2.3
  • Idempotency: applying an operation again produces the same result. section 5.2.3
  • Commutativity: changing the order of arguments leaves the result unchanged. section 5.2.3

From examples to generated inputs

Instead of checking only that 1 + 1 = 2, define a rule such as “adding two positive numbers produces a larger number” and try it against generated inputs. For a string-processing function, a fuzzing library could generate unusual formats and extreme values to check that the function does not crash. section 5.1.3 section 5.2.3

Fuzzing and PBT overlap: both use randomized inputs. Fuzzing commonly seeks crashes or unexpected behavior; PBT can also check more specific rules about correctness. Neither replaces carefully chosen traditional tests. The difficult, valuable step is choosing properties that express the behavior the software must preserve. section 5.2.3 section 5.2.4

Properties need not stop at individual functions. At an API or system level, they might require balanced financial transactions, retries that create no duplicate transactions, or a specified response-time threshold. Such invariants can catch missed cases and document essential system rules. section 5.2.4

How The Coder Cafe covers it

How deep it goes
Defines properties, contrasts PBT with fuzzing, and surveys function-to-system-level invariants without giving a step-by-step implementation.
Examples
Uses positive-number addition, generated strings, balanced transactions, and duplicate-free retries as example properties.
What it assumes
Introduces properties through familiar input-and-expected-output tests; assumes basic familiarity with functions and software tests.
What you can do after
Practice/perform it by defining a property, generating varied inputs, and checking for violations at function or system level.
How it connects
Connects PBT to fuzzing, unit tests, invariants, branch coverage, APIs, and system-level testing.

Also explained in

Compare how 11 books explain property-based testing →

Related concepts

This explainer was generated by Manning's knowledge graph from the text of The Coder Cafe. The section links point to the parts of the book it draws on.