liveBook · Manning

Test runner

As explained in F# in Action: A Revised Edition of Get Programming with F#

A test runner is software that discovers tests and executes them, typically reporting their results.

A test runner discovers tests, executes them, and reports their results. In F#, it can connect automated tests with IDEs such as Visual Studio or run them from the command line. section 14.2 section 14.4

Expecto treats tests as ordinary F# values: individual tests or nested lists can be composed, selected, generated, and passed to the runner. This avoids classes, inheritance, and attributes, while allowing conditions such as CI-only tests or data-driven repetition. section 14.2.4

Programmatic test generation

A test list can read rows from testData.txt, create an addition test for each row, and decide whether to include a CI-only test from an environment variable. The runner reports each generated test and summarizes passes, failures, and errors. section 14.2.4

XUnit uses a more conventional discovery model. Tests usually live in a class library, carry attributes such as [<Fact>], and run through dotnet test. This reduces manual composition but binds discovery and execution to the framework. section 14.4

Listing 14.4 Programmatically creating tests in Expecto
let add a b = a + b                                               #1
 
let myTests = testList "My Tests" [
    if System.Environment.GetEnvironmentVariable
 "CI_SERVER" <> null then                                       #2
        test "CI Only Test" { Expect.equal (add 1 2)
 3 "Something is very wrong!" }
 
    for line in System.IO.File.ReadAllLines "testData.txt" do     #3
        test $"Addition test: {line}" {                           #4
            match line.Split ',' with
            | [| inputA; inputB; expected |] ->
                let actual = add (int inputA) (int inputB)
                Expect.equal actual (int expected) "Calculator is broken"
            | _ ->
                failwith $"Invalid test data format: {line}"
        }
]
#1 Our production code, a simple addition function
#2 Programmatically decides whether to yield back a test based on the existence of an environment variable
#3 Reads data from a file using basic .NET code
#4 For each line in the file, yields back a new test whose name is based on the line in the file

Inspect a complete Expecto example that builds tests from CI conditions and input-file rows. section 14.2.4

How F# in Action: A Revised Edition of Get Programming with F# covers it

How deep it goes
Explains discovery, execution, reporting, composition, and framework trade-offs, but stops short of runner implementation internals.
Examples
Uses Expecto lists, CI-controlled tests, input-file-driven addition tests, result summaries, and XUnit functions marked with Fact attributes.
What it assumes
Assumes basic F# functions and lists, while demonstrating test construction without requiring classes, inheritance, or advanced language features.
What you can do after
Operate Expecto or XUnit to compose, discover, execute, and evaluate F# tests from programs or the command line.
How it connects
Ties test runners to automated testing, test discovery, IDE integration, continuous integration, Expecto, XUnit, NUnit, and attributes.

Also explained in

Compare how 5 books explain test runner →

Related concepts

This explainer was generated by Manning's knowledge graph from the text of F# in Action: A Revised Edition of Get Programming with F#. The section links point to the parts of the book it draws on.