liveBook · Manning

Black-box testing

As explained in Write Powerful Rust Macros

Black-box testing evaluates software through externally observable behavior and public interfaces without relying on implementation details.

Black-box testing checks software from the outside: exercise its public interface and verify the result, without depending on how the implementation achieves it. It focuses on the value produced by code rather than temporary structures, helper functions, or generated details behind that interface. section 6.4

This approach is especially useful for procedural macros. A usage crate can invoke a derive macro exactly as an application would, and compilation plus observable behavior establish whether the macro generated the promised API. Because tests do not encode internal output layout, refactoring implementation details usually requires fewer test changes than white-box testing. section 6.4.1 section 6.4

Test the generated builder through its public API

Define a struct with `#[derive(Builder)]`, then call `ExampleStructNoFields::builder().build()` and assign the result to that struct type. The test proves that the expected generated functions exist, the code compiles, and `build` returns the advertised type—without inspecting the macro’s token stream. section 6.4.1

Use black-box tests to establish the most important external contracts first:

  • Start with a happy-path compilation test; a macro that cannot compile in expected use has no practical value. section 6.5
  • Add unhappy-path compilation cases when the macro has meaningful failure modes; `trybuild` supports mixtures of successful and failing scenarios. section 6.5 section 6.7
  • Keep white-box tests for tricky internal errors or important implementation behavior, but expect them to break when harmless output details move. section 6.9
  • Add integration-level tests when macro-generated code communicates with external systems. section 6.9

Black-box failures can be less specific than internal assertions. When a generated API does not compile, inspect the compiler message and, if needed, expand the test output with `cargo expand --tests --bin builder-usage` to see the generated Rust code. The test remains external; expansion is a debugging aid rather than its assertion target. section 6.4.1

Listing 6.8 A first black-box test (in builder-usage, main.rs)
use builder_macro::Builder;

fn main() {}

#[cfg(test)]
mod tests {

    #[test]
    fn should_generate_builder_for_struct_with_no_properties() {
        #[derive(Builder)]
        struct ExampleStructNoFields {}     #1

        let _: ExampleStructNoFields = ExampleStructNoFields::builder()
            .build();                       #2
    }
}

This minimal test demonstrates the boundary: it derives the macro and uses only `builder()` and `build()` from the resulting public API. section 6.4.1

How Write Powerful Rust Macros covers it

How deep it goes
Defines outsider-focused testing and develops a compiling builder use case, while deferring detailed internal-output checks to white-box tests.
Examples
Uses a derived empty struct whose generated `builder().build()` call must compile and return the struct type.
What it assumes
Assumes ability to write Rust tests and use a derive macro; teaches the outsider perspective from zero.
What you can do after
Apply it via Rust tests to validate generated public APIs with happy-path and selected failing compilation cases.
How it connects
Contrasts black-box testing with white-box testing, test-driven development, integration testing, pure functions, and procedural macros.

Also explained in

Compare how 6 books explain black-box testing →

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Write Powerful Rust Macros. The section links point to the parts of the book it draws on.