liveBook · Manning

Mocking

As explained in Data Pipelines with Apache Airflow, Second Edition

A mock is a test double that replaces a dependency with controlled behavior for verifying software interactions.

Mocking replaces an operation or object with controlled behavior so a test can run without calling an external system. In Airflow operator tests, a mock can make a hook return a test connection instead of querying the metastore; the test can also check whether the hook was called as expected. Mocking requires knowing which internals to replace, including the correct import location. section 10.1.4 section 10.4.1

Listing 10.15 Mocking an object in a test
def test_movielenspopularityoperator(mocker):
   mocker.patch.object(
       MovielensHook,
       "get_connection",
       return_value=Connection(
           conn_id="test",
           login="airflow",
           password="airflow",
       ),
   )
   task = MovielensPopularityOperator(
       task_id="test_id",
       conn_id="test",
       start_date="2015-01-01",
       end_date="2015-01-03",
       top_n=5,
   )
   result = task.execute(context=None)
   assert len(result) == 3

Example: patch a MovieLens hook to return a test connection, then execute the operator without querying the metastore. section 10.1.4

Listing 10.17 Validating the behavior of a mocked function
mock_get = mocker.patch.object(  #1
    MovielensHook,
    "get_connection",
    return_value=Connection(...),
)
task = MovielensPopularityOperator(..., conn_id="testconn")
task.execute(...)

assert mock_get.call_count == 1  #2
mock_get.assert_called_with("testconn")  #3

A mock can expose call details so a test can check both how often a function ran and which argument it received. section 10.1.4

How Data Pipelines with Apache Airflow, Second Edition covers it

How deep it goes
Explains return-value substitution, patching, interaction assertions, and import location, while requiring insight into tested internals.
Examples
Uses a mocked MovieLens hook connection in an operator test and a mocked API call in a DAG test.
What it assumes
Assumes Python test code and requires enough understanding of the tested internals to identify what to replace.
What you can do after
Apply it via pytest-mock to replace external calls and verify their call counts and arguments.
How it connects
Connects pytest-mock and Python monkey patching with Airflow hooks, operators, and DAG testing.

Also explained in

Compare how 21 books explain mocking →

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Data Pipelines with Apache Airflow, Second Edition. The section links point to the parts of the book it draws on.