liveBook · Manning

Left-shift operation

As explained in Data Pipelines with Apache Airflow, Second Edition

The left shift operator is a Python operator that shifts the binary representation of an integer to the left and is overloaded by Airflow for task dependency notation.

The left-shift operator (<<) normally shifts an integer’s binary representation left. Airflow overloads it so tasks can express execution dependencies instead of performing bit shifting. section 2.2

Order a launch-data workflow

A download task must finish before image processing, which must finish before notification. Express that sequence as: download_launches << get_pictures << notify. In Airflow’s dependency notation, the equivalent right-shift form shown in the workflow is download_launches >> get_pictures >> notify: each downstream task waits for its upstream task to complete successfully. section 2.2

Listing 2.5 Defining the order of task execution
download_launches >> get_pictures >> notify       #1

Inspect the compact task-ordering listing and compare the readable dependency chain with the workflow’s task relationships. section 2.2

How Data Pipelines with Apache Airflow, Second Edition covers it

How deep it goes
Explains Python’s original operator meaning and Airflow’s overload, then demonstrates a linear dependency chain without deeper implementation details.
Examples
Uses download, image-processing, and notification tasks for rocket-launch data.
What it assumes
Assumes basic Python operators and Airflow tasks while explaining dependency notation from the workflow scenario.
What you can do after
Apply it via Airflow to declare task order with readable dependency notation.
How it connects
Connects left-shift notation with Python operators, Airflow tasks, DAGs, and task dependencies.

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Data Pipelines with Apache Airflow, Second Edition. The section links point to the parts of the book it draws on.