Partial application

Q: How deeply does Data Pipelines with Apache Airflow, Second Edition cover partial application?

Defines the parameter split and demonstrates PythonOperator syntax, but does not explain implementation internals.

As explained in Data Pipelines with Apache Airflow, Second Edition

partial() is an Airflow task-mapping method that specifies operator parameters kept constant across mapped task instances.

Partial application configures the parameters that remain constant while Dynamic Task Mapping creates separate task instances for changing values. In Airflow, partial() holds fixed operator arguments, including task_id, while expand() supplies the values to map over. section 12.1.8

Map addition over several values

PythonOperator.partial(task_id="add", python_callable=add_function, op_kwargs={"y": 10}).expand(op_args=[[1], [2], [3]]) keeps y constant and creates three mapped task instances with different x values.

Process ratings returned at runtime

PythonOperator.partial(task_id="print_rating", python_callable=_print_rating).expand(op_args=fetch_ratings.output) maps one printing task over every rating returned by fetch_ratings, allowing independent processing even when the count changes between DAG runs.

How Data Pipelines with Apache Airflow, Second Edition covers it

Cover of Data Pipelines with Apache Airflow, Second Edition

liveBook manning.com

Read Data Pipelines with Apache Airflow, Second Edition on liveBook Get the book at manning.com

Also explained in

liveBook manning.com

Learn Haskell by ExamplePartial function application is the evaluation of a function with fewer arguments than its full arity, producing a function that accepts the remaining arguments.
liveBook manning.com

The Joy of JavaScript
liveBook manning.com

Functional Programming in JavaScript
liveBook manning.com

Functional Programming in Scala, Second EditionPartial application is the creation of a function by supplying some arguments to a function while leaving the remaining arguments to be supplied later.
liveBook manning.com

Clojure: The Essential ReferencePartial application is the technique of fixing some arguments of a function to produce another function that accepts the remaining arguments.
liveBook manning.com

Julia as a Second LanguagePartial application is a functional programming technique that fixes some arguments of a function to produce a new function requiring fewer arguments.
liveBook manning.com

Fabulous Adventures in Data Structures and AlgorithmsPartial application is the creation of a function by binding one or more arguments of another function to particular values.
liveBook manning.com

Python How-ToA partial function is a callable object created from another callable by pre-filling some of its arguments.
liveBook manning.com

The Well-Grounded Rubyist, Fourth EditionPartial function application is a functional programming technique that applies fewer arguments than a function's arity and returns a function accepting the remaining arguments.

Compare how 10 books explain partial application →

Related concepts

functionproduces · A function is a callable program unit that performs an operation, may accept inputs, and can produce a result or side effect.
curryingequivalent to · Currying is the transformation of a function accepting multiple arguments into a sequence of functions that accept arguments incrementally, typically one at a time.
Q# operationproduces · A Q# operation is a callable Q# construct that can accept parameters, act on qubits, and return a value.
constused by · const is a standard Haskell function that returns its first argument while ignoring its second argument.
Hughes listused by · A Hughes list is a functional list representation implemented with a binary tree while exposing linked-list features.

This explainer was generated by Manning's knowledge graph from the text of Data Pipelines with Apache Airflow, Second Edition. The section links point to the parts of the book it draws on.