XCom

Suppose a model-training task registers a model and receives an identifier. A later deployment task needs that identifier, not the model itself. XCom (“cross-communication”) lets Airflow tasks exchange small values within a DAG run. section 6.5.1 section 6.5

With PythonOperator, the training function can publish the identifier through the task instance’s xcom_push method. The deployment function retrieves it with xcom_pull, specifying the producing task ID and matching key. Values are registered for a task, DAG, and execution date; a pull defaults to the current DAG and logical date. Avoid pulling from other DAGs or dates without a compelling reason. section 6.5.1

def _train_model(**context):
    model_id = str(uuid.uuid4())
    context["task_instance"].xcom_push(key="model_id", value=model_id)

train_model = PythonOperator(
    task_id="train_model",
    python_callable=_train_model,
)

The training task publishes a generated model ID under the key model_id. section 6.5.1

def _deploy_model(**context):
    model_id = context["task_instance"].xcom_pull(
        task_ids="train_model", key="model_id"
    )
    print(f"Deploying model {model_id}")

deploy_model = PythonOperator(
    task_id="deploy_model",
    python_callable=_deploy_model,
)

The deployment task requests that same key from train_model. section 6.5.1

The pull does not schedule the producer. Declare the task dependency separately so training runs before deployment: an XCom lookup otherwise creates a dependency that is invisible in the DAG structure. XCom values can also be pulled in templates. A PythonOperator can publish its returned value implicitly under the return_value key. section 6.5.2 section 6.5.1

For Python tasks, the Taskflow API makes the handoff more explicit: decorate the functions with @task, return the identifier from training, and pass that task’s result to deployment. Airflow handles the XCom exchange, while the function call establishes the task relationship. This also avoids repeating an XCom key in both functions. section 6.6.1

model_id = train_model()
deploy_model(model_id)

Passing the training task’s result to the deployment task expresses both the data handoff and the dependency. section 6.6.1

Keep XCom handoffs small and operationally safe:

• Use them for identifiers, filenames, paths, or small API responses. Store larger datasets in an external database or blob storage instead. section 6.7
• Pass only serializable values, and account for backend size limits. Default XCom storage is the Airflow metastore; even a generous database limit is not a reason to store huge values. section 6.5.2
• Preserve task atomicity. Passing a short-lived API token to a later task can make that task fail on rerun; fetching and using the token together may be safer. section 6.5.2
• Clean up accumulated values with airflow db clean when using default storage. Custom XCom backends need their own monitoring and cleanup. section 6.5.4