liveBook · Manning

HashiCorp Vault

As explained in Data Pipelines with Apache Airflow, Second Edition

HashiCorp Vault is a platform for securely storing, managing, and issuing secrets and credentials.

HashiCorp Vault centralizes passwords, certificates, keys, and other secrets so applications can request them without storing local copies. Its secret rotation and versioning capabilities extend beyond Airflow’s built-in variables and connections. section 16.5

Airflow accesses Vault through its secrets backend while DAG code continues using ordinary connection and variable interfaces. Configure the backend with AIRFLOW__SECRETS__BACKEND and AIRFLOW__SECRETS__BACKEND_KWARGS; Vault paths use a shared prefix, connections use conn_uri, and variables use value. section 16.5

Fetch an HTTP connection without embedding credentials

Store the connection at a path such as secret/connections/secure_api, then let an HttpOperator reference http_conn_id="secure_api". The DAG contains no explicit Vault reference; the configured backend resolves the connection details.

How Data Pipelines with Apache Airflow, Second Edition covers it

How deep it goes
Explains backend configuration, Vault paths, required keys, and DAG usage, but does not detail Vault administration.
Examples
Uses an HttpOperator, the secure_api connection, the secret/connections/secure_api path, and the conn_uri key.
What it assumes
Assumes familiarity with Airflow connections, variables, DAGs, and operators while retaining those interfaces for secret retrieval.
What you can do after
Configure Airflow to retrieve HTTP connection details from Vault without embedding credentials in DAG code.
How it connects
Ties HashiCorp Vault to Airflow secrets backends, connections, variables, secret rotation, and versioning.

Also explained in

Compare how 2 books explain HashiCorp Vault →

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Data Pipelines with Apache Airflow, Second Edition. The section links point to the parts of the book it draws on.