liveBook · Manning

Encryption

As explained in Data Pipelines with Apache Airflow, Second Edition

Encryption is the transformation of readable data into ciphertext that requires a cryptographic key for recovery.

When Airflow secrets are stored unencrypted in its database, anyone with database access can read them. Fernet encryption stores secrets as ciphertext and uses the same key to encrypt and decrypt them. section 16.2

Protect encrypted secrets by managing the key carefully:

  • Generate a Fernet key and provide it to Airflow through AIRFLOW__CORE__FERNET_KEY. section 16.2
  • Keep the key secret: anyone who has it can decrypt the secrets. If the key is lost, those secrets can no longer be decrypted. section 16.2

Generate a Fernet key

from cryptography.fernet import Fernet fernet_key = Fernet.generate_key() print(fernet_key.decode()) section 16.2

Figure 16.7 A Fernet key encrypts data before storing it in the database and decrypts data before reading it from the database. Without access to the Fernet key, passwords are useless to an intruder.

Shows how a Fernet key encrypts data before database storage and decrypts it when read. section 16.2

Encryption also protects Airflow traffic over HTTPS. A self-signed certificate can serve a local UI, but users must trust it; for an address accessible to others, use a certificate from a trusted authority. section 16.4.2

How Data Pipelines with Apache Airflow, Second Edition covers it

How deep it goes
Explains symmetric Fernet encryption, key configuration, and key risks; HTTPS coverage focuses on certificate trust.
Examples
Demonstrates generating a Fernet key in Python, configuring Airflow, and creating a self-signed certificate with OpenSSL.
What you can do after
Operate Airflow secret encryption by generating and configuring a Fernet key while guarding against key loss or exposure.
How it connects
Connects symmetric encryption with Fernet keys, Airflow connections, variables, user passwords, and HTTPS certificates.

Also explained in

Compare how 16 books explain encryption →

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Data Pipelines with Apache Airflow, Second Edition. The section links point to the parts of the book it draws on.