liveBook · Manning

Credential vending

As explained in Architecting an Apache Iceberg Lakehouse

Storage credential vending is the controlled issuance of temporary credentials for accessing storage resources.

Credential vending is a catalog-controlled security pattern that issues short-lived, scoped credentials to query engines or clients. The credentials permit access only to storage paths required for an authorized query or job, then expire automatically. section 7.2.2 section 7.6

The catalog remains the authority for identities, roles, namespaces, tables, and operations. Engines authenticate to the catalog rather than receiving long-lived object-storage credentials, centralizing governance and reducing blast radius, credential-management overhead, and audit complexity. section 7.2.2 section 7.6

A practical credential-vending flow typically includes:

  • Authenticate a user, service principal, or query engine with the catalog. section 7.4.10
  • Evaluate catalog permissions through roles assigned at catalog, namespace, or table scope. section 7.4.4
  • Issue temporary credentials limited to the authorized storage locations and operation. section 7.6
  • Allow the engine to read or write the required table data without exposing permanent storage credentials. section 7.4.10

Multi-engine lakehouse access

A Spark ETL job, Flink ingestion pipeline, and Trino BI workload use service principals with scoped catalog privileges. Each engine receives temporary storage credentials at runtime, while the catalog preserves consistent access rules across shared Iceberg tables. section 7.4.10 section 7.4.4

Credential vending works best with non-overlapping storage locations and fine-grained catalog authorization. It does not, by itself, standardize row-level or column-level security; those catalog-level mechanisms remain under development across the Iceberg ecosystem. section 7.4.7 section 7.6

How Architecting an Apache Iceberg Lakehouse covers it

How deep it goes
Explains runtime issuance, scope restriction, expiration, and catalog authorization, but stops short of standardized row- and column-level enforcement.
Examples
Uses Spark, Flink, and Trino engines receiving scoped credentials for shared Iceberg tables.
What it assumes
Assumes a catalog-centered access model, authenticated principals, catalog permissions, and storage-backed Iceberg tables.
What you can do after
Operate multi-engine lakehouse access without distributing long-lived object-storage credentials to users or query engines.
How it connects
Connects catalog access control with identity providers, RBAC, service principals, Iceberg REST catalogs, and object storage.

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Architecting an Apache Iceberg Lakehouse. The section links point to the parts of the book it draws on.