liveBook · Manning

Metadata governance

As explained in Architecting an Apache Iceberg Lakehouse

Metadata governance is the policies, controls, and practices for ensuring that metadata is accurate, secure, traceable, and consistently managed.

Metadata governance establishes policies and controls that keep catalog metadata accurate, secure, traceable, and consistently managed. A governed catalog records where data resides, how it is structured, and who may access it. section 7.2.2

Implement governance through connected controls:

  • Organize assets with namespaces aligned to business domains or organizational units. section 7.2.2
  • Integrate enterprise identities through OAuth, LDAP, SAML, or SSO, then apply RBAC or ABAC authorization. section 7.2.2
  • Use credential vending to issue short-lived, scoped credentials and reduce storage-access blast radius. section 7.2.2
  • Track lineage and audit metadata so teams can support compliance, traceability, and root-cause analysis. section 7.2.2

Apache Gravitino

Apache Gravitino centralizes metadata across tabular data, files, message systems, and machine-learning artifacts. It combines role-based policies, tags, audit logging, enterprise authentication, and an Apache Iceberg REST Catalog interface, while its broad scope can increase operational complexity. section 7.4.6

Figure 7.6 Catalogs can have different levels of access control, allowing you to control who has access to which namespaces or tables.

Compare catalog access levels across namespaces and tables to see how governance controls scope permissions. section 7.2.2

How Architecting an Apache Iceberg Lakehouse covers it

How deep it goes
Connects namespaces, identity integration, authorization, credential vending, lineage, and auditing, without detailing implementation procedures.
Examples
Uses namespaces, scoped credentials, lineage chains, catalog access diagrams, and Apache Gravitino as concrete governance examples.
What you can do after
Evaluate catalog options against access control, identity, lineage, auditability, and heterogeneous metadata requirements.
How it connects
Connects catalogs with access control, RBAC, ABAC, identity providers, credential vending, lineage, auditing, and Apache Iceberg.

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Architecting an Apache Iceberg Lakehouse. The section links point to the parts of the book it draws on.