liveBook · Manning

Consumption layer

As explained in Architecting an Apache Iceberg Lakehouse

A lakehouse architectural layer that exposes data to business-intelligence, artificial-intelligence, and real-time applications.

The consumption layer exposes lakehouse data to business intelligence, artificial intelligence, machine learning, and real-time applications. It completes a modular architecture in which storage, ingestion, catalog, and federation support shared data products without forcing every workload into one vendor ecosystem. section 2.7 section 2.7.5

Instead of copying data into specialized warehouses, tools query governed Iceberg tables in place. This reduces duplication, inconsistent definitions, operational analytics costs, and latency while allowing teams to use different interfaces against a shared source of truth. section 9.0 section 9.1

Design the layer around the requirements that govern how people and systems consume data:

  • Optimize interactive workloads for low latency through predicate pushdown, caching, incremental results, and deliberate cost controls. section 9.2.1
  • Support visual BI, SQL, Python and R notebooks, programmatic APIs, and AI-native interfaces so users and agents access the same curated datasets. section 9.2.1
  • Enforce inherited governance through authentication, authorization, row-level security, column masking, lineage tracking, logging, and usage policies. section 9.2 section 9.2.1
  • Centralize metric definitions in a semantic layer and expose interoperable interfaces such as JDBC, ODBC, and Arrow Flight to limit semantic drift. section 9.2.2

One data product, many consumers

A governed Iceberg data product can power Tableau, Power BI, Looker, or Preset dashboards; Databricks, SageMaker, Bedrock, Hugging Face, Jupyter, or LangChain workflows; and GraphQL or REST applications without separate warehouse copies. section 2.7.5

Figure 2.6 Components of a complete data lakehouse implementation

Use the lakehouse component diagram to situate consumption alongside storage, ingestion, catalog, and federation. section 2.7

How Architecting an Apache Iceberg Lakehouse covers it

How deep it goes
Connects performance, accessibility, governance, semantic consistency, and interfaces, without descending into tool-specific implementation steps.
Examples
Uses dashboards, notebooks, AI agents, Tableau, Power BI, Looker, Jupyter, LangChain, GraphQL, and REST APIs.
What it assumes
Assumes storage, ingestion, catalog, federation, and organizational requirements while establishing consumption responsibilities from first principles.
What you can do after
Apply it via interoperable tools that query shared Iceberg data while preserving performance, governance, and semantic consistency.
How it connects
Connects Apache Iceberg, lakehouse architecture, semantic layers, catalogs, BI tools, machine learning, APIs, and AI agents.

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Architecting an Apache Iceberg Lakehouse. The section links point to the parts of the book it draws on.