liveBook · Manning

Microsoft OneLake

As explained in Architecting an Apache Iceberg Lakehouse

Microsoft OneLake is the unified logical data-lake storage layer underlying the Microsoft Fabric analytics platform.

Microsoft OneLake is the unified logical data-lake storage layer underlying Microsoft Fabric, a SaaS analytics platform spanning data engineering, data science, real-time analytics, and business intelligence. It centralizes access while retaining compatibility with open table formats and external storage. section 5.4.5

OneLake’s architecture combines interoperability with virtualized access:

  • Supports Apache Iceberg and Delta Lake, allowing external engines and tools to interoperate with Fabric data. section 5.4.5
  • Creates shortcuts to ADLS Gen2, Amazon S3, Google Cloud Storage, and S3-compatible services without physically moving data. section 5.4.5 section 8.6.1
  • Pushes filters and column projections toward remote storage while keeping most joins and aggregations local to the querying engine. section 8.6.1

Virtualized external dataset

An S3 dataset can appear as a native OneLake table through a shortcut. Queries can request only required columns or filtered data, avoiding a full data copy while limiting cross-system query expressiveness. section 8.6.1

How Architecting an Apache Iceberg Lakehouse covers it

How deep it goes
Explains storage integration and shortcut behavior, but stops short of detailed implementation or table-format internals.
Examples
Uses shortcuts to ADLS Gen2, Amazon S3, Google Cloud Storage, NetApp, and MinIO as external storage examples.
What you can do after
Operate/configure unified Fabric access by linking external storage through shortcuts and selecting local-versus-remote execution tradeoffs.
How it connects
Connects Microsoft Fabric with Apache Iceberg, Delta Lake, ADLS Gen2, Amazon S3, and data virtualization.

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Architecting an Apache Iceberg Lakehouse. The section links point to the parts of the book it draws on.