liveBook · Manning

Flat namespace

As explained in Architecting an Apache Iceberg Lakehouse

A flat namespace addresses objects directly by unique keys without organizing them in nested directories.

A flat namespace addresses objects directly through unique keys rather than nested directories. This structure scales across very large object collections and avoids directory-depth limits, while keys can still resemble familiar paths. section 5.2.2

Figure 5.6 Files are stored as blobs (binary large objects) alongside an object that holds the metadata and ID used to access them. A user requests a file by key, and the system uses the metadata to locate and retrieve it.

Shows how object data, metadata, and identifiers work together when a client requests an object by key. section 5.2.2

A flat namespace supports object-storage architectures by:

  • Addressing each object independently through an S3-compatible API, enabling concurrent retrieval without directory locking. section 5.3.2
  • Using metadata for search, lifecycle management, access control, versioning, and replication without changing object contents. section 5.2.2
  • Supporting lakehouse tables composed of independently readable and writable immutable data and metadata files. section 5.3.2

Path-like keys

A key such as logs/2024/01/file.parquet resembles a directory path, but the object store treats it as one directly addressable key rather than a nested filesystem location. section 5.2.2

How Architecting an Apache Iceberg Lakehouse covers it

How deep it goes
Defines key-based addressing, metadata, scalability, and concurrent access, then connects the model to Iceberg commits.
Examples
Uses path-like object keys and Iceberg data and metadata files as concrete examples.
What you can do after
Operate object storage by addressing objects with unique keys and applying metadata-driven management policies.
How it connects
Connects object storage, S3 API, hierarchical filesystems, metadata, and Apache Iceberg.

Also explained in

Compare how 2 books explain flat namespace →

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Architecting an Apache Iceberg Lakehouse. The section links point to the parts of the book it draws on.