liveBook · Manning

Requirements

As explained in Architecting an Apache Iceberg Lakehouse

A storage requirement is an explicit, testable condition that a storage system or architecture must satisfy.

A requirement is an explicit, testable condition that a storage system or architecture must satisfy. In a lakehouse, requirements translate audited constraints and stakeholder priorities into outcome-focused criteria for design decisions. section 4.0 section 4.3.1

Begin with an audit and stakeholder discovery, then connect use cases to the lakehouse’s storage, ingestion, catalog, federation, and consumption layers. Group requirements by component, write them in plain language, and prioritize them by business value and implementation effort. section 4.1 section 4.3

Strong requirements express needs rather than prematurely selecting solutions:

  • Storage: specify compliance, performance, security, integrity, cost, scalability, and operational constraints. section 4.3.1 section 5.1
  • Ingestion: define throughput, latency, fault tolerance, schema management, auditability, and operational complexity. section 6.1
  • Catalog: evaluate interoperability, governance, access control, operational overhead, scale, support, and cost. section 7.5
  • Federation and consumption: require external connectivity, shared semantics, accessible interfaces, performance, compliance, governance, and cost alignment. section 8.2 section 9.2

Testable storage requirements

A regulated platform might require EU-collected PII to remain in EU data centers, expose S3-compatible APIs, and archive cold data after a defined period while keeping it queryable through Iceberg metadata. section 4.3.1 section 4.4.1

Figure 4.1 The journey from gathering information to establishing requirements and implementing a solution

Use the audit-to-implementation journey to connect stakeholder discovery, requirements, architectural design, and implementation. section 4.1

How Architecting an Apache Iceberg Lakehouse covers it

How deep it goes
Explains an audit-to-requirements process across five lakehouse layers but stops short of selecting specific tools or vendors.
Examples
Uses residency, encryption, archival, low-latency ingestion, regulatory extracts, shared semantics, and dashboard scenarios as requirement examples.
What it assumes
Assumes access to stakeholder knowledge and audit findings while teaching the translation from business use cases to technical conditions.
What you can do after
Apply requirements to compare architectures and tools against business value, constraints, implementation effort, and operational tradeoffs.
How it connects
Connects stakeholder discovery, use cases, data-platform audits, storage, ingestion, catalogs, federation, consumption, and architectural tradeoffs.

Also explained in

Compare how 13 books explain requirements →

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Architecting an Apache Iceberg Lakehouse. The section links point to the parts of the book it draws on.