Storage abstraction
A storage layer is a software subsystem or interface responsible for persisting and retrieving data while separating applications from underlying storage technology.
Architecting an Apache Iceberg Lakehouse
A storage layer is a lakehouse architectural layer that stores data on cost-effective storage systems.
- How it is used
- Frames storage selection as an architectural decision balancing reliability, scalability, performance, security, cost, and operational complexity.
- How deep it goes
- Explains requirements, storage models, standards, consistency, and selection tradeoffs, but stops short of detailed vendor configuration.
- Examples
- Uses HDFS, MinIO, S3, Parquet, Docker Compose, Spark, Nessie, Dremio, and Superset in architecture and deployment examples.
- What you can do after
- Evaluate storage technologies against organizational requirements and select a durable Iceberg lakehouse strategy without assuming one universal platform.
Read how Architecting an Apache Iceberg Lakehouse explains storage abstraction →
Designing AI Systems
A storage abstraction is an interface or design boundary that separates data-storage operations from the underlying persistence technology.
- How it is used
- Separates session-service behavior from persistence choices so deployments can select storage according to reliability, latency, scale, or operations needs.
- How deep it goes
- Defines interface delegation and constructor injection, then demonstrates a gRPC service implementation while leaving backend-specific mechanics behind the interface.
- Examples
- Contrasts PostgreSQL, Redis, DynamoDB, MongoDB, and other backends, then applies the pattern to sessions and vector similarity search.
- What you can do after
- Build it from scratch by defining a storage interface, implementing a backend, and injecting that implementation into a service.
Read how Designing AI Systems explains storage abstraction →
Also covered in
Related concepts
- data lakehousepart of · A data lakehouse is a data architecture pattern combining the flexible storage of data lakes with the governance, reliability, and analytical capabilities of data warehouses.
- Kafka brokerincludes · A Kafka broker is a server process in an Apache Kafka cluster that stores records and handles producer, consumer, and other client requests.
- MinIOhas part · MinIO is an open-source object-storage server that provides an Amazon S3-compatible API for storing unstructured data.
- vector databaseuses · A vector database is a data-management system that stores and indexes numerical vector representations and supports retrieval through similarity or distance searches.
- ingestion layercontrasts with · An ingestion layer is a system layer that collects data from source systems, transforms and cleans it, and loads it into storage tables.
Questions about storage abstraction
What is storage abstraction?
A storage layer is a software subsystem or interface responsible for persisting and retrieving data while separating applications from underlying storage technology.
Which Manning books explain storage abstraction?
Architecting an Apache Iceberg Lakehouse and Designing AI Systems.
What is the difference between storage abstraction and ingestion layer?
A storage layer is a software subsystem or interface responsible for persisting and retrieving data while separating applications from underlying storage technology. An ingestion layer is a system layer that collects data from source systems, transforms and cleans it, and loads it into storage tables.
What kinds of storage abstraction are there?
Kinds of storage abstraction include Kafka broker.
What is storage abstraction part of?
Storage abstraction is part of data lakehouse. A data lakehouse is a data architecture pattern combining the flexible storage of data lakes with the governance, reliability, and analytical capabilities of data warehouses.
What are the parts of storage abstraction?
Storage abstraction includes MinIO.
These guides are generated from the text of Manning books by Manning's knowledge graph and link to the sections they draw on. The full books are available on liveBook and at manning.com.

