liveBook · Manning

Storage abstraction

Also called storage layer

A storage layer is a software subsystem or interface responsible for persisting and retrieving data while separating applications from underlying storage technology.

How 2 Manning books explain it

Architecting an Apache Iceberg Lakehouse

A storage layer is a lakehouse architectural layer that stores data on cost-effective storage systems.

How it is used
Frames storage selection as an architectural decision balancing reliability, scalability, performance, security, cost, and operational complexity.
How deep it goes
Explains requirements, storage models, standards, consistency, and selection tradeoffs, but stops short of detailed vendor configuration.
Examples
Uses HDFS, MinIO, S3, Parquet, Docker Compose, Spark, Nessie, Dremio, and Superset in architecture and deployment examples.
What you can do after
Evaluate storage technologies against organizational requirements and select a durable Iceberg lakehouse strategy without assuming one universal platform.

Read how Architecting an Apache Iceberg Lakehouse explains storage abstraction →

Designing AI Systems

A storage abstraction is an interface or design boundary that separates data-storage operations from the underlying persistence technology.

How it is used
Separates session-service behavior from persistence choices so deployments can select storage according to reliability, latency, scale, or operations needs.
How deep it goes
Defines interface delegation and constructor injection, then demonstrates a gRPC service implementation while leaving backend-specific mechanics behind the interface.
Examples
Contrasts PostgreSQL, Redis, DynamoDB, MongoDB, and other backends, then applies the pattern to sessions and vector similarity search.
What you can do after
Build it from scratch by defining a storage interface, implementing a backend, and injecting that implementation into a service.

Read how Designing AI Systems explains storage abstraction →

Also covered in

Related concepts

Questions about storage abstraction

What is storage abstraction?

A storage layer is a software subsystem or interface responsible for persisting and retrieving data while separating applications from underlying storage technology.

Which Manning books explain storage abstraction?

Architecting an Apache Iceberg Lakehouse and Designing AI Systems.

What is the difference between storage abstraction and ingestion layer?

A storage layer is a software subsystem or interface responsible for persisting and retrieving data while separating applications from underlying storage technology. An ingestion layer is a system layer that collects data from source systems, transforms and cleans it, and loads it into storage tables.

What kinds of storage abstraction are there?

Kinds of storage abstraction include Kafka broker.

What is storage abstraction part of?

Storage abstraction is part of data lakehouse. A data lakehouse is a data architecture pattern combining the flexible storage of data lakes with the governance, reliability, and analytical capabilities of data warehouses.

What are the parts of storage abstraction?

Storage abstraction includes MinIO.

These guides are generated from the text of Manning books by Manning's knowledge graph and link to the sections they draw on. The full books are available on liveBook and at manning.com.