MinIO
MinIO is an open-source object-storage server that provides an Amazon S3-compatible API for storing unstructured data.
Architecting an Apache Iceberg Lakehouse
MinIO is an S3-compatible object-storage system that can be deployed on premises or in other infrastructure.
- How it is used
- Provides the object-storage layer for Iceberg tables in local, on-premises, hybrid, and edge lakehouse architectures.
- How deep it goes
- Explains deployment, S3 integration, performance, and storage features, but stops short of detailed implementation internals.
- Examples
- Uses a Docker Compose lakehouse, a warehouse bucket, Spark ingestion, MinIO console inspection, and an on-premises bank deployment.
- What you can do after
- Operate/configure it as an S3-compatible Iceberg storage backend and verify stored table data through the MinIO console.
Read how Architecting an Apache Iceberg Lakehouse explains MinIO →
Machine Learning Platform Engineering
MinIO is an open-source object-storage server that provides an Amazon S3-compatible interface for storing unstructured data.
- How it is used
- Stores external ML datasets and feature files for tracked training runs, Feast feature definitions, and Kubeflow data-preparation workflows.
- How deep it goes
- Shows bucket creation, client configuration, uploads, port-forwarding, and S3-style references; leaves MinIO versioning details to documentation.
- Examples
- Uses run-specific training datasets, a Parquet demographic feature source, and uploads of object-detection images and labels.
- What you can do after
- Operate/configure it to store prepared datasets in a bucket and point MLflow or Feast at their external locations.
Read how Machine Learning Platform Engineering explains MinIO →
Data Pipelines with Apache Airflow, Second Edition
MinIO is an object-storage server that provides an Amazon S3-compatible API.
- How it is used
- Provides S3-compatible object storage for Airflow workflows that transfer data between databases, APIs, file shares, and processing tasks.
- How deep it goes
- Defines MinIO and demonstrates its containerized pipeline role, without detailing storage internals.
- Examples
- Uses Airbnb listings and New York transportation pipelines to demonstrate MinIO alongside databases, APIs, file shares, and Airflow.
- What you can do after
- Operate/configure MinIO through Docker Compose as an S3-compatible storage endpoint in a multi-service data pipeline.
Read how Data Pipelines with Apache Airflow, Second Edition explains MinIO →
Related concepts
- object storekind of · An object store is a storage system that stores and retrieves data as independently addressable objects containing content and associated metadata.
- S3 APIimplements · The S3 API is an object-storage interface for storing, retrieving, and managing objects and buckets according to Amazon S3 conventions.
- cloud storageuses · Cloud storage is remotely managed storage delivered as a network-accessible service for persistently storing and retrieving data.
- Docker Composeconfigured by · Docker Compose is a tool for defining and running applications composed of multiple containers through declarative configuration.
These guides are generated from the text of Manning books by Manning's knowledge graph and link to the sections they draw on. The full books are available on liveBook and at manning.com.