liveBook · Manning

Inline compression

As explained in Architecting an Apache Iceberg Lakehouse

Inline compression is the automatic compression of data as it is written to storage, with decompression when it is retrieved.

Inline compression compresses data as it is written to storage, reducing its footprint without a separate postprocessing step. Retrieval remains transparent to users and applications. section 5.4.6

In storage architecture, inline compression:

  • Reduces the amount of data written and read, minimizing I/O and potentially improving query performance. section 5.4.6
  • Provides particular value for columnar formats such as Parquet and ORC. section 5.4.6
  • Operates as a built-in storage capability, including in MinIO object storage. section 5.4.6

How Architecting an Apache Iceberg Lakehouse covers it

How deep it goes
Defines write-time compression and its operational benefits, but does not detail algorithms, configuration, or decompression internals.
Examples
Uses Parquet and ORC data stored in object-storage systems as representative compression targets.
What you can do after
Evaluate storage designs by weighing transparent compression against footprint, I/O, and query-performance goals.
How it connects
Connects inline compression with MinIO, object storage, Parquet, ORC, I/O, and query performance.
Cover of Architecting an Apache Iceberg Lakehouse
liveBookmanning.com
Read Architecting an Apache Iceberg Lakehouse on liveBook Get the book at manning.com

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Architecting an Apache Iceberg Lakehouse. The section links point to the parts of the book it draws on.