Columnar storage
Columnar storage is a data layout that stores values grouped by column rather than by row.
Cloud Observability in Action
Columnar storage is an arrangement of tabular data that stores values from the same column together for efficient analytical access.
- How it is used
- Supports analytical observability workloads involving high-cardinality telemetry and large historical datasets.
- How deep it goes
- Explains physical layout, OLTP-versus-OLAP trade-offs, telemetry setup, querying, and profiling storage.
- Examples
- Uses row-versus-column diagrams, a ClickHouse Docker Compose deployment, OpenTelemetry configuration, SQL queries, and persistent profile storage.
- What you can do after
- Evaluate storage layouts and select columnar backends for high-cardinality analytical workloads while weighing operational costs.
Read how Cloud Observability in Action explains columnar storage →
Fast Python: High performance techniques for large datasets
- How it is used
- Optimizes tabular analytics by reducing memory use, accelerating I/O and computation, and enabling efficient data exchange.
- How deep it goes
- Explains layout, typing, encoding, compression, and interoperability, but defers detailed persistent formats and on-disk implementations.
- Examples
- Uses pandas dtype optimization, PyArrow CSV conversion, Parquet storage, and New York City taxi records.
- What you can do after
- Operate/configure pandas, Arrow, and Parquet workflows to select, type, compress, store, and transfer tabular columns.
Read how Fast Python: High performance techniques for large datasets explains columnar storage →
Just Use Postgres!
A columnar store is a storage layout that organizes values by column to improve analytical scans over selected columns and large data ranges.
- How it is used
- Optimizes analytical scans over large or historical time intervals when indexed row-store access becomes less efficient.
- How deep it goes
- Defines partition-level columnar storage and its placement, but defers Hypercore implementation details to official documentation.
- Examples
- Uses a policy that retains three recent months in indexed row storage and moves older partitions into an unindexed columnar store.
- What you can do after
- Operate/configure TimescaleDB by separating recent indexed row data from older columnar partitions.
Architecting an Apache Iceberg Lakehouse
A columnar format stores values from the same field together to support efficient analytical reading and compression.
- How it is used
- Supports analytical scans, aggregations, dashboards, and long-range queries without reading unused fields from transactional rows.
- How deep it goes
- Explains storage layout, column pruning, compression, vectorized execution, and the division between Parquet and Iceberg.
- Examples
- Uses Parquet files, Iceberg tables, and conversion from CSV, JSON, Avro, or row-based data as concrete artifacts.
- What you can do after
- Evaluate storage formats and plan columnar conversion for analytical pipelines and Iceberg migrations.
Read how Architecting an Apache Iceberg Lakehouse explains columnar storage →
Also covered in
What to know first
- ParquetApache Parquet is an open, column-oriented binary file format for storing and processing structured data with schema and metadata support.
- CSVCSV is a text format for tabular data in which records contain fields separated by commas, with conventions for quoting fields that contain special characters.
- dataset splitA dataset split is the division of a dataset into distinct subsets assigned different purposes, such as training, validation, or evaluation.
- compressionCompression is the encoding of data or information in a more compact representation to reduce its size while retaining content needed for recovery or use.
Related concepts
- Parquetincludes · Apache Parquet is an open, column-oriented binary file format for storing and processing structured data with schema and metadata support.
- analytical processinguses · Interactive processing of data through complex queries, transformations, and aggregations to support analysis, reporting, or decision-making.
- Apache Arrowincludes · Apache Arrow is an open, language-independent specification and software ecosystem for representing and processing columnar data in memory.
- ClickHouseincludes · ClickHouse is an open-source, column-oriented database management system designed for high-performance analytical queries.
- data lakeused by · A data lake is a repository that stores large volumes of data in raw or lightly processed form and varied formats for later processing and analysis.
Questions about columnar storage
What is columnar storage?
Columnar storage is a data layout that stores values grouped by column rather than by row.
Which Manning books explain columnar storage?
Cloud Observability in Action, Fast Python: High performance techniques for large datasets, Just Use Postgres! and Architecting an Apache Iceberg Lakehouse.
What should I know before learning columnar storage?
Manning books that teach columnar storage usually introduce Parquet, CSV, dataset split and compression first.
What is the difference between columnar storage and row store?
Columnar storage is a data layout that stores values grouped by column rather than by row. A row store is a storage layout that keeps the fields of each record together, supporting efficient access to complete records.
What kinds of columnar storage are there?
Kinds of columnar storage include Parquet, Apache Arrow and ClickHouse.
These guides are generated from the text of Manning books by Manning's knowledge graph and link to the sections they draw on. The full books are available on liveBook and at manning.com.



