liveBook · Manning

Data tiering

As explained in Apache Kafka in Action

Tiered storage is a storage strategy that places data on different storage tiers according to factors such as age, cost, or access frequency.

Data tiering keeps frequently accessed data in Kafka while moving older or less critical data to lower-cost storage. It helps organizations retain historical data without paying the full cost of keeping everything in the primary Kafka environment. section 18.4.2 section 18.6

A practical tiering arrangement separates recent operational access from historical retention:

  • Keep frequently accessed data in Kafka for responsive retrieval by interested systems. section 18.4.2
  • Offload older or less critical data to more cost-effective storage. section 14.4 section 18.6
  • Plan retrieval across storage locations because data access can depend on its age. section 18.4.2

Recent events and historical events

An organization can retain recent event data in Kafka for systems that need near-real-time access, then move older historical events to lower-cost storage. This preserves long-term retention while requiring retrieval processes to account for where data resides. section 18.4.2 section 18.6

How Apache Kafka in Action covers it

How deep it goes
Explains the retention split and retrieval complication but does not provide configuration steps or implementation internals.
Examples
Uses recent Kafka data and offloaded historical events to illustrate age-based access and retention-cost trade-offs.
What it assumes
Assumes familiarity with Kafka as a near-real-time data exchange and extended-retention system.
What you can do after
Evaluate retention designs by balancing access performance, historical availability, retrieval complexity, and storage cost.
How it connects
Connects Kafka with data retention, historical data, near-real-time access, performance, and storage cost management.

Also explained in

Compare how 2 books explain data tiering →

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Apache Kafka in Action. The section links point to the parts of the book it draws on.