Retention policy
A retention policy specifies how long data or records are kept and the conditions under which they may be archived, deleted, or otherwise disposed of.
Architecting an Apache Iceberg Lakehouse
A retention policy specifies how long table states and associated files or metadata may be retained before expiration or deletion.
- How it is used
- Controls Iceberg storage growth, historical access, secure deletion, and governance through coordinated snapshot and file lifecycle rules.
- How deep it goes
- Explains snapshot expiration, orphan cleanup, write-mode effects, references, and orchestration while deferring full workload-specific policy design.
- Examples
- Uses a seven-day Airflow expiration DAG, snapshot references, tags, branches, and COW or MOR file-lifecycle scenarios.
- What you can do after
- Operate/configure Iceberg lifecycle jobs that expire snapshots, distinguish orphan cleanup, and preserve required recovery or governance references.
Read how Architecting an Apache Iceberg Lakehouse explains retention policy →
Kafka for Architects
A retention policy is a rule governing how long or under what conditions stored Kafka records remain available.
- How it is used
- Governs topic history available for consumer recovery and replay, independently of whether records have been processed.
- How deep it goes
- Explains time, size, timestamps, and topic configuration, but leaves storage-segment internals to an external resource.
- Examples
- Uses a five-day dashboard topic configuration and a purchase-notification traffic surge to illustrate policy choices.
- What you can do after
- Operate/configure it with retention.ms, choosing a window that accommodates consumer disconnections and storage needs.
Fundamentals of Cybersecurity
A data retention policy specifies the types of data an organization preserves and the duration for which it preserves them.
- How it is used
- Sets record-specific keeping periods to meet obligations and business needs while preserving information needed for investigations.
- How deep it goes
- Defines the policy and explains competing retention needs with examples, but does not provide a step-by-step method for setting periods.
- Examples
- Contrasts financial records with employee emails and cites PCI DSS requirements for audit-log retention and immediate availability.
- What you can do after
- Apply it by assigning different periods to record types and periodically reviewing records for unnecessary retention.
Read how Fundamentals of Cybersecurity explains retention policy →
100 SQL Server Mistakes and How to Avoid Them
A data retention policy specifies how long stored data must be preserved before it may be deleted or archived.
- How it is used
- Uses retention duration to forecast database, backup, and storage requirements before capacity becomes an operational problem.
- How deep it goes
- Calculates linear data growth and associated storage needs, but stops short of defining deletion or archival procedures.
- Examples
- Illustrates retention through MarketingArchive, 5 GB daily ingestion, three-year retention, and air-gapped or immutable backups.
- What you can do after
- Evaluate storage forecasts and backup safeguards using retention duration, growth rate, headroom, and protection requirements.
Read how 100 SQL Server Mistakes and How to Avoid Them explains retention policy →
MongoDB 8.0 in Action, Third Edition
A retention policy defines how long backup copies or recovery data are preserved.
- How it is used
- Sets how long Atlas backup snapshots remain available to meet recovery and compliance needs.
- How deep it goes
- Shows schedule output with separate frequency and retention columns, but does not supply a command for changing retention settings.
- Examples
- Displays an Atlas CLI schedule with hourly through yearly snapshots and their retention values.
- What you can do after
- Operate/configure it by inspecting an Atlas backup schedule and distinguishing snapshot retention from a point-in-time recovery window.
Read how MongoDB 8.0 in Action, Third Edition explains retention policy →
Effective Data Analysis
A data retention policy specifies how long data may be kept and the conditions under which it must be anonymized or deleted.
- How it is used
- Uses retention limits to minimize stored personal data and support responsible analytics and privacy compliance.
- How deep it goes
- Explains retention periods and post-period actions, but does not specify detailed schedules, systems, or deletion procedures.
- Examples
- Uses e-commerce behavior data, anonymized user IDs, generalized locations, and a one-year retention example.
- What you can do after
- Operate/configure retention rules by setting periods and selecting deletion, scrubbing, anonymization, or access restrictions.
Read how Effective Data Analysis explains retention policy →
Apache Kafka in Action
A retention period is the duration for which Kafka retains records before allowing them to be deleted.
- How it is used
- Controls Kafka data availability, partition growth, retrieval windows, and historical-storage cost.
- How deep it goes
- Explains time- and size-based triggers, segment deletion, configuration, and partition effects without detailing broader retrieval implementation.
- Examples
- Demonstrates retention.ms=60000 on a topic and inspects the resulting partition files and segment replacement.
- What you can do after
- Operate/configure it by setting retention.ms or retention.bytes and checking segment deletion after records expire.
Just Use Postgres!
A data retention policy is a rule that automatically removes time-series chunks containing data older than a specified age.
- How it is used
- Automates removal of expired time-series chunks from hypertables and continuous aggregates.
- How deep it goes
- Defines retention intervals and cleanup schedules, but defers broader administration and lifecycle design.
- Examples
- Uses 30-day heart-rate measurement retention and seven-day low-heart-rate aggregate retention.
- What you can do after
- Operate/configure TimescaleDB cleanup with add_retention_policy and adjustable schedule intervals.
Elasticsearch in Action, Second Edition
A snapshot retention policy specifies the conditions and duration governing how long snapshots remain available.
- How it is used
- Automates snapshot expiry within scheduled Elasticsearch backup policies.
- How deep it goes
- Defines retention as a duration and demonstrates a seven-day JSON setting, without detailing expiration internals.
- Examples
- Uses midnight backups of movie and review indexes with date-based names and seven-day expiration.
- What you can do after
- Operate/configure retention through the Elasticsearch _slm API and a policy JSON request.
Read how Elasticsearch in Action, Second Edition explains retention policy →
Think Like a CTO
A retention policy specifies how long data must be kept and when or how it may be deleted or archived.
- How it is used
- Treats retention as a data-management lens for deciding how long each data bucket is kept and when it changes location or is deleted.
- How deep it goes
- Explains durations across operational stores and backups, storage aging, and deletion triggers without specifying exact periods.
- Examples
- Uses a client leaving as a business event that may trigger deletion, including from backups.
- What you can do after
- Operate data handling by assigning retention periods, aging rules, and deletion triggers to data buckets.
Build AI-Enhanced Web Apps
A retention policy specifies how long stored backups or other records are preserved before deletion.
- How it is used
- Governs storage duration and deletion for chat histories, backups, and other records within data protection and deployment operations.
- How deep it goes
- Explains retention decisions, deletion obligations, user communication, and backup practice, but does not provide implementation code or scheduling details.
- Examples
- Uses chat histories and daily backups with short retention as concrete policy applications.
- What you can do after
- Operate/configure a retention approach by setting periods, communicating them, and permanently deleting expired records.
Read how Build AI-Enhanced Web Apps explains retention policy →
Context Engineering
A retention policy specifies rules for how long records are kept and when they are deleted.
- How it is used
- Governs deletion of stale or revoked records in an agent’s long-term memory.
- How deep it goes
- Identifies deletion triggers but does not detail policy rules or implementation.
- Examples
- Uses stale or revoked records and user privacy requests as deletion cases.
Also covered in
- Cloud Observability in Action
- GitHub Actions in Action: Continuous integration and delivery for DevOps
- Build Your Own Robot: Using Python, CRICKIT, and Raspberry Pi
- The Art of Code
- Designing AI SystemsIn early access (MEAP)
- Learn Go with Pocket-Sized Projects
- Amazon Web Services in Action, Third Edition: An in-depth guide to AWS
- Kafka Streams in Action, Second Edition
- Microsoft Azure in Action
- The Art of AI Product Development
- LLM Customization and Fine-TuningIn early access (MEAP)
- Python How-To
What to know first
- GDPRThe General Data Protection Regulation (GDPR) is a European Union regulation governing the processing and protection of personal data and establishing related rights for individuals.
- complianceCompliance is conformity with applicable legal, regulatory, contractual, policy, or other governing requirements.
- HIPAAHIPAA is a United States federal law establishing requirements for the privacy, security, and handling of protected health information and related healthcare transactions.
- authorizationAuthorization is the determination of which actions or resources an identity or other entity is permitted to access under defined permissions or policies.
Related concepts
- logconfigures · A log is a time-ordered record of events, messages, or activity generated by a system or process.
- training datasetconfigures · A training dataset is a collection of examples supplied to a learning procedure to fit or adapt a model’s parameters or behavior.
- GDPRconfigured by · The General Data Protection Regulation (GDPR) is a European Union regulation governing the processing and protection of personal data and establishing related rights for individuals.
- automated snapshotconfigures · An automated snapshot is a database snapshot created automatically by a managed database service according to a configured schedule and retention policy.
- continuous aggregateuses · A continuous aggregate is a TimescaleDB query-result object that stores precomputed aggregate data and refreshes it incrementally according to a defined policy.
Questions about retention policy
What is retention policy?
A retention policy specifies how long data or records are kept and the conditions under which they may be archived, deleted, or otherwise disposed of.
Which Manning books explain retention policy?
Architecting an Apache Iceberg Lakehouse, Kafka for Architects, Fundamentals of Cybersecurity, 100 SQL Server Mistakes and How to Avoid Them, MongoDB 8.0 in Action, Third Edition, Effective Data Analysis, Apache Kafka in Action, Just Use Postgres! and 4 more.
What should I know before learning retention policy?
Manning books that teach retention policy usually introduce GDPR, compliance, HIPAA and authorization first.
These guides are generated from the text of Manning books by Manning's knowledge graph and link to the sections they draw on. The full books are available on liveBook and at manning.com.











