liveBook · Manning

Log pruning

As explained in Cloud Observability in Action

Log pruning is the selective removal or reduction of log events before storage or further processing.

Log pruning selectively removes or reduces log events before storage or further processing, controlling ingestion and destination costs. section 2.3.3

Common pruning strategies include:

  • Sampling: drop a chosen percentage of events according to a heuristic. section 2.3.3
  • Type-based filtering: remove selected categories, such as DEBUG-level logs in production. section 2.3.3
  • Apply pruning at scale with offerings such as Cribl, including across providers. section 2.3.3

Pruning sits alongside agent and destination decisions. Because central storage can become expensive, pair event reduction with retention by data temperature: hot, warm, and cold logs have progressively slower and cheaper access. section 2.3.3

Table 2.1 Overview of the costs of logs (view table figure)

 

Overhead

Considerations

Instrumentation

Small and simple to add a log line

Widely adopted and supported by programming languages

Agent

Some runtime, lots of options, and few standards for formats (syslog, OTLP)

A runtime footprint, ingestion performance, reliability, and security

Destination

Long-term storage can be expensive.

Data temperature (access recency or frequency) can guide retention

Use the cost overview to compare agent overhead, ingestion considerations, and the storage implications that motivate pruning. section 2.3.3

How Cloud Observability in Action covers it

How deep it goes
Defines sampling and type filtering, then connects pruning to agents, destinations, and retention without specifying configuration syntax.
Examples
Uses heuristic sampling, production DEBUG filtering, Cribl, and hot-warm-cold retention categories as concrete operational examples.
What you can do after
Apply via Cribl or log agents by sampling events and filtering selected log types before central storage.
How it connects
Ties log pruning to log sampling, log filtering, log retention, agents, destinations, and data temperature.

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Cloud Observability in Action. The section links point to the parts of the book it draws on.