liveBook · Manning

Throttle filter

As explained in Logs and Telemetry

A throttle plugin is a Fluent Bit plugin that limits the rate at which events are released by dropping events that exceed the configured allowance.

The throttle filter controls Fluent Bit event throughput by releasing only the configured allowance and dropping events that exceed it. Use it to protect analytical or backend systems from overload while preserving higher-value processing capacity. section 7.1 section 7.5.1

Configure the filter around four operational choices:

  • rate sets the maximum number of events permitted per interval. section 7.5.1
  • interval defines the time period used to evaluate that allowance. section 7.5.1
  • window controls the window size used by the throttle calculation. section 7.5.1
  • print_status reports the configured limit and current rate, making saturation visible in Fluent Bit logs. section 7.5.1
  • Prefer throttling lower-severity trace, debug, and info logs when losing events is acceptable; otherwise, archive all logs before protecting analytics with throttling. section 7.5.1

Throttle a dummy stream

A filter with rate 2, window 3, and interval 2s allows up to two events per two-second interval. Run it with `fluent-bit -c fluentbit/throttle-demo.conf`; status output reports `limit is 2.00 per 2s` and the current rate. The related throttle_size option applies windowing while evicting by message or message-element size. section 7.5.1

Listing 7.5 Example of throttling: throttle-demo.conf
[SERVICE]
  flush 10

[INPUT]
    name dummy
    copies 5       #1
    rate 1
    dummy {"message":"dummy"}
    tag throttle_demo

[FILTER]
    name throttle
    match *
    rate 2           #2
    window 3          #3
    interval 2s            #4
    print_status true       #5

[OUTPUT]
    name stdout
    match *

Inspect the complete throttle-demo.conf listing, including the dummy input, throttle settings, status reporting, and stdout output. section 7.5.1

How Logs and Telemetry covers it

How deep it goes
Explains rate, window, interval, status reporting, severity selection, and file-based loss mitigation through a runnable configuration.
Examples
Uses a dummy input, throttle-demo.conf, stdout output, and status logs showing a two-per-two-second limit.
What you can do after
Operate/configure Fluent Bit to cap event throughput and inspect whether the configured rate is being reached.
How it connects
Connects throttling with filters, log levels, backend systems, analytical platforms, and size-based windowing.

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Logs and Telemetry. The section links point to the parts of the book it draws on.