A service block is the Fluent Bit configuration section for global behavior. It controls output retry scheduling and can expose an embedded HTTP server for health and plugin metrics. section 5.2.1 section 5.3.1
Use service-block attributes to control resilience and observability:
- Set scheduler.base to the minimum delay before retrying and scheduler.cap to the maximum delay. section 5.2.1
- Use retry_limit in an output configuration to stop retries after a defined number of attempts and limit backpressure. section 5.2.1
- Enable http_server, then set http_listen and http_port to expose the monitoring API. section 5.3.1
- Set storage.metrics to on when plugin metrics collection is required. section 5.3.1
Bounded retries for a forward output
[SERVICE] flush 1 log_level warn scheduler.base 2 scheduler.cap 15 [OUTPUT] name forward match * host 0.0.0.0 port 9980 retry_limit 6 Fluent Bit randomizes retry delays between the base and an exponentially increasing upper bound, never exceeding the cap. The retry limit prevents indefinite retries when the destination is unavailable. section 5.2.1
Listing 5.1 Controlling retries: http-forward-retries.conf
[SERVICE] flush 1 log_level warn scheduler.base 2 #1 scheduler.cap 15 #2 [INPUT] name http listen 0.0.0.0 port 9881 successful_response_code 201 success_header x-fluent-bit received tag http [OUTPUT] name forward match * host 0.0.0.0 port 9980 retry_limit 6 #3
Inspect the complete retry configuration, including the HTTP input, forward output, scheduler values, and retry limit. section 5.2.1
Table 5.1 Standard URLs for Fluent Bit metrics and health data
|
URI (http://host:port)
|
Description
|
|---|---|
/
|
Returns the build flags that have been set. The build process allows us to enable or disable the inclusion of features and plugins using build flags. So, if we need to compact the Fluent Bit binary footprint, we can exclude particular plugins, such as
FLB_HAVE_AWS, if we don’t want to interact with the Amazon Web Services (AWS) plugins, such as Kinesis.
In addition, we have the version of Fluent Bit and the edition returned, which represents whether we’ve got the open source (community) version or an enhanced version from an organization such as Calyptia.
|
/api/v1/health
|
Same as the root response.
|
/api/v1/metrics
|
Provides the throughput of the plugin as the number of records and data volume in bytes. Other measures may be provided, depending on the plugin. Output, for example, will include errors and retries. These figures are cumulative.
|
/api/v1/metrics/prometheus
|
Presents the same metrics data but in a manner that Prometheus can collect and process, allowing us to combine it with tools like Grafana to visualize what Fluent Bit is doing.
|
/api/v1/storage
|
Provides meaningful information about how the buffers are being used with the number of chunks, the amount of physical storage involved, and the amount of memory being consumed.
|
/api/v2/metrics/prometheus
|
Exposes any metrics provided by loaded plugins (such as those used in the loaded configuration, in addition to the core measurements that the v1 API produces. The response is formatted as a Prometheus metrics payload.
|
/api/v2/metrics
|
Uses a new CMetrics (
https://github.com/fluent/cmetrics) library, which has taken its inspiration from a Go Prometheus package format (
https://mng.bz/o01v).
The key difference between this metrics output and the Prometheus v2 path is that the Prometheus
#HELP and
#TYPE entries are not included in the output. The actual metric value is still in Prometheus format but prefixed with a timestamp.
|
/api/v2/reload
|
Triggers the Fluent Bit node to execute a hot reload of its configuration without requiring the Fluent Bit instance to restart. It requires Fluent Bit to have the hot reload option switched on using the command-line parameter
--enable-hot-reload,
http_listen, and
http_port
.
|
Browse the standard monitoring URLs used to inspect build information, health, throughput, record counts, and plugin data. section 5.3.1
