3 Capturing inputs
This chapter covers
- Reading log events from files
- Capturing console logging
- Ingesting OpenTelemetry data
- Integrating with logging frameworks
This chapter is all about how we can capture the metrics, traces, and, most importantly, logs using Fluent Bit. Fluent Bit has a respectable portfolio of plugins. To help understand the relationships between the different parts of Fluent Bit, let's see how the inputs fit with respect to the overall logical architecture, as shown in Figure 3.1
Figure 3.1 Logical architecture of Fluent Bit, with this Chapter's focus highlighted.
If we tried to illustrate every plugin available, we could easily fill this book and never get as far as looking at what to do with the captured events. To address this challenge, we can group some of the inputs with similar characteristics and then look at representative plugins for each group. As Fluent Bit has a high level of consistency, an example for each group would provide a sound basis for extrapolating what can be expected for other similar source plugins that we don’t discuss. We can characterize the inputs like this:
- OS level events from systemd, Windows Event logs, serial interfaces
- System metrics such as CPU, Disk I/O, and memory metrics
- Application-level sources such as log files with prebuilt or custom log formats
- Event-driven sources such as MQTT, Open Telemetry, and other Fluent nodes
- Network/web-based pulled data such as Prometheus scraping, Kubernetes style health checking