3 Capturing inputs

 

This chapter covers

  • Reading log events from files
  • Capturing console logging
  • Ingesting OpenTelemetry and data from other Fluent Bit instances
  • Integrating with logging frameworks

This chapter is all about how we can capture the metrics, traces, and, most importantly, logs using Fluent Bit. Fluent Bit has a respectable portfolio of plugins. To help understand the relationships between the different parts of Fluent Bit, let's see how the inputs fit with respect to the overall logical architecture, as shown in Figure 3.1

Figure 3.1 Logical architecture of Fluent Bit, with this Chapter's focus highlighted.

If we tried to illustrate every plugin available, we could easily fill this book and never get as far as looking at what to do with the captured events. To address this challenge, we can group some of the inputs with similar characteristics and then look at representative plugins for each group. As Fluent Bit has a high level of consistency, an example for each group would provide a sound basis for extrapolating what can be expected for other similar source plugins that we don’t discuss. We can characterize the inputs like this:

3.1 OS and device sources

3.1.1 Tuning monitoring sources

3.1.2 Device sources

3.2 Stdout

3.3 File-Based log events

3.4 Capturing log files

3.4.1 Simple file consumption

3.4.2 Supporting long-running processes

3.4.3 Capturing logs for short-lived applications

3.5 Network Events and communicating between Fluent Bit (and Fluentd) nodes

3.5.1 Network Input Sources

3.5.2 HTTP Source

3.5.3 Securing communication with SSL/TLS

3.5.4 Forward Source

3.5.5 Inter node communication

3.5.6 OpenTelemetry

3.6 Fluent Bit Buffers and Chunks

3.7 Other Sources

3.7.1 Container related

3.7.2 Getting data from other processes

3.7.3 Monitoring the observers

3.8 Summary

sitemap