5 Outputting events

 

This chapter covers

  • Examining the common characteristics of output plugins
  • Generating logs for the console, files, and other local outputs
  • Storing logs and metrics with Prometheus and PostgreSQL
  • Forwarding signals to another OpenTelemetry of Fluent Bit node
  • Exploring other monitoring tools using HTTP and hyper scaler endpoints.

We’ve explored capturing events (logs, traces, and metrics) from various sources using different types of plugins such as files, OS data, and network-based events such as OpenTelemetry.

The next step is to store the events captured. We can see in the following diagram which part of the architecture we’re looking at. While it may look like we’re jumping a lot of steps, as with mastering any programming language. We always start with I/O before we start mastering the clever constructs available. But we will touch on some very basic aspects of details, such as filters.

Figure 5.1 Fluent Bit architecture and which areas will be covered by this chapter.

5.1 Common characteristics of Fluent Bit output plugins

5.1.1 The retry Output resilience – with scheduling

5.1.2 Network Controls

5.1.3 Worker threads

5.2 Null output

5.2.1 Fluent Bit monitoring

5.2.2 Configuring Null output

5.3 Sending log events to the console

5.3.1 Formatting outputs

5.3.2 Seeing matching at work

5.4 Writing to files

5.5 Prometheus Outputs

5.5.1 Prometheus Node Exporter

5.5.2 Prometheus Fluent Bit Exporter

5.5.3 Prometheus Remote Writer

5.6 PostgreSQL output

5.6.1 Performance Considerations

5.7 HTTP output

5.8 Forwarding to other Fluent nodes

5.9 OpenTelemetry

5.10 Hyperscaler Native and SaaS Observability

5.11 Summary

sitemap