Part 3 Plugins and queries

Part 3 explores Fluent Bit’s advanced capabilities. Chapter 8 covers stream processing, showing how it allows us to perform queries on events and time series calculations as events pass through Fluent Bit. This gives Fluent Bit a Kafkaesque ability to perform time series calculations. We’ll explore the limits of what can be queried and turn our query answers into new events.

Chapter 9 examines the idea behind processors, a feature of input and output plugins that allows us to incorporate advanced manipulation of logs, traces, and metrics. If this feature isn’t sufficient to solve our input and output needs, we have the option of creating custom plugins. Each approach has pros and cons, so we’ll suggest the right one for different needs.

This leads us neatly into chapter 10, which takes us through the process of building custom plugins. We apply only one of the options in chapter 9, but it is the easiest to follow and fits most of the needs we’re likely to encounter. By the end of this chapter, we’ll build one input and one output plugin that can work with databases such as MySQL and can be easily extended to work with other databases.