9 Building processors and Fluent Bit extension options

 

This chapter covers

  • Applying processors to interact and manipulate metrics, traces, and logs
  • Using SQL-like expressions to work with signals using a processor
  • Exploring the need for a custom plugin
  • Reviewing the technology options for creating custom plugins
  • Understanding of the technologies involved in custom plugins

This chapter looks at different approaches to extending Fluent Bit logic. Using Lua, we’ll look at new feature processors that allow us to extend logic around the input and output of all signal types (logs, traces, and metrics). If this isn’t sufficient, we can consider developing our own plugins. We can build new plugins by forking or contributing to the Fluent Bit project or using the technologies provided to create independent plugins. In this chapter, we’ll examine what is involved in the options and their pros and cons.

9.1 Architectural context

Processors provide the opportunity to enhance input and output plugin behavior in several ways without resorting to custom plugins. However, processors allow us to go only so far; sometimes, we must consider developing custom plugins to extend Fluent Bit. To support this, we’ll explore the design decisions when selecting a strategy to develop an extension. Figure 9.1 highlights where the choice of strategy can affect Fluent Bit. In chapter 10, we will apply one of these custom plugin strategies.

9.2 Fluent Bit processor: Changing the behavior of existing plugins

9.2.1 Processor with Lua for logs

9.2.2 Content modifier processor

9.2.3 Processor for traces

9.2.4 Processor to metrics

9.2.5 Processor using SQL

9.3 Why we need to extend Fluent Bit

9.4 C language

9.4.1 Considerations

9.4.2 Benefits

9.4.3 Drawbacks

9.4.4 Tools for the job

9.5 Go language

9.5.1 Benefits

9.5.2 Drawbacks