6 Parsing to extract more meaning
This chapter covers
- Exploring the relationship between filters, parsers, and decoders.
- Examining the prebuilt parsers and how they work
- Using filters to run parser processes
- Implementing REGEX (Regular Expression) and JSON parsers to extract meaning from log content.
In this chapter, we will start working with Fluent Bit’s capabilities to examine and manipulate the data it collects and outputs. As we can see in the following figure, this means that parsers and filters sit within the middle of the pipeline of processing log events after we’ve ingested the data.
Figure 6.1 Logical architecture of Fluent Bit, with this Chapter's focus on parsers highlighted. Parsers are used in special cases with input plugins, but primarily through their relationship with Filters.
The benefit of parsers being separated from the input and output plugins is that we can apply the same parsing processes to different sources. This makes parsers highly reusable, and as we’ll see. In addition to being able to use Parsers with inputs, there is also a strong relationship between Parsers and Filters as we’ll see.
6.1 The goal of parsing
Parsers are one of the most important tools when it comes to extracting meaning from log events. The use of parsers allows us to find important values. The main reasons for parsing are: