7 Filtering and transforming events

 

This chapter covers

  • Creating content and attribute-based filtering
  • Modifying event content to enrich, structure, and route events
  • Transforming content from logs to metrics
  • Rewriting tags to help downstream matching
  • Grepping events to identify events for filtering
  • Limiting events to prevent backend overloading
  • Coding filters using Lua

Chapter 6 looked at parsers and the filter that enables them. This chapter looks at the rest of the filters available to us. We will look carefully at filters that can manipulate the tag value, as this is central to how we route logs, metrics, and traces to different output destinations.

7.1 Architectural context

Figure 7.1 shows how filters fit into our logical Fluent Bit architecture.

Figure 7.1 The application of filters we’re going to consider in this chapter is highlighted in our logical architecture.
figure

We can group filters in the following ways:

7.2 Integrating and enriching with filters

7.2.1 Directing and securing logs with GeoIP

7.2.2 Using the CheckList filter

7.3 Extending and amending with filters

7.3.1 Taking a brief look at the nest filter

7.3.2 Illustrating the record_modifier filter

7.3.3 Illustrating the modify filter

7.3.4 Bringing it together

7.3.5 Testing filters

7.4 Routing and controlling

7.4.1 Using the record accessor

Summary

sitemap