1 Introduction to Fluentd

Before getting stuck into the workings of Fluentd this chapter is going to focus in on the motivations for using a tool such as Fluentd. We’ll highlight the kinds of activities that it can help or enable. As we do this, it is worth taking time to understand how Fluentd has evolved, and why it holds the position it does within the industry. If you’re considering Fluentd as a possible tool, or looking to make a case for it’s adoption then it is useful to understand the ‘origins story’ as this helps inform how Fluentd maybe perceived, where it might be in terms of adoption – niche, mainstream or possibly even something that is dated.

Let’s also take a step back and understand some contemporary thinking around monitoring. After all, a tool is only as good as the person using it. So if we take a moment to touch upon the ideas, we’ll be able to make better use of our tool.

1.1           Why do we produce logs?

1.1.1   Four Golden Signals

1.1.2   Three Pillars of Observability

1.2           Unifying logs vs log analytics

1.3           Log routing vs log aggregation

1.4           Software stacks

1.5           Log Event Lifecycle

1.6           Log routing as a vehicle for security

1.7           Evolution of Fluentd

1.7.1   Treasure Data

1.7.2   CNCF

1.7.3   Relationship to major cloud vendors PaaS/IaaS

1.8           The relationship between Fluentd and Fluent-Bit

1.9           Where can it be used

1.9.1   Platform constraints

1.10       Fluentd configuration UI based editing

1.11       Plugins

1.12       Use Cases for Fluentd and Fluent Bit

1.12.1   Actionable Log Events

1.12.2   Making Logs More Meaningful

1.12.3   Polyglot Environments

1.12.4   Multiple Types of Source and Target

1.12.5   Log Data Costs

1.12.6   Logs to metrics

1.13       Summary