chapter two

2 Concepts, architecture, and deployment of Fluentd

 

This chapter covers

  • Outlining Fluentd's architecture & core concepts,
  • Reviewing the prerequisites and performing the deployment of Fluentd, Fluent Bit, and Fluent UI
  • Executing configurations of Fluentd and Fluent Bit to complete the classic first step in programming a 'Hello World' setup,
  • Introducing the structure of the configuration files.

In Chapter 1, we looked at the theory, industry trends, and use cases that Fluentd can help us. In this chapter, this is translated to how Fluentd works, and as a result, deploying and running the simplest of configurations to implement the traditional developer's 'hello world'.

2.1 Architecture and Core Concepts

Like driving a car, it is a lot easier when you have some basic appreciation of how the vehicle is powered (petrol, diesel, electric, Liquified Petroleum Gas (LPG)). The mental models that come with such understanding mean we can better relate to what to expect – whether we can expect to hear the engine rev, can the engine stall, how do the gears work (if there are any). For the same reason, before we start working with Fluentd and Fluent Bit. Based on this, we should run through some of the building blocks of Fluentd that will help with the mental models.

2.1.1 The makeup of a Log Event

One of the most important things to understand is how Fluentd handles a log event. Each log event is managed as a single JSON object made up of three elements, as shown in Figure 2.1.

2.1.2 Handling time

2.1.3 Architecture of Fluentd

2.1.4 Fluent configuration execution order

2.1.5 Directives

2.1.6 Exercise – Questions about log events and time settings

2.2 Deployment of Fluentd

2.2.1 Deploying Fluentd for the book's examples

2.2.2 Fluentd deployment considerations

2.2.3 Fluentd minimum footprint

2.2.4 Simple deployment of Ruby

2.2.5 Simple deployment of Fluentd

2.2.6 Deploying a log generator

2.2.7 Installing Postman

2.3 Bringing Fluentd to life with 'Hello World'

2.3.1 Hello World scenario

2.3.2 Hello World configuration

2.3.3 Starting Fluentd

2.4 Hello World with Fluent Bit

2.4.1 Starting Fluent Bit

2.4.2 Alternate Fluent Bit startup options

2.4.3 Fluent Bit configuration file comparison

2.4.4 Fluent Bit configuration file in detail

2.4.5 Putting the dummy plugin into action

2.5 Fluentd deployment with Kubernetes and Containers

2.5.1 Fluentd Daemonset

2.5.2 Dockerized Fluentd

2.6 Using Fluentd UI