chapter two

2 Concepts, architecture and deployment of Fluentd

 

This chapter covers

  • Fluentd Architecture & Core Concepts
  • Pre-requisites and the deployment of Fluentd, Fluent Bit and Fluent UI
  • Installation of the pre-requisites and then Fluentd, Fluent Bit, and Fluentd-UI,
  • Execution of the Fluentd and Fluent Bit versions of the classic first step in programming a ‘Hello World’ setup,
  • Introduction to the structure of the configuration files.

In Chapter 1, we looked at the theory, industry trends, and use cases that Fluentd can help us with. In this chapter, this is translated how Fluentd actually works, and as a result, deploying and running the simplest of use cases.

2.1           Architecture and Core Concepts

Like driving a car, it is a lot easier, when you have some basic appreciation of how the car is powered (petrol, diesel, electric, Liquified Petroleum Gas (LPG)) as the mental models that come with such understanding mean we can better relate to what to expect – whether we can expect to hear the engine rev, can the engine stall, how do the gears work (if there are any). For the same reason before we start working with Fluentd, and Fluent Bit. Let’s run through some of the ideas that will help with the mental models.

2.1.1   The makeup of a Log Event

One of the most important things to understand is how Fluentd handles a log event. Each log event is managed as a single JSON object made up of three elements as shown in Figure 2.1.

Figure 2.1 make up of Log Event

2.1.2   Handling time

2.1.3   Architecture of Fluentd

2.1.4   Fluent configuration execution order

2.1.5   Exercise 2.1

2.2           Deployment of Fluentd

2.2.1   Deployment considerations

2.2.2   Minimum footprint

2.2.3   Simple deployment of Fluentd

2.2.4   Deploying a log generator

2.2.5   Installing Postman

2.3           Bringing Fluentd to life with ‘Hello World’

2.3.1   Hello World scenario

2.3.2   Hello World configuration

2.3.3   Starting Fluentd

2.4           Hello World with Fluent Bit

2.4.1   Starting Fluent Bit

2.4.2   Alternate Fluent Bit startup options

2.4.3   Fluent Bit configuration file comparison

2.4.4   Fluent Bit configuration file in detail

2.4.5   Exercise 2.2

2.5           Fluentd deployment with Kubernetes and Containers