chapter two

2 Concepts, architecture, and deployment of Fluentd

 

This chapter covers

  • Fluentd Architecture & Core Concepts
  • Pre-requisites and the deployment of Fluentd, Fluent Bit, and Fluent UI
  • Installation of the pre-requisites and then Fluentd, Fluent Bit, and Fluentd-UI,
  • Execution of the Fluentd and Fluent Bit versions of the classic first step in programming a ‘Hello World’ setup,
  • Introduction to the structure of the configuration files.

In Chapter 1, we looked at the theory, industry trends, and use cases that Fluentd can help us with. In this chapter, this is translated to how Fluentd works, and as a result, deploying and running the simplest of configurations to implement the traditional developer’s ‘hello world’.

2.1       Architecture and Core Concepts

Like driving a car, it is a lot easier, when you have some basic appreciation of how the car is powered (petrol, diesel, electric, Liquified Petroleum Gas (LPG)) as the mental models that come with such understanding mean we can better relate to what to expect – whether we can expect to hear the engine rev, can the engine stall, how do the gears work (if there are any).  For the same reason before we start working with Fluentd, and Fluent Bit. Let’s run through some of the building blocks of Fluentd that will help with the mental models.

2.1.1   The makeup of a Log Event

2.1.2   Handling time

2.1.3   Architecture of Fluentd

2.1.4   Fluent configuration execution order

2.1.5   Directives

2.1.6   Exercise – Questions about log events and time settings

2.2       Deployment of Fluentd

2.2.1   Deploying Fluentd for the book’s examples

2.2.2   Fluentd deployment considerations

2.2.3   Fluentd minimum footprint

2.2.4   Simple deployment of Ruby

2.2.5   Simple deployment of Fluentd

2.2.6   Deploying a log generator

2.2.7   Installing Postman

2.3       Bringing Fluentd to life with ‘Hello World’

2.3.1   Hello World scenario

2.3.2   Hello World configuration

2.3.3   Starting Fluentd

2.4       Hello World with Fluent Bit

2.4.1   Starting Fluent Bit

2.4.2   Alternate Fluent Bit startup options

2.4.3   Fluent Bit configuration file comparison

2.4.4   Fluent Bit configuration file in detail

2.4.5   Exercise – configuring the dummy plugin

2.5       Fluentd deployment with Kubernetes and Containers

2.5.1   Fluentd Daemonset

2.5.2   Dockerized Fluentd

2.6       Using Fluentd UI