liveBook · Manning

Configuration

As explained in Logs and Telemetry

A configuration is a set of declarative settings that controls Fluent Bit's behavior and processing pipeline.

A Fluent Bit configuration is a declarative description of how the service runs and how events move through its pipeline. It can be supplied on the command line for simple cases or kept in files for the maintainable configurations used beyond basic input-to-output flows. Configuration choices also shape deployment: a container can receive files through mounted volumes, a startup pull, or—in Kubernetes—a ConfigMap. section 2.1 section 2.1.1

File-based configuration comes in classic and YAML forms. Classic syntax remains compatible and widely used, while YAML aligns with cloud-native and Kubernetes tooling, supports YAML templating, and is where newer capabilities may appear first. YAML was introduced in Fluent Bit 1.9 and is considered production fit from 2.0; classic notation is retained for backward compatibility. section 2.2

A configuration can be organized and parameterized without changing its operational intent:

  • Split a growing configuration into root-level files with @include. Inclusion is text substitution, so declaration order matters—for example, variables must be defined before use—and indentation has constraints. section 2.4
  • Set shared values with @set, including values consumed by included files. This provides platform-agnostic variable setup, but a matching @set value overrides the operating-system environment variable. section 2.5.2
  • Use the CLI when a minimal, direct setup is sufficient; move to files as inputs, filters, outputs, and deployment concerns make the command line hard to manage. section 2.1.1

Hello World assembled from includes

A top-level file can contain @include hello-world-input.conf and @include hello-world-output.conf. Running fluent-bit -c hello-world-includes.conf produces the same result as the single-file Hello World configuration, while separating source and destination definitions for easier maintenance. section 2.4

Configuration expresses operational behavior as well as pipeline wiring. An HTTP input can select its listening address and port, attach a tag, return an explicit successful HTTP status, and repeat success_header settings to identify the receiving node or consumer. In containerized environments, configuration management must account for orchestration and for preserving logs quickly enough to investigate a terminated container or node. section 3.6.2 section 4.0

Table 2.4 Classic Fluent Bit vs. Fluentd configuration comparison
Classic Fluent Bit
Fluentd
[SERVICE]
    flush      1
    daemon     Off
    log_level  info

# define the Dummy source 
[INPUT]
    name  Dummy
    dummy {"hello": "my world"}


# Accept all log events regardless of tag and write 
# them to the console
[OUTPUT]
    name  stdout
    match *
<system>
    log_level info
</system>

# define the Dummy source
<source>
    @type Dummy
    dummy {"hello": "my world"}
    
</source> # after a directive
# Accept all log events regardless of tag and write them to the console
<match *>
   @type stdout
</match>

Compare classic Fluent Bit directives with Fluentd equivalents when planning migration. The side-by-side example shows SERVICE, INPUT, and OUTPUT blocks alongside system, source, and match forms, including the syntax difference between square-bracket directives and XML-style tags. section 2.2.1

A complete configuration may also identify custom plugins and give each input or output instance its own connection, routing, and behavior settings. For example, a database pipeline configures a custom input and output, tags events, selects MySQL or Postgres-compatible database details, and uses matching to connect the flow. Network addressing may need adjustment for the Docker host environment. section 10.8

How Logs and Telemetry covers it

How deep it goes
Explains formats, includes, variables, HTTP settings, and custom-plugin pipelines, while deferring exhaustive edge cases and full configurations to referenced files.
Examples
Demonstrates Hello World includes, an HTTP input tested with Postman, and database-backed custom input and output plugins.
What it assumes
Assumes Fluent Bit and a configuration-file editor; introduces configuration approaches from a minimal Hello World setup.
What you can do after
Operate/configure Fluent Bit with file formats, composition directives, runtime variables, and pipeline-specific settings.
How it connects
Connects CLI, YAML, ConfigMap, Fluentd, environment variables, HTTP input, Kubernetes, and custom plugins.

Also explained in

Compare how 23 books explain configuration →

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Logs and Telemetry. The section links point to the parts of the book it draws on.