Fluentd
Fluentd is an open-source data collector that gathers, processes, transforms, and routes logs and other machine-generated event data through pluggable integrations.
Learn Docker in a Month of Lunches, Second Edition
Fluentd is an open-source unified logging layer that collects, processes, enriches, filters, and forwards log data.
- How it is used
- Collects container logs centrally so they can be enriched, filtered, and forwarded.
- How deep it goes
- Works through driver setup and tag-based routing, but leaves fuller Fluentd configuration practice to a lab.
- Examples
- Uses a timecheck container for forwarding, an image gallery for filtering, and a numbers app for practice.
- What you can do after
- Operate/configure it to route tagged container logs and enrich records without changing application images.
Read how Learn Docker in a Month of Lunches, Second Edition explains fluentd →
Logs and Telemetry
Fluentd is an open-source data collector that unifies the collection and routing of logs and other machine data through pluggable integrations.
- How it is used
- Connects diverse observability-data sources and targets through pluggable integrations and interoperates with Fluent Bit.
- How deep it goes
- Defines Fluentd’s ecosystem role, deployment fit, interoperability, and basic configuration directives without detailing plugin implementation.
- Examples
- Demonstrates a Dummy source routed through `<match *>` to stdout and compares its configuration with Fluent Bit.
- What you can do after
- Apply it via Fluentd directives to define an input and route all matching events to stdout.
Kafka for Architects
Fluentd is an open-source data collector that gathers, transforms, and routes event and log data.
- How it is used
- Collects application logs from files and forwards them to Kafka within centralized logging architectures.
- How deep it goes
- Traces the collection path through Fluentd, Kafka, Kafka Connect, and Elasticsearch without detailing collector internals.
- Examples
- Uses file-based application logs, Fluentd agents, Kafka topics, Kafka Connect, and Elasticsearch as the concrete pipeline.
- What you can do after
- Operate/configure a decoupled log pipeline that forwards application events through Kafka into Elasticsearch.
Cloud Observability in Action
Fluentd is an open-source data collector that unifies the collection and forwarding of logs and other machine data.
- How it is used
- Routes containerized application logs toward centralized search and visualization backends.
- How deep it goes
- Defines Fluentd, contrasts its footprint and plug-ins with Fluent Bit, and demonstrates a pipeline without detailing Fluentd configuration internals.
- Examples
- Uses a ho11y container, tagged records, Fluent Bit routing, and OpenSearch querying in a Docker Compose pipeline.
- What you can do after
- Operate/configure a containerized log pipeline that forwards tagged application records to a searchable backend.
Related concepts
- Elasticsearchuses · Elasticsearch is a distributed search and analytics engine that indexes data for searching, retrieval, and analysis.
- EFK stackused by · The EFK stack is a logging architecture composed of Elasticsearch for storage and search, Fluentd for collection and processing, and Kibana for exploration and visualization.
- FluentBitcontrasts with · Fluent Bit is a lightweight, open-source telemetry agent for collecting, processing, enriching, and forwarding logs and other event data.
- Apache Kafkauses · Apache Kafka is a distributed event-streaming platform for publishing, storing, processing, and consuming streams of records.
- forward input pluginused by · A forward input plugin is a Fluent Bit input component that receives events or log records encoded with the Fluentd Forward protocol.
- Fluentd logs driverused by · The Fluentd logging driver is a Docker logging integration that forwards container log events to a Fluentd service using the Fluent Forward protocol.
These guides are generated from the text of Manning books by Manning's knowledge graph and link to the sections they draw on. The full books are available on liveBook and at manning.com.