liveBook · Manning

Centralized logging system

As explained in Learn Docker in a Month of Lunches, Second Edition

Centralized logging is a logging architecture in which logs from multiple applications or containers are collected and managed in a shared system.

A centralized logging system collects logs from multiple containers, forwards them to a shared store, and makes them searchable from one interface. A practical Docker setup uses Fluentd for collection, Elasticsearch for storage, and Kibana for search. section 19.3

A Docker-based setup follows this workflow:

  • Start Fluentd and the Elasticsearch-Kibana stack with Docker Compose. section 19.3
  • Opt containers into centralized logging with Docker’s Fluentd logging driver. section 19.3
  • Create a Kibana data view using the fluentd* index pattern and @timestamp field. section 19.3
  • Search and filter structured log documents containing fields such as container name, container ID, and timestamp. section 19.3

Run a container with Fluentd logging

Start the logging stack, then run a timecheck container with --log-driver=fluentd. Open Kibana at localhost:5601/app/discover and search the collected logs. section 19.3

cd ch19/exercises/fluentd
# run Fluentd publishing the standard port and using a config file:
docker run -d -p 24224:24224 `
 --name fluentd -v "$(pwd)/conf:/fluentd/etc" `
 -e FLUENTD_CONF=stdout.conf `
 diamol/fluentd:2e
# now run a timecheck container set to use Docker's Fluentd log driver:
docker run -d --log-driver=fluentd `
 --name timecheck5 `
 diamol/ch19-timecheck:2e-v3
# check the timecheck container logs:
docker logs timecheck5
# and check the Fluentd container logs:
docker logs --tail 1 fluentd

Use the runnable commands to start Fluentd, configure its container, and verify logs with docker logs. section 19.3

How Learn Docker in a Month of Lunches, Second Edition covers it

How deep it goes
Explains the collection pipeline, Compose deployment, driver configuration, Kibana indexing, and structured search.
Examples
Uses timecheck and image-gallery containers, Fluentd commands, Compose files, Kibana data views, and searchable Elasticsearch indexes.
What you can do after
Operate and configure a Docker logging stack that forwards container output to Elasticsearch for Kibana-based searching.
How it connects
Connects Docker logging drivers with Fluentd, Elasticsearch, Kibana, Docker Compose, and structured log documents.

Also explained in

Compare how 3 books explain centralized logging system →

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Learn Docker in a Month of Lunches, Second Edition. The section links point to the parts of the book it draws on.