chapter eight

8 Driving logs with Docker & Kubernetes

 

This chapter covers

  • Configuring and using Docker to use Fluentd as its log driver.
  • Exploring how the Docker and Kubernetes configurations fit together.
  • Understanding the components involved in Kubernetes logging.
  • Navigating around Kubernetes nodes to see the challenges faced in setting up Fluentd logging.
  • Tailoring the prebuilt Kubernetes DeamonSets to your needs.
  • Configuring Fluentd to collect Kubernetes component log events.
  • Discovering how Kubernetes node monitoring can be integrated into the broader log capture considerations.
  • Executing the containerized LogGenerator and seeing how Fluentd picks up the events and forwards them.

Previous chapters have referred to Fluentd's relationship with Docker and Kubernetes, but we have focused on running Fluentd independently of these technologies to minimize complexity. This has helped underpin the point that despite the association with CNCF, Fluentd certainly is not restricted to cloud-native use cases.

8.1  Fluentd out of the box from Docker Hub

8.1.1  Official docker images

8.1.2  Docker log drivers

8.1.3  Getting setup for Docker Log Drivers

8.2  Using Docker Log Drivers

8.2.1  Docker Drivers via the command line

8.2.2  A quick check of network connections

8.2.3  Running Docker Command Line

8.2.4  Switching to driver configuration through a configuration file

8.3  Kubernetes components logging and the use of Fluentd

8.3.1  Kubernetes components and structured logging

8.3.2  Kubernetes default log retention and log rotation

8.3.3  kubectl with logging

8.4  Demonstrating logging with Kubernetes

8.4.1  Kubernetes setup

8.4.2  Creating logs to capture

8.4.3  Understanding how Fluentd DaemonSets are put together

8.5  Getting a peek at host logs

8.6  Configuring a Kubernetes logging DaemonSet

8.6.1  Putting the implementation of a Fluentd for Kubernetes into action

8.6.2  Getting the Fluentd configuration ready to be used

8.6.3  Creating our Kubernetes deployment configuration

8.6.4  Deploying to Minikube

8.6.5  Tidying up

8.7  Kubernetes configuration in action

8.7.1  Solution for production-ready configuration in action

8.8  More Kubernetes monitoring & logging to watch for

8.8.1  Node Monitoring

8.8.2  Termination Messages

8.9  Summary