Kubernetes filter plugin

The Kubernetes filter enriches Fluent Bit log events with Kubernetes context when raw container output alone cannot distinguish Pod instances after cutovers or scaling. It is suited to Fluent Bit running as a DaemonSet, or another deployment that receives tags in the form pod name, namespace, container name, and Docker ID. section 4.6.3

The filter relies on tag expansion from inputs such as tail and systemd. Expanded tags carry file-path-derived details, including Pod and container identifiers; the filter parses those details and queries Kubernetes APIs for additional metadata. A trailing asterisk on an input tag expands it with the full path, represented with dots instead of path slashes. section 4.6.4

[INPUT]
  name tail
  path ./chapter3/basic-file.txt
  read_from_head true
  tag basic-file.*    #1

Use a trailing wildcard on a tail input tag to expose the file path as part of the event tag. This provides the tag information the Kubernetes filter can parse when container log paths encode Kubernetes attributes. section 4.6.4

In a DaemonSet deployment, one Fluent Bit instance runs per node. With suitable mounts, it can read stdout and stderr logs routed by Kubernetes. The filter can then match Kubernetes-tagged records, use the in-cluster API URL plus service-account CA and token files, cache metadata, and include labels or annotations. It can also merge processed log content under a separate key. section 4.6.4

[FILTER]
  name kubernetes
  match kube.*  #1
  kube_URL https://kubernetes.default.svc:443     #2
  kube_CA_file 
 /var/run/secrets/kubernetes.io/serviceaccount/ca.crt #3
  kube_token_file 
 /var/run/secrets/kubernetes.io/serviceaccount/token   #3
 kube_meta_cache_TTL 60s    #4
 cache_use_docker_id on
 labels on         #5
 annotations on    #5
 use_journal off
 kube_tag_prefix kube.var.log.containers.  #6
 merge_log on
 merge_log_key log_processed
 merge_log_trim on             #7

This filter fragment shows the operational choices: match Kubernetes tags, authenticate to the in-cluster API, set metadata cache lifetime, enable labels and annotations, parse the Kubernetes tag prefix, and merge log content. section 4.6.4

Choose the enrichment path that fits the log source:

• Use the Kubernetes filter when tags or container log paths contain the Pod and container details needed for Kubernetes API lookup. section 4.6.4
• Inject context from the log source when hostname configuration, container scripts, or downward-API environment variables can supply the needed values before Fluent Bit receives events. section 4.6.3
• Automate deployment with Fluent Operator or Helm charts where Kubernetes node monitoring is required, while accounting for cloud-provider differences in visible managed-service infrastructure. section 11.3.2