Fluent Bit is an open-source logs and metrics processor and shipper that collects container events and forwards them through a cloud-native observability stack. In a Kubernetes microservice system, it runs close to the workloads so logs can be collected before reaching a searchable backend. section 9.0 section 9.3.8
A practical Kubernetes deployment uses Fluent Bit as follows:
- Install its Helm chart, which creates a DaemonSet and runs one collector pod on each Kubernetes node. section 9.3.8
- Read container log files under /var/log/containers and stream logs generated by pods. section 9.3.8
- Configure the outputs section with the Elasticsearch host, credentials, TLS settings, and log format. section 9.3.9
- Set Retry_Limit False when failed forwarding should be retried indefinitely, while recognizing that local TLS verification is disabled in the example. section 9.3.9
Forward Kubernetes logs to Elasticsearch
Install Fluent Bit with Helm, then upgrade it using a values file whose output matches kube.* and targets quickstart-es-http. Supply the Elasticsearch user and password, enable TLS, select Logstash formatting, and configure the deployment to forward application logs into the Elasticsearch and Kibana observability workflow. section 9.3.8 section 9.3.9
helm repo add fluent https://fluent.github.io/helm-charts helm repo update helm install fluent-bit fluent/fluent-bit
Use the supplied Helm commands to add the Fluent Bit repository, refresh chart metadata, and install the collector. section 9.3.8
config:
outputs: |
[OUTPUT]
Name es
Match kube.* #1
Host quickstart-es-http #2
HTTP_User elastic #3
HTTP_Password $PASSWORD #4
tls On
tls.verify Off #5
Logstash_Format On
Retry_Limit False
Inspect the output configuration showing Kubernetes log matching, Elasticsearch connection details, TLS behavior, Logstash formatting, and unlimited retries. section 9.3.9
