liveBook · Manning

PostgreSQL plugin

As explained in Logs and Telemetry

The PostgreSQL plugin is a Fluent Bit output plugin that writes event records to PostgreSQL-compatible databases.

The PostgreSQL plugin, named `pgsql`, is a Fluent Bit output that stores log events in a Postgres table. It provides external, queryable storage for events after Fluent Bit receives and routes them, making Postgres a practical destination when relational and nonrelational data models are useful. section 5.7 section 5.12

A basic output configuration supplies database authentication with `user` and `password`, network location with `host` and `port`, and the target `database` and `table`. `timestamp_key` selects the event field used as the stored timestamp. Credentials in the main configuration work for a simple test but are not ideal for operational deployments. section 5.7

Listing 5.9 PostgreSQL output example: db-output.conf
[SERVICE]
  flush 1
  log_level info

[INPUT]
  name http    
  listen 0.0.0.0
  port 9881
  tag input1

[OUTPUT]
  name  pgsql       #1
  match  *
  host  localhost      #2
  port  5455
  user  postgresUser      #3
  password  postgresPW
  database  postgres      #4
  table   fluentbit
  timestamp_key ts        #5

Use this configuration as a minimal flow: Fluent Bit accepts HTTP events tagged `input1`, matches them with `*`, and writes them to the `fluentbit` table. Replace the localhost address, port, credentials, database, and table values for the deployed database. section 5.7

Check these deployment details before enabling the output:

  • Confirm that `pgsql` is available with `fluent-bit --help`. The plugin is controlled by build flags and may be absent from a custom build. section 5.7
  • Expect the official Linux binary to include the plugin by default; official Windows and macOS downloads do not include it by default. section 5.7
  • Use a DNS address rather than `localhost` for typical containerized or Kubernetes deployments. Localhost is appropriate for a sidecar deployment, but not when Fluent Bit and Postgres are separate containers. section 5.7
  • Prepare PostgreSQL before running the pipeline; setup scripts can simplify starting a local database with Docker. section 5.7

HTTP events to a Postgres table

Run Fluent Bit with the PostgreSQL configuration after starting the database. Events received through the HTTP input are matched by the output and inserted into the configured table, where they can be viewed and queried through PostgreSQL tooling such as pgAdmin. section 5.7

How Logs and Telemetry covers it

How deep it goes
Explains a runnable HTTP-to-Postgres configuration, connection attributes, build availability, and container networking choices.
Examples
Uses an HTTP listener on port 9881 that writes matched events to a `fluentbit` Postgres table.
What it assumes
Assumes a prepared PostgreSQL instance and introduces a simple Fluent Bit HTTP input-to-output pipeline.
What you can do after
Operate/configure it by verifying `pgsql`, setting database connection fields, and directing matched events to a table.
How it connects
Connects PostgreSQL output with Fluent Bit, HTTP input, Docker, Kubernetes, DNS, and pgAdmin.

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Logs and Telemetry. The section links point to the parts of the book it draws on.