liveBook · Manning

PostgreSQL

As explained in Logs and Telemetry

PostgreSQL is an open-source object-relational database management system.

PostgreSQL can serve as a general-purpose destination for log events, metrics, and tracers. Its native JSON support accommodates varying log structures without requiring extensive payload restructuring, while its relational and nonrelational models make it a useful converged database for observability data. section 5.7

Fluent Bit writes events to PostgreSQL through the pgsql output plugin. A basic pipeline accepts HTTP events, matches them to the output, authenticates with a database user and password, and sends them to a selected database table over a host and port. section 5.7

Configure the PostgreSQL output with these operational choices:

  • Set name to pgsql and match to the tags that should be stored. section 5.7
  • Provide host and port for the PostgreSQL server; localhost is suitable for a simple local scenario. section 5.7
  • Provide user and password credentials, while recognizing that placing credentials directly in core configuration is not ideal. section 5.7
  • Select the target database and table, and optionally name the timestamp field with timestamp_key. section 5.7
  • Confirm that pgsql is available with fluent-bit --help before relying on the output. section 5.7
Listing 5.9 PostgreSQL output example: db-output.conf
[SERVICE]
  flush 1
  log_level info

[INPUT]
  name http    
  listen 0.0.0.0
  port 9881
  tag input1

[OUTPUT]
  name  pgsql       #1
  match  *
  host  localhost      #2
  port  5455
  user  postgresUser      #3
  password  postgresPW
  database  postgres      #4
  table   fluentbit
  timestamp_key ts        #5

Use this complete configuration to send HTTP input events into a PostgreSQL table. It shows the service, input, pgsql output, connection credentials, target table, and timestamp key together. section 5.7

Plugin availability depends on build flags. The official Linux binary includes pgsql by default, whereas Windows and macOS downloads do not; self-compiled deployments need the plugin explicitly enabled. PostgreSQL can also be used when developing a database-output plugin and testing a Postgres driver. section 5.7 section 10.6

How Logs and Telemetry covers it

How deep it goes
Explains pgsql connection and table settings through a complete configuration, while deferring PostgreSQL setup details.
Examples
Demonstrates HTTP events routed into a fluentbit PostgreSQL table and viewed through pgAdmin.
What it assumes
Assumes a running PostgreSQL instance and an available pgsql plugin, then begins with a simple HTTP input pipeline.
What you can do after
Operate/configure it as a Fluent Bit pgsql output by setting routing, connection, credentials, database, table, and timestamp options.
How it connects
Connects PostgreSQL with Fluent Bit, JSON, HTTP input, Elasticsearch, OpenSearch, Kafka, Prometheus, and plugin development.

Also explained in

Compare how 11 books explain PostgreSQL →

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Logs and Telemetry. The section links point to the parts of the book it draws on.