liveBook · Manning

Prometheus remote write API

As explained in Logs and Telemetry

A remote write endpoint is a network address that accepts metrics transmitted according to the Prometheus remote write specification.

A Prometheus remote write endpoint accepts metrics through the standardized remote write specification. It enables push-based delivery from Fluent Bit or another producer to Prometheus, Thanos, or compatible cloud services instead of relying only on polling. section 5.6.4

Configure the destination by choosing:

  • A remote write plugin and matching rule for the metrics stream. section 5.6.4

Local and cloud destinations

A Fluent Bit configuration can send internal metrics to Grafana Cloud over HTTPS at /api/prom/push, or to a local Prometheus instance at /api/v1/write on port 9090. Disable TLS locally and enable it for the cloud endpoint.

Listing 5.8 Prometheus writer: prometheus-remote-output.conf
[SERVICE]
  flush 5
  log_level trace

[INPUT]
    name fluentbit_metrics
    tag internal_metrics
    scrape_interval 5

[OUTPUT]
    name prometheus_remote_write    #1
    match *
    host prometheus-prod-05-gb-south-0.grafana.net    #2
    uri /api/prom/push   #3
    port 443 
    tls on
    log_response_payload on
    add_label fluentbit cloud
    http_user ${my-userid}           #4
    http_passwd ${my-password}


[OUTPUT]
    name prometheus_remote_write   
    match *
    host 192.168.0.205  #5
    uri  /api/v1/write     #6
    port 9090     #7
    tls off  
    log_response_payload on    #8
    add_label  fluentbit local       #9

Inspect the complete output configuration and the command used to run Fluent Bit. section 5.6.4

How Logs and Telemetry covers it

How deep it goes
Configures plugins, endpoints, transport security, credentials, and labels, but does not detail the wire protocol internals.
Examples
Demonstrates local Prometheus delivery, Grafana Cloud delivery, HTTPS, authentication, and label enrichment.
What it assumes
Assumes a Fluent Bit metrics source and an available Prometheus-compatible destination.
What you can do after
Operate/configure Fluent Bit to push metrics to local or hosted Prometheus-compatible endpoints.
How it connects
Connects Prometheus remote write with Prometheus, Thanos, Fluent Bit, Grafana Cloud, and time series data stores.

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Logs and Telemetry. The section links point to the parts of the book it draws on.