liveBook · Manning

Protocol buffer

As explained in Logs and Telemetry

Protocol Buffers is a language-neutral, platform-neutral mechanism for serializing structured data using generated code and compact binary messages.

Protocol Buffers (Protobuf) encode structured telemetry payloads as compact binary data. Within OpenTelemetry Protocol (OTLP) traffic, they are used with gRPC over HTTP/2, while JSON over HTTP provides a more broadly compatible fallback. OTLP defines the telemetry data structures, encoding, and transmission rules that make these choices interoperable. section 1.3.2 section 3.6.7

The transport choice has operational consequences. gRPC requires HTTP/2 and normally carries Protobuf3 payloads. If a recipient cannot handle HTTP/2, a sender is expected to fall back to HTTP/1.1; compressed Protobuf3 over HTTP/1.1 is permitted, and JSON-formatted messages over HTTP/1.1 are also supported. Compatibility is not guaranteed because OpenTelemetry implementations may support different standard mechanisms. section 3.6.7

Choose an encoding and transport according to the receiving endpoint:

  • Use gRPC over HTTP/2 when the endpoint supports it; this is OTLP’s preferred communication approach and uses Protobuf payloads. section 1.3.2
  • Use Protobuf3 over HTTP/1.1 where supported when HTTP/2 is unavailable. section 3.6.7
  • Use JSON over HTTP/1.1 as the basic, widely handled option when compatibility and payload inspection matter more than compact binary encoding. section 3.6.7

Receiving OpenTelemetry signals

An OpenTelemetry input can expose separate local listeners for logs, traces, and metrics, then send matched records to standard output. Using JSON payloads over HTTP/1.1 makes the request bodies easier to inspect and works with Fluent Bit v2 endpoints; the same OTLP environment can also receive Protobuf3-formatted traffic where the endpoint and transport support it. section 3.6.7

Figure 1.3 Fluent Bit’s relationship with OpenTelemetry with apps generating OTel logs, metrics, and traces and Fluent Bit facilitating their transmission to an OTel-compliant point of aggregation or processing. Applications can send OTLP data directly or via an OTel component, and we can route data to other OTel services or analysis tools.
figure

This data-flow view places Protobuf-backed OTLP traffic among applications, Fluent Bit, OpenTelemetry collectors, and downstream analysis services. section 1.3.2

How Logs and Telemetry covers it

How deep it goes
Explains Protobuf3 through transport choices, fallback behavior, and compatibility limits rather than payload-schema internals.
Examples
Uses Fluent Bit OpenTelemetry listeners for logs, traces, and metrics, with JSON-over-HTTP/1.1 requests for inspectable testing.
What you can do after
Operate/configure OTLP endpoints by selecting gRPC-Protobuf or HTTP/JSON according to receiver capabilities.
How it connects
Connects Protocol Buffers with OpenTelemetry Protocol, gRPC, HTTP/2, HTTP/1.1, JSON, and Fluent Bit.

Also explained in

Compare how 10 books explain protocol buffer →

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Logs and Telemetry. The section links point to the parts of the book it draws on.