Protobuf
Protocol Buffers is a language- and platform-neutral system for defining structured data schemas and serializing messages in a compact binary format, with support for generated code.
Kafka Streams in Action, Second Edition
Protocol Buffers is a language-neutral, platform-neutral mechanism for serializing structured data using defined message schemas.
- How it is used
- Serializes Kafka event records through schemas, generated objects, Schema Registry serializers, and ksqlDB formats.
- How deep it goes
- Explains proto3 messages, numbered fields, defaults, imports, generated types, and Kafka configuration; defers full specification details.
- Examples
- Uses Avenger messages, imported Person and Company schemas, multi-event wrappers, and user-activity ksqlDB tables.
- What you can do after
- Apply it via Kafka Protobuf serializers, generated Java types, Schema Registry, imports, and ksqlDB format settings.
Read how Kafka Streams in Action, Second Edition explains protobuf →
Logs and Telemetry
Protocol Buffers is a language-neutral, platform-neutral mechanism for serializing structured data using generated code and compact binary messages.
- How it is used
- Carries OTLP telemetry payloads compactly when gRPC communication uses HTTP/2.
- How deep it goes
- Explains Protobuf3 through transport choices, fallback behavior, and compatibility limits rather than payload-schema internals.
- Examples
- Uses Fluent Bit OpenTelemetry listeners for logs, traces, and metrics, with JSON-over-HTTP/1.1 requests for inspectable testing.
- What you can do after
- Operate/configure OTLP endpoints by selecting gRPC-Protobuf or HTTP/JSON according to receiver capabilities.
Learn Go with Pocket-Sized Projects
Protobuf is a language-neutral, platform-neutral mechanism for serializing structured data using defined message schemas.
- How it is used
- Defines structured messages and service endpoints for a habits-tracking gRPC API.
- How deep it goes
- Explains message fields, stable identifiers, service declarations, code generation, and manual testing, while focusing on the basics rather than a complete Protobuf reference.
- Examples
- Uses Habit, CreateHabit request and response messages, and a grpcurl command to demonstrate a tracking service.
- What you can do after
- Apply it via protoc and grpcurl to define a gRPC API, generate language-specific structures, and test an endpoint.
Read how Learn Go with Pocket-Sized Projects explains protobuf →
Designing AI Systems
Protocol Buffers is a language-neutral, platform-neutral serialization format for structured data and service interface definitions.
- How it is used
- Supports efficient service-to-service communication and explicit contracts behind a function-like SDK.
- How deep it goes
- Traces schemas, generated stubs, message conversion, and gateway forwarding without detailing binary encoding internals.
- Examples
- Uses Session Service methods and a model chat request with nested tool definitions to illustrate contracts.
- What you can do after
- Apply it via gRPC generated stubs to translate Python objects into service requests and responses.
gRPC Microservices in Go
Protocol Buffers is a language- and platform-neutral mechanism for defining and serializing structured messages and service interfaces.
- How it is used
- Defines and serializes messages exchanged between microservices over gRPC.
- How deep it goes
- Works through .proto definitions, stub generation, and basic binary encoding, but does not detail compatibility rules here.
- Examples
- Uses Order-to-Payment requests and responses, generated Go structs, and an integer-field encoding example.
- What you can do after
- Apply it via protoc to generate Go stubs for a defined gRPC service interface.
Akka in Action, Second Edition
Protocol Buffers is a language-neutral schema format and serialization mechanism for structured data with generated language bindings.
- How it is used
- Defines portable data and RPC contracts for Akka gRPC services and serializes events or clustered messages.
- How deep it goes
- Explains .proto messages, scalar mappings, service RPCs, compilation, and generated Scala artifacts without detailing schema-evolution procedures.
- Examples
- Uses a Cargo/AddCargo container service and MarketClosed, MarketOpened, and MarketCancelled Kafka event messages.
- What you can do after
- Build it from scratch by defining .proto messages and services, then compiling generated Scala classes and gRPC interfaces.
Building Web APIs with ASP.NET Core
Protocol Buffers is a language-neutral serialization format and interface-definition system for structured messages and service contracts.
- How it is used
- Defines gRPC message formats and service contracts for a board-game API.
- How deep it goes
- Defines proto3 services and messages, then defers detailed proto3 syntax.
- Examples
- Uses grpc.proto with GetBoardGame, BoardGameRequest, and BoardGameResponse.
- What you can do after
- Build it from scratch by defining a proto3 contract and wiring generated gRPC server types into ASP.NET Core.
Read how Building Web APIs with ASP.NET Core explains protobuf →
Microservices Patterns, Second Edition
Protocol Buffers is a language-neutral, platform-neutral mechanism for serializing structured data using compact binary encoding and numeric field tags.
- How it is used
- Carries structured messages in microservice communication when compactness and efficient processing matter.
- How deep it goes
- Explains typed IDL, generated code, field tags, and unknown-tag skipping, but not the binary wire encoding.
- Examples
- Uses an Order Service API with CreateOrderRequest fields, numeric tags, and generated gRPC client and server code.
- What you can do after
- Apply it via the Protocol Buffer compiler to generate typed messages and client/server code for communication.
Read how Microservices Patterns, Second Edition explains protobuf →
Quarkus in Action
Protocol Buffers is a language-neutral serialization format that represents structured data using schemas and generated types.
- How it is used
- Defines compact typed messages and gRPC contracts for cloud-oriented service communication.
- How deep it goes
- Explains proto3 schemas, field numbering, generated Java builders, serialization, and gRPC stubs without detailing wire encoding.
- Examples
- Uses Car messages, InventoryService RPC methods, generated builders, and an inventory.proto contract.
- What you can do after
- Build typed messages and gRPC service contracts, then generate and use Java serialization code and communication stubs.
Go in Practice, Second Edition
- How it is used
- Structures strongly typed, cross-language messages and RPC contracts for microservices and HTTP API communication.
- How deep it goes
- Explains schemas, generated bindings, commands, servers, clients, and trade-offs, but does not detail binary encoding internals.
- Examples
- Uses a proto3 chat schema, protoc generation, a Python gRPC server, and a Go client calling RouteComments.
- What you can do after
- Build cross-language gRPC messages and services, generate bindings, run a server, and invoke it from a client.
Cloud Observability in Action
Protocol Buffers is a language-neutral, platform-neutral mechanism for serializing structured data using schema-defined messages.
- How it is used
- Structures profiling data for pprof tools and supports compact storage, transfer, decoding, and analysis.
- How deep it goes
- Explains schema-based representation, efficiency, readability trade-offs, and decoding, but does not detail protobuf serialization internals.
- Examples
- Uses the pprof Profile schema and a protoc command that finds main.slowTask in decoded profile data.
- What you can do after
- Diagnose protobuf-encoded profiling data with protoc and select supporting tools for inspection or API work.
Latency
Protocol Buffers is a cross-platform binary serialization format commonly used for encoding messages in remote procedure calls.
- How it is used
- Uses binary message serialization at remote procedure call boundaries and in latency-sensitive application workflows.
- How deep it goes
- Explains serialization costs, memory representation, fields, precision, and byte order, but stops short of protocol-buffer encoding internals.
- Examples
- Uses an online-store order with order ID and quantity, represented as a Rust struct and serialized for JSON, RPC, or storage.
- What you can do after
- Evaluate serialization work as a source of latency, CPU consumption, and memory-copy overhead across application boundaries.
Also covered in
- Troubleshooting Java, Second Edition
- Kafka for Architects
- LLMs in Production
- Domain-Specific Small Language Models
- The Design of Web APIs, Second Edition
- Apache Kafka in Action
What to know first
- JSONJSON is a language-independent text-based data-interchange format that represents structured values using objects, arrays, strings, numbers, booleans, and null.
- http/2HTTP/2 is a version of the Hypertext Transfer Protocol that uses binary framing to multiplex request and response streams over a connection.
- serializationSerialization is the conversion of structured data or in-memory values into a representation suitable for storage or transmission and possible later reconstruction.
- gRPCgRPC is an open-source remote procedure call framework for defining services and invoking them across process or network boundaries, commonly using Protocol Buffers and HTTP/2.
Related concepts
- gRPCused by · gRPC is an open-source remote procedure call framework for defining services and invoking them across process or network boundaries, commonly using Protocol Buffers and HTTP/2.
- JSONalternative to · JSON is a language-independent text-based data-interchange format that represents structured values using objects, arrays, strings, numbers, booleans, and null.
- Akka gRPCused by · Akka gRPC is an Akka library that generates and supports gRPC clients and servers for Scala and Java applications.
- binary formatkind of · A binary format is a defined representation that encodes information as machine-readable bits or bytes rather than as human-readable text.
- interface definition languagekind of · An interface definition language is an implementation-independent language for specifying software interfaces, data structures, operations, and communication contracts.
- oneOfhas part · oneof is a Protocol Buffers field construct that allows at most one member of a set of alternative fields to be set in a message.
Questions about protobuf
What is protobuf?
Protocol Buffers is a language- and platform-neutral system for defining structured data schemas and serializing messages in a compact binary format, with support for generated code.
Which Manning books explain protobuf?
Kafka Streams in Action, Second Edition, Logs and Telemetry, Learn Go with Pocket-Sized Projects, Designing AI Systems, gRPC Microservices in Go, Akka in Action, Second Edition, Building Web APIs with ASP.NET Core, Microservices Patterns, Second Edition and 4 more.
What should I know before learning protobuf?
Manning books that teach protobuf usually introduce JSON, http/2, serialization and gRPC first.
What is the difference between protobuf and JSON?
Protocol Buffers is a language- and platform-neutral system for defining structured data schemas and serializing messages in a compact binary format, with support for generated code. JSON is a language-independent text-based data-interchange format that represents structured values using objects, arrays, strings, numbers, booleans, and null.
What is protobuf a kind of?
Protobuf is a kind of binary format. A binary format is a defined representation that encodes information as machine-readable bits or bytes rather than as human-readable text.
What are the parts of protobuf?
Protobuf includes oneOf.
These guides are generated from the text of Manning books by Manning's knowledge graph and link to the sections they draw on. The full books are available on liveBook and at manning.com.











