liveBook · Manning

Avro

As explained in Kafka for Architects

Avro is a data serialization system that represents structured records with schemas and a compact binary encoding.

When systems exchange customer events, they need an agreed structure that does not depend on either system’s programming language. Avro supplies a schema for that contract and a compact binary encoding for the data. The encoded data omits field names and relies on the schema to describe its structure. section 6.3.1

Listing 6.3 An example of the Avro format
{
  "type": "record",  #1
  "name": "Client",
  "namespace": "com.example",  #2
  "fields": [
    { "name": "clientId", "type": "string" },  #3
    { "name": "clientName", "type": "string" },
    { "name": "age", "type": "int" },
    { "name": "email", "type": ["null", "string"], "default": null },  #4
    { "name": "isPremium", "type": "boolean", "default": false },
    { "name": "address", 
      "type": {
        "type": "record",  #5
        "name": "Address",
        "fields": [
          { "name": "street", "type": "string" },
          { "name": "city", "type": "string" },
          { "name": "postalCode", "type": "string" }
        ]
      }
    },
    { "name": "createdAt", "type": "long", "logicalType": "timestamp-millis" }
  ]
}

The Client schema shows a record with a namespace, typed fields, a nullable email, a default value, a nested address, and a timestamp field. section 6.3.1

Applications can generate language-specific structures from an Avro schema or work dynamically with name–value pairs. Compared with a text-based JSON approach, Avro favors compact data and strong compatibility rules, but its binary messages are not human-readable. Its libraries and tools across popular languages also make it a practical choice for prototyping and troubleshooting. section 6.3.1 section 6.3.2

A data contract can be defined directly as an Avro schema, kept in Git, and referenced from an AsyncAPI specification. This avoids translating an abstract event model into a separate concrete schema. In Kafka deployments using Schema Registry, producers send a schema ID with the serialized data rather than sending the full schema in each message; consumers use that ID to retrieve the schema for deserialization. section 6.3.5 section 6.4.1

Kafka producers and consumers must be configured with matching Avro serializers or deserializers and a Schema Registry URL. In Kafka Connect, an AvroConverter instead translates between ConnectRecord objects and Avro-encoded Kafka data; the connector remains responsible for the external system’s native format. section 6.4.1 section 7.3.3

How Kafka for Architects covers it

How deep it goes
Explains schema components, compact encoding, generated versus dynamic access, and registry-based serialization without an end-to-end implementation walkthrough.
Examples
Uses a Client record schema, an externally referenced user-profile schema, and Kafka producer and consumer configurations.
What it assumes
Defines records, namespaces, fields, and optionality without requiring language-specific generated classes.
What you can do after
Apply it via Avro schemas and KafkaAvroSerializer to encode Kafka messages and maintain externally referenced event contracts.
How it connects
Connects Avro schemas to AsyncAPI data contracts and Kafka Schema Registry.

Also explained in

Compare how 3 books explain Avro →

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Kafka for Architects. The section links point to the parts of the book it draws on.