liveBook · Manning

Schema id

As explained in Kafka for Architects

A schema identifier is a unique value associated with a registered schema and embedded or referenced in serialized data.

A schema ID is a unique identifier assigned when a schema is registered with Schema Registry. Kafka messages carry the ID with serialized data rather than carrying the complete schema, reducing duplication while giving consumers a consistent lookup mechanism. section 6.4.1 section 6.1

The producer validates data against a schema, serializes it, and embeds the schema ID in the message. The consumer extracts the ID, retrieves the corresponding schema from Schema Registry, or uses a cached copy, and deserializes the data. Brokers treat messages as raw bytes and do not validate either the ID or the payload. section 6.4.1 section 6.1

Looking up an identifier

A registry request such as `curl http://schema-registry:8081/schemas/ids/1` retrieves the schema associated with ID 1. A consumer can then use that schema to interpret the serialized message. section 6.4.1

Schema IDs make centralized schema management consistent across producers and consumers, but they are not self-describing business metadata: consumers still need prior knowledge of the message structure. IDs can also differ between development and production, complicating cross-cluster migration and sometimes requiring byte-level identifier replacement. section 6.1 section 6.4.6

How Kafka for Architects covers it

How deep it goes
Explains registration, embedding, caching, lookup, validation boundaries, and cross-environment migration risks.
Examples
Uses a REST lookup for ID 1 and diagrams showing producer embedding and consumer retrieval.
What it assumes
Assumes familiarity with Kafka producers and consumers while explaining schema registration, serialization, and deserialization.
What you can do after
Operate/configure clients that embed, retrieve, cache, and apply schema IDs during serialized message exchange.
How it connects
Connects Schema Registry, serialization, deserialization, schema evolution, compatibility, and Kafka topics.

Also explained in

Compare how 2 books explain schema id →

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Kafka for Architects. The section links point to the parts of the book it draws on.