liveBook · Manning

Gradual migration

As explained in Kafka for Architects

Gradual migration is the staged transition of functionality, components, or traffic from an older system version to a newer one.

Gradual migration replaces an all-at-once rollout with staged change: new functionality is introduced incrementally while older functionality is retired over time. In event-driven systems, producers and consumers using different event versions can coexist, reducing interruption during upgrades. section 6.2.5

Coordinate staged event changes through compatibility direction:

  • Use backward compatibility when consumers upgrade first; the new consumer must read older events. section 6.2.5
  • Use forward compatibility when producers upgrade first; older consumers must process the new events. section 6.2.5
  • Validate schemas against the selected compatibility strategy to prevent deserialization failures, while accepting restrictions on permissible schema changes. section 6.2.5

Coexisting releases

Deploy new producers, consumers, and a channel alongside the old versions. Validate the new path, then retire the old deployment after it proves suitable. section 6.2.5

Figure 6.8 When using backward compatibility, consumers are upgraded first. The new version of the consumer can handle the old messages.
A diagram of a diagram of a diagram

AI-generated content may be incorrect.

Compare consumer-first backward compatibility with producer-first forward compatibility before selecting an upgrade sequence. section 6.2.5

How Kafka for Architects covers it

How deep it goes
Explains coexistence, compatibility directions, schema validation, and parallel deployment, without detailing implementation code.
Examples
Uses backward- and forward-compatibility diagrams plus parallel producer, consumer, and channel deployments.
What it assumes
Assumes familiarity with applications, event producers, consumers, event structures, and schemas.
What you can do after
Operate staged event upgrades by choosing compatibility direction and validating schemas before retiring older versions.
How it connects
Connects data contracts with event producers, consumers, schema compatibility, and schema repositories.

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Kafka for Architects. The section links point to the parts of the book it draws on.