liveBook · Manning

Smart endpoints, dumb pipes

As explained in Kafka for Architects

The smart endpoints, dumb pipes pattern places processing and integration logic in communicating endpoints while transport channels only carry messages.

Smart endpoints, dumb pipes places processing and integration logic in communicating endpoints while transport channels only carry messages. In Kafka-based architectures, brokers transport messages without interpreting them, so endpoints own the processing. section 7.1.1

The pattern contrasts with enterprise service bus designs:

  • Avoids concentrating validation, authorization, encryption, routing, and auditing inside a logically centralized middleware component. section 7.1.1
  • Keeps endpoint-specific transformation and business processing decentralized, while cross-cutting integration concerns can be standardized through configurable infrastructure. section 7.1.1
  • Uses a service mesh to separate business logic from integration responsibilities such as encryption, auditing, retries, and observability. section 7.1.1

Decentralized Kafka architecture

A Kafka deployment can use brokers purely as communication channels, with each endpoint handling its own processing while messages flow through the pipes. section 7.1.1

Figure 7.1 All the logic resides on the smart endpoints; the pipes serve only for transport.
A screenshot of a login form

Description automatically generated

Shows processing concentrated at smart endpoints while pipes provide transport only. section 7.1.1

How Kafka for Architects covers it

How deep it goes
Contrasts endpoint-centric processing with ESB middleware and extends the discussion to service-mesh separation of concerns.
Examples
Uses a decentralized Kafka architecture and diagrams contrasting dumb pipes, smart ESB middleware, and service-mesh sidecars.
What it assumes
Assumes familiarity with Kafka messaging, endpoints, middleware, microservices, and integration concerns.
What you can do after
Evaluate where processing, business logic, and cross-cutting integration concerns should reside in event-driven architectures.
How it connects
Connects Kafka, enterprise service buses, microservices, service meshes, cross-cutting concerns, and decentralized architecture.

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Kafka for Architects. The section links point to the parts of the book it draws on.