liveBook · Manning

Resource isolation

As explained in Kafka for Architects

Resource isolation is the separation of compute, storage, or network resources so one workload cannot materially interfere with another.

Resource isolation separates log traffic from critical business streams by placing logging on a dedicated Kafka cluster. This prevents high-volume log workloads from affecting latency-sensitive applications and permits each cluster to match its workload’s operational priorities. section 5.1.3

A dedicated logging cluster can be optimized for:

  • Higher disk I/O throughput rather than low-latency storage. section 5.1.3
  • Greater memory and network bandwidth for high log volume. section 5.1.3
  • Different reliability priorities from financial or other critical business data. section 5.1.3

Separated log-processing flow

Applications send logs to Kafka; downstream processing indexes them in Elasticsearch and visualizes them in Kibana. Logging-system downtime need not stop producers or lose retained data. section 5.1.3

How Kafka for Architects covers it

How deep it goes
Explains workload-specific cluster sizing and performance trade-offs, but does not provide implementation steps or configuration details.
Examples
Uses an application-to-Kafka-to-Elasticsearch-to-Kibana logging pipeline as the concrete isolation example.
What it assumes
Assumes Kafka-based log collection and distinguishes throughput, latency, reliability, storage, memory, and bandwidth requirements.
What you can do after
Operate a logging architecture with a dedicated Kafka cluster tuned for throughput without consuming critical business-stream resources.
How it connects
Connects Kafka with log data management, Elasticsearch, Kibana, data replication, and retention policies.

Also explained in

Compare how 3 books explain resource isolation →

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Kafka for Architects. The section links point to the parts of the book it draws on.