A Kafka use case can have well-defined events and still lack the information needed to size and operate it. Functional requirements describe what the system must do; nonfunctional requirements specify operating targets and constraints, such as message size, latency, and throughput. Gather them before configuring producers, brokers, and consumers or estimating infrastructure costs. section 10.1.4 section 10.5 section 10.0
Turn the use case into measurable inputs:
- Record average and maximum message size. These affect batching, compression, memory and network buffers, and safe broker and consumer limits. section 10.1.4
- Define which delay matters. Producer acknowledgment, broker delivery, consumer fetching, processing, and the final business effect measure different parts of latency. section 10.1.4
- State expected volume, peak rate, activity pattern, and how quickly a burst backlog must be processed. A daily average alone does not describe peak load. section 10.1.4
A throughput requirement
A use case might expect 300,000 messages per day, reach 50 events per second during business hours and 100 during bursts, and require a peak-load backlog to clear within five minutes. Those figures provide a more useful basis for sizing and testing than a single daily total. section 10.1.4
Use the requirements to judge platform fit as well as Kafka settings. Scalability, real-time processing, and transactional guarantees can favor different messaging platforms. Then test the proposed Kafka configuration under controlled load: specify message size and rate, configure test topics and clients, and compare measured throughput and latency with the targets. Requirements also inform service-level agreements, budgets, and disaster-recovery planning. section 5.5 section 10.3.3 section 1.6.1
Table 10.1 Different types of latency
| Latency type |
Description |
| Producer latency |
Time between calling |
| Broker delivery latency |
Time the message spends in Kafka before being fetched. |
| Consumer latency |
Time between the message being available in Kafka and the consumer fetching it. This reflects the delivery delay from Kafka. |
| Processing latency |
Time taken by the consumer to process the message after fetching it, including deserialization, validation, enrichment, and DB writes. |
| End-to-end latency |
Time from message production to the final business effect (e.g., from notification sent to decision recorded). This encompasses all of the above stages. |
Compare the latency stages before choosing a target; a fast producer acknowledgment does not by itself measure the time to the final business effect. section 10.1.4
Table 10.2 Defining throughput requirements
| Throughput requirement |
Example |
| Expected volume |
300,000 messages per day |
| Peak rate |
50 events/sec during business hours; up to 100/sec during bursts |
| Pattern |
Real-time events, with high activity from 9 AM to 5 PM on weekdays |
| Burst handling |
System must buffer and process backlog within 5 minutes during peak load |
Use the sample volume, peak, pattern, and backlog targets as a model for recording throughput requirements. section 10.1.4
