9 Management: Tools and logging

 

This chapters covers

  • Scripting administration client options
  • Examining REST APIs, tools, and utilities
  • Managing Kafka and ZooKeeper logs
  • Finding JMX metrics
  • Advertised listeners and clients
  • Tracing using interceptors with headers

We have spent some time discussing brokers in depth in chapter 6 and client concerns throughout the earlier chapters. We saw some development practices that can be applied in most situations, but there will always be environments where special handling is required. The best way to keep a cluster moving along is to understand the data that is flowing through it and to monitor that activity at run time. Although operating Apache Kafka may not be the same as writing and running Java applications per se, it still requires monitoring log files and being aware of what is happening with our workloads.

9.1 Administration clients

So far, we have performed most of our cluster management activities with the command line tools that come with Kafka. And, in general, we need to be comfortable with a shell environment to set up and install Kafka. However, there are some helpful options we can use to branch out from these provided scripts.

9.1.1 Administration in code with AdminClient

 
 

9.1.2 kcat

 

9.1.3 Confluent REST Proxy API

 
 

9.2 Running Kafka as a systemd service

 

9.3 Logging

 
 
 
 

9.3.1 Kafka application logs

 

9.3.2 ZooKeeper logs

 
 
 

9.4 Firewalls

 
 

9.4.1 Advertised listeners

 
 

9.5 Metrics

 
 
 

9.5.1 JMX console

 

9.6 Tracing option

 
 
 

9.6.1 Producer logic

 
 
 

9.6.2 Consumer logic

 
 
 
 

9.6.3 Overriding clients

 
 
 

9.7 General monitoring tools

 
 

Summary

 
 
 
 

References

 
 
 
sitemap

Unable to load book!

The book could not be loaded.

(try again in a couple of minutes)

manning.com homepage
test yourself with a liveTest