Kafka Monitoring System

Metrics-one of the metrics measurement tools for Java

Metrics-Java metric tool 2

Java metrics metric tool Introduction 1

Java metrics measurement tool-metrics core Translation

Http://blog.synyx.de/2013/09/yammer-metrics-made-easy-part-i/

 

Http://kafka.apache.org/documentation.html#monitoring

Https://cwiki.apache.org/confluence/display/KAFKA/JMX+Reporters

Kafka monitoring, http://www.tianjiaguo.com/system-architecture/kafka/kafka%E7%9B%91%E6%8E%A7/

How to monitor and http://my.oschina.net/u/218540/blog/263704 Kafka

Apache Kafka monitoring series-kafkaoffsetmonitor

Bytes ------------------------------------------------------------------------------------------------------------------------------

Java metrics

Kafka uses Java metrics for internal status monitoring. For details, refer to the above blog and make it clear.

Java Metrics is relatively easy to use and has the following types of measurements,Gauges,Counters,Histograms,Meters,Timers.
Health checks is also supported.

Multiple report methods and metrics-core methods are supported.
The default method is JMX. Metrics always registers all your metrics as JMX mbeans. You can view them directly through jconsole or visualvm. For details, see the following.
Console mode. Metrics provides consolereporter, which periodically prints the registered metric to the console.
In the csv format, metricsuses csvreporter, and cyclically stores a series of. CSV files.

There are other reporter, such as sending data to a ready-made monitoring platform such as ganlia or graphite.
Ganliareporter returns the metrics to the ganglia server in streaming mode.
Graphitereporter returns the metrics to the graphite server in streaming mode.

 

JMX, Java Management Extension

In Kafka, JMX is used to report metrics, so let's take a look at what JMX is?
Bytes

For example, if a variety of Java components are deployed in a large system, and the monitoring and management interfaces of each component are different, it is very troublesome to develop a unified monitoring and management tool.
JMX is responsible for providing unified interfaces to the upper-Layer Management System to shield the diversity of underlying resources.

For more information about the architecture, see the following figure,

 

javax.management.MBeanServerImplemented the Agent Function and provided interfaces for the management system to access the JMX framework in a standard way.
javax.management.MBeansImplemented the subagent function and provided the interface for JMX framework to access resources in a standard way.

Open jconsole and you will be able to see mbeans representing resources. On mbeans, you can not only view the current status of these resources, but also dynamically change configurations and perform some operations.
For example, java. Lang mbean provides monitoring and management interfaces for Java virtual machines,
We can see the number of threads in the virtual machine, the current stack of the thread, memory management, the time occupied by GC, the objects in the Virtual Machine and the parameters of the current virtual machine, and other important parameters and runtime information.
You can also check and reconfigure the configuration information, view and modify the verbose parameter of the current JVM remotely, or even execute a GC command on the remote JVM.

 

How can I use jconsole to monitor metrics of Kafka?

First of all, when executing the Kafka script, add jmx_port, other JMX-related configuration in the kafka-run-class.sh kafka_jmx_opts has been configured

Jmx_port = 9999 nohup bin/kafka-server-start.sh config/server. Properties &

Open jconsole,

Input, service: JMX: RMI: // JNDI/RMI: // 11.11.11.11: 9999/jmxrmi

Then we can see that cool !!!

 

What Kafka metrics can be seen from metrics?

Reference, http://kafka.apache.org/documentation.html#monitoring

The table is messy. Sort out the table. Each metrics identifier is divided into three layers: mbean name, type, and metrics name.

Kafka. Server (mbean name)

Brokertopicmetrics (type)
Alltopicsmessagesinpersec, number of messages per second for all topics
Alltopicsbytesinpersec, number of bytes of all topics in per second
Alltopicsbytesoutperse, number of bytes of out, no number of messages of out, why? Haha

Replicamanager
Underreplicatedpartitions, that is, the actual number of replica instances is smaller than the set number (| ISR | <| all replicas |)
Partitioncount, number of Partition
Leadercount, number of leader partitions. If the replica factor is 1, it is the number of partitions.
Isrshrinkspersec: ISR occurs only when the broker go down. Shrink and expand are usually 0.
Isrexpandspersec, same as above

Replicafetchermanager
Replica-maxlag, the largest lag between follower replica and leader replica

Producerrequestpurgatory
Purgatorysize, requests waiting in the producer purgatory

Fetchrequestpurgatory
Purgatorysize, requests waiting in the fetch purgatory

 

Kafka. Network

Requestmetrics
{Produce | fetch-consumer | fetch-follower}-requestspersec, number of producer or consumer requests per second, follower refers to replica Broker
{Produce | fetch-consumer | fetch-follower}-totaltimems, request total time
{Produce | fetch-consumer | fetch-follower}-queuetimems, time the request waiting in the Request queue
{Produce | fetch-consumer | fetch-follower}-localtimems, time the request being processed at the leader
{Produce | fetch-consumer | fetch-follower}-remotetimems, time the request waits for the follower
{Produce | fetch-consumer | fetch-follower}-responsesendtimems, time to send the response

 

Kafka. Log

Log
Topic-partition-logendoffset, end offset of each partition
Topic-partition-numlogsegments, number of segements
Topic-partition-size, partition data size

 

Kafka. Controller

Kafkacontroller
Activecontrollercount, which has several active controllers

Controllerstats
Leaderelectionrateandtimems, non-zero when there are broker failures
Uncleanleaderelectionspersec

 

Kafka. Consumer

Consumerfetchermanager
([-. \ W] +)-maxlag, number of messages the consumer lags behind the producer

 

JMX reporter supported in Kafka

Https://cwiki.apache.org/confluence/display/KAFKA/JMX+Reporters