"Turn" Apache Kafka surveillance series-kafkaoffsetmonitor

Apache Kafka Surveillance Series-kafkaoffsetmonitortime 2014-05-27 18:15:01 csdn Blog Original http://blog.csdn.net/lizhitao/article/details/27199863 ThemeApache Kafka

Apache Kafka China Community QQ Group: 162272557

Overview

Recently the Kafka server messaging service was online, and the JMX indicator parameters were also written to Zabbix, but there was always a lack of something to visualize an operable interface. The data in the Zabbix is more scattered, and the whole cluster cannot be concentrated. or a cluster in the broker list, write their own web-console more time-consuming, use the prototype tool to draw some management interface things, the key itself is not the front-end aspects of technology, this is relatively weak. This does not open source community provides the Kafka Web management platform Kafkaoffsetmonitor. Just get it up and running quickly. Let's not worry, let's get it straight.

Description

This app monitors the consumer of your Kafka service and their offset (offset) in partition in real time.

You can browse the current consumer group, and all partition consumption of each topic can be viewed at a glance. This is actually very useful, from here you quickly know whether each partition message is quickly consumed (without blocking). He can guide you (Kafka producer and Consumer) to optimize the code.

This web management platform retains the historical data of partition offset and consumer lag, so you can easily understand the consumer consumption of these days.

Kafkaoffsetmonitor Features:

1. From the title can be seen, Kafka offset monitor, is to monitor the consumer consumption situation, and can list each consumer offset, lag data.

2. Consumer group List

3. List of all parition for each topic (Topic,pid,offset,logsize,lag,owner)

4. View topic's historical consumption information.

Although the functional coverage is not complete, but very practical.

1. Download

Kafkaoffsetmonitor

2. Installation

Kafkaoffsetmonitor is easy to run because all the running files, resource files, jar files are packaged into the Kafkaoffsetmonitor-assembly-0.2.0.jar and run directly, this way is great. Neither compiles nor configures, hehe, nor is it absolutely not configured.

A. Create a new directory kafka-offset-console, and then copy the jar to the directory.

B. Create a new script, because you may not be a Kafka cluster. Scripts can be used to start multiple

[Email protected]: Vim mobile_start_en.sh

#!/bin/bash
JAVA-XMS512M-XMX512M-XSS1024K-XX:PERMSIZE=256M-XX:MAXPERMSIZE=512M-CP kafkaoffsetmonitor-assembly-0.2.0.jar \
COM.QUANTIFIND.KAFKA.OFFSETAPP.OFFSETGETTERWEB \
--zk 192.168.2.101:2181,192.168.2.102:2182,192.168.2.103:2181/config/mobile/xxx \
--port 8086 \
--refresh 10.seconds \
--retain 7.days 1>mobile-logs/stdout.log 2>mobile-logs/stderr.log &

3. Running

[Email protected]: chmod +x mobile_start_en.sh

[Email protected]:./mobile_start_en.sh

Serving Resources From:jar:file:/opt/xxx/kafka-offset-console/kafkaoffsetmonitor-assembly-0.2.0.jar!/offsetapp

6 Demo:

Consumer group List

Topic List of all Partiton consumption conditions

Kafka running Topic

Topic list in Kafka cluster

Broker list in Kafka cluster

Please specify reproduced from: http://blog.csdn.net/lizhitao/article/details/27199863

"Turn" Apache Kafka surveillance series-kafkaoffsetmonitor