Apache Kafka Surveillance Series-kafkaoffsetmonitor

Apache Kafka China Community QQ Group: 162272557

Overview

Recently the Kafka server messaging service is on-line, and the JMX-based indicator is also written in Zabbix. But always think that something is missing. Visualize the operable interface.

The data in the Zabbix is more dispersed, and the whole cluster cannot be concentrated.

or a cluster in the broker list. Write your own web-console more time-consuming, use the prototype tool to draw some management interface things, the key itself is not the front-end technology, which is relatively weak. This does not open source community provides Kafka Web management platform Kafkaoffsetmonitor. Just take it quickly and execute it. Let's not worry, let's get this straight.

Description:

This app monitors the consumer of your Kafka service and their offset (offset) in partition in real time.

You can browse the current consumer group, and all partition consumption of each topic can be viewed at a glance. This is actually very practical, from where you know very quickly whether each partition message is consumed very quickly (without clogging). He can guide you (Kafka producer and Consumer) to optimize the code.

This web management platform retains the historical data of partition offset and consumer lag, so you can easily understand the consumer consumption of these days.

Kafkaoffsetmonitor Features:

1. From the title can be seen, Kafka offset monitor, is to monitor the consumer consumption situation, and can list each consumer offset, lag data.

2. Consumer group List

3. List of all parition for each topic (Topic,pid,offset,logsize,lag,owner)

4. View topic's historical consumption information.

Although the feature coverage is not complete, it is very useful.

1. DownloadGitHub website Download

Kafkaoffsetmonitor

Baidu Cloud Download (fast speed)

Baidu Cloud Kafkaoffsetmonitor Download

description : Baidu cloud download for the change version number, because kafkaoffsetmonitor some resource files (CSS,JS) is to visit the outside network. In particular, access to Google resources, we all know, often can not access to ask.

We recommend downloading the modified version

2. Installation

Kafkaoffsetmonitor execution is simpler because all files are executed. Resource files, jar files are packaged into Kafkaoffsetmonitor-assembly-0.2.0.jar, and can be executed directly. That's a great way to do it. Neither compile nor configure, hehe. Nor is it absolutely not configured.

A. Create a new folder Kafka-offset-console, and then copy the jar to the folder.

B. Create a new script, because you may not be a Kafka cluster. Script to start multiple

[Email protected]: Vim mobile_start_en.sh

#!/bin/bash
JAVA-XMS512M-XMX512M-XSS1024K-XX:PERMSIZE=256M-XX:MAXPERMSIZE=512M-CP kafkaoffsetmonitor-assembly-0.2.0.jar \
COM.QUANTIFIND.KAFKA.OFFSETAPP.OFFSETGETTERWEB \
--zk 192.168.2.101:2181,192.168.2.102:2182,192.168.2.103:2181/config/mobile/xxx \
--port 8086 \
--refresh 10.seconds \
--retain 7.days 1>mobile-logs/stdout.log 2>mobile-logs/stderr.log &

Note:/config/mobile/xxx represents the root folder of ZK, which needs to be created manually or not set

3. Implementation

[Email protected]: chmod +x mobile_start_en.sh

[Email protected]:./mobile_start_en.sh

Serving Resources From:jar:file:/opt/xxx/kafka-offset-console/kafkaoffsetmonitor-assembly-0.2.0.jar!/offsetapp

6 Demo:

Consumer group List

List of all Partiton consumption of topic

The meaning of the term is explained in the following example:

Topic: Topic Name at creation time

Partition: Partition number

Offset: Indicates how many message the parition has consumed

LogSize: Indicates how many message the partition has written

Lag: Indicates how many message lines have not been consumed.

Owner: Represents the Consumer

Created: the partition creation time

Last Seen: Consumption status Refresh latest time.

Kafka is executing the topic

Topic list in Kafka cluster

Broker list in Kafka cluster

watermark/2/text/ahr0cdovl2jsb2cuy3nkbi5uzxqvbgl6agl0yw8=/font/5a6l5l2t/fontsize/400/fill/i0jbqkfcma==/ Dissolve/70/gravity/southeast ">

Please specify reproduced from: http://blog.csdn.net/lizhitao/article/details/27199863

Apache Kafka Surveillance Series-kafkaoffsetmonitor