Zookeeper Monitoring (Taokeeper)

http://jm.taobao.org/?p=1450

Within the company, there are many applications that have relied heavily on Zookeeper,zookeeper's working state to directly influence their normal work. At present, there is no mature zookeeper-monitor in the open source world, so we begin to zookeeper monitor this work.

At present, Zookeeper-monitor can do what things, to talk about this, first of all to see which factors on the zookeeper normal work relatively large impact:

1. The directory used for zookeeper write logs is large enough and strongly recommended on a separate disk (mount point), which is one of the biggest factors affecting ZK performance.

2. The number of connections.

3. Number of watcher registered.

4. Whether the znode is readable or writable.

5. Whether the delay of the ZK event notification is too large.

around the above, the completion of the first phase of the development of Taokeeper, the main completion of the following aspects of monitoring: (Project address: Https://github.com/alibaba/taokeeper )

L cpu/mem/load Monitoring

L ZK log directory where the disk remaining space monitoring

• Peak alarm for single connection number

L Peak alarm for single watcher number

L- node self-test : Refers to the path:/yinshi on the ZK node where each IP is located in the cluster. MONITOR. ALIVE. CHECK three times the following process: Node connection-Data Release-Modify notification-get data-data comparison, within the specified delay, three processes are successfully treated as the node is in a normal state.

First, installation

Direct deployment

Install Java JKD and Tomcat environments first (support for high version JDK and Tomcat, JDK 1.8.0_77, apache-tomcat-8.0.33, Tomcat 6 7 8 are supported)

Download JDK bin package, not Tomcat bin package, add environment variable

Such as:

# # JAVA

Export JAVA_HOME=/USR/LOCAL/JAVA/JDK

Export classpath=.: $JAVA _home/lib/dt.jar: $JAVA _home/lib/tools.jar

# # TOMcat

Export Catalina_home=/usr/local/apache-tomcat

# # PATH

Export Path= $MAVEN _home/bin:/usr/local/apache-tomcat/bin

: Http://pan.baidu.com/s/1kVet0rX

1.1 Download Taokeeper.sql, initialize database (Mysql).
1.2 Download Taokeeper-monitor.war file, extract to TOMCAT WebApps Currently, make sure the final directory structure is as follows: %tomcat_home%\webapps\taokeeper-monitor.war

1.3 New users on each zookeeper server a user, such as ZK password ZK

1.4 Install NC software on each zookeeper server , yum-y install NC # will cause zookeeper monitoring information not to come out, ZK's role, connection Lou and other information is through SSH remote execution NC command, such as NC 10.100.11.13 2181

1.5 Download the Taokeeper-monitor-config.properties file and store it in a specified directory, such as

/usr/local/taokeeper/taokeeper-monitor-config.properties, the contents of which are as follows, modify them as needed.

##-------------------------------------

#Daily-TB

Systeminfo.envname=daily-tb

#DBCP

Dbcp.driverclassname=com.mysql.jdbc.driver

Dbcp.dbjdbcurl=jdbc:mysql://localhost:3306/taokeeper

Dbcp.characterencoding=gbk

Dbcp.username=root

Dbcp.password=admin

Dbcp.maxactive=30

dbcp.maxidle=10

dbcp.maxwait=10000

#SystemConstant

#SystemConstent. datastorebasepath=/home/admin/taokeeper-monitor/datastore/

Systemconstent.datastorebasepath=/var/lib/taokeeper

# SSH User

Systemconstant.usernameofssh=zk

# SSH User Password

Systemconstant.passwordofssh=zk

# SSH Port

Systemconstant.portofssh=22

#Optional

Systemconstant.configofmsgcenter=

##-------------------------------------

1.6 Adding java_opts in the Tomcat startup script

head/usr/local/tomcat/bin/catalina.sh

#!/bin/sh

# Description:start and stop Tomcat

# Chkconfig:-21 79

Java_home=/usr/local/java/jdk

Catalina_home=/usr/local/tomcat

Java_opts=-dconfigfilepath= "/usr/local/ali_taokeeper/taokeeper-monitor-config.properties"

PS: can copy/usr/local/tomcat/bin/catalina.sh to/etc/init.d/tomcat

Second, use:

1.1 Taokeeper-monitor Boot, no zookeeper cluster has been configured, click "Join Monitoring" to add the cluster.

650) this.width=650; "alt=" Taokeeper_monitor_add_cluster "src=" http://jm.taobao.org/wp-content/uploads/2012/01/ Taokeeper_monitor_add_cluster.png "width=" 442 "height=" 165 "style=" Border:0px;float:none; "/>

1.2 Configuring Zookeeper Cluster information ( Click to view larger image )

650) this.width=650; "alt=" Taokeeper_monitor_config_cluster "src=" http://jm.taobao.org/wp-content/uploads/2012/01 /taokeeper_monitor_config_cluster.png "width=" 613 "height=" 272 "style=" border:0px;float:none; "/>

650) this.width=650; "Src=" http://s3.51cto.com/wyfs02/M02/7E/E6/wKioL1cMpKHiS4_ 0aadmmetjqba519.png "title=" 01.png "alt=" wkiol1cmpkhis4_0aadmmetjqba519.png "/> Click the Status corresponding button to view the details of all subscribers on the current server:

650) this.width=650; "src=" Http://s5.51cto.com/wyfs02/M02/7E/E6/wKioL1cMpY-yItabAADIJW2k7l4680.png "title=" 02. PNG "alt=" wkiol1cmpy-yitabaadijw2k7l4680.png "/>zookeeper Monitoring alarm settings

650) this.width=650; "src=" Http://s3.51cto.com/wyfs02/M00/7E/E7/wKioL1cMpfHzfPtBAADLCgM7Zy4598.png "title=" 03. PNG "alt=" Wkiol1cmpfhzfptbaadlcgm7zy4598.png "/>

Zookeeper Cluster Status Trend chart

Click to view trends

650) this.width=650; "src=" Http://s1.51cto.com/wyfs02/M02/7E/E9/wKiom1cMpfPgV7GIAAErFliUtH4496.png "title=" 04.png "alt=" Wkiom1cmpfpgv7giaaerfliuth4496.png "/>

Zookeeper Monitoring (Taokeeper)