Hbase+opentsdb stand-alone version construction

Monday, February 19, 2018 Lee

This experimental step is relatively simple, can only be used to demonstrate the construction process, the actual production environment is very complex. Lab Environment:

centos6.5 x86_64
ip:10.0.20.25

Here the experiment is not used in HDFs, production environment, hbase or to save data to hdfs more secure. 1. Installing the stand-alone zookeeper

cd/root/
Tar XF zookeeper-3.4.8.tar.gz-c./
MV Zookeeper-3.4.8/opt/zk

Cd/opt/zk
The cat conf/zoo.cfg content is as follows:

tickTime=2000initLimit=10syncLimit=5dataDir=/tmp/zookeeperclientPort=2181

/opt/zk/bin/zkserver.sh start/opt/zk1/zk1.cfg

/opt/zk/bin/zkcli.sh-server 10.0.20.25:2181
?

2. Installing a standalone HBase

Tar XF hbase-1.2.6-bin.tar.gz–c/usr/local
cd/usr/local/
Ln–s hbase-1.2.6 HBase
mkdir data//data for HBase "production environment, for security reasons, HBase is stored in HDFs"
CD conf

Vim hbase-env.sh Last added 3 lines:

export JAVA_HOME=/usr/java/jdk1.8.0_91export PATH=${JAVA_HOME}/bin:$PATHexport HBASE_MANAGES_ZK=false

Cat Hbase-site.xml Add the code for the Red section:

<?xml version="1.0"?><?xml-stylesheet type="text/xsl" href="configuration.xsl"?><configuration> <property>    <name>hbase.rootdir</name>    <value>/usr/local/hbase/data/</value>  </property><property>    <name>hbase.master.info.port</name>    <value>16001</value></property> <property>    <name>hbase.cluster.distributed</name>    <value>true</value>  <!-- false是单机模式，true是分布式模式。若为false,Hbase和Zookeeper会运行在同一个JVM里面。-->  </property>  <property>    <name>hbase.zookeeper.quorum</name>    <value>localhost:2181</value>  </property></configuration>

For more details, see also: https://www.cnblogs.com/yangsy0915/p/5544533.html

The cat regionservers content is as follows:
localhost
Note that the content written in Regionservers is the same as the hostname in the above hbase.zookeeper.quorum.

?

Start HBase:

cd /usr/local/hbase/bin./start-hbase.sh  即可启动hbase

?

Verify:

ss–lnt | grep 16001
PS Aux|grep HBase

Add, hdfs+hbase+3 node ZK's hbase-site.xml configuration file notation:

<configuration>//Set directory to write data to HDFs <property> <name>hbase.rootdir</name> &LT;VALUE&GT;HD Fs://master:9000/usr/local/hadoop-2.6.0/hbasedata</value> </property>//Set HBase mode to cluster mode <property > <name>hbase.cluster.distributed</name> <value>true</value> </property>//Set H Base's master port address <property> <name>hbase.master</name> <value>hdfs://master:60000</value ></property>//hbase Master Web interface bound port, default is 0.0.0.0<property> <name>hbase.master.info.port</ name> <value>60010</value></property>//Connection Zookeeper port settings <property> <name>h base.zookeeper.property.clientport</name> <value>2183</value> </property>//Set The connection address of the zookeeper (must be a base) <property> <name>hbase.zookeeper.quorum</name> <value >master,slave1,slave2</value> <Path configuration in/property>//zookeeper zoo.conf, storage location of snapshots <property> <name>hbase.zookeeper.property.datadir </name> <value>/usr/local/zookeeper-3.4.6/data</value></property>//zookeeper Connection Timeout period < Property> <name>zookeeper.session.timeout</name> <value>60000</value></property ></configuration>

3. Installing OPENTSDB

Official website: http://opentsdb.net/docs/build/html/installation.html#id1

RPM–IVH opentsdb-2.3.0.rpm
?
Cd/etc/opentsdb
Vim opentsdb.conf content is as follows:

tsd.network.port = 4242tsd.network.bind = 0.0.0.0tsd.network.worker_threads = 8tsd.http.staticroot = /usr/share/opentsdb/static/tsd.http.cachedir = /tmp/opentsdbtsd.core.auto_create_metrics = truetsd.core.plugin_path = /usr/share/opentsdb/pluginstsd.storage.hbase.zk_quorum = localhost:2181

?
Before starting, you need to go to hbase to create the table:

cd /usr/share/opentsdb/toolsenv COMPRESSION=NONE HBASE_HOME=/usr/local/hbase ./create_table.sh

Startup and Verification:
/etc/init.d/opentsdb start

?
ss-lnt | grep 4242
PS Aux|grep Opentsdb
Accessing the browser's http://10.0.20.25:4242
?

4. Write data test HTTP API Write data method:

Official Document: Http://opentsdb.net/docs/build/html/api_http/put.html

You can use the Chrome plugin to post data to Opentsdb as follows. "Return value 204 indicates successful submission"

You can also use Tcollector to capture system data and write to Opentsdb as follows:

?
git clone https://github.com/OpenTSDB/tcollector.git
CD Tcollector
If you do not modify the tcollector.py file, the default grabber will send the collected things to localhost local opentsdb inside. "Default parameters such as"

?
./tcollector Start//The capture program can be started in the background
PS aux | grep tcollector

?
If you need to collect other software such as MySQL monitoring status, you need to collectors/etc/this directory in the corresponding script, add the relevant account name and password, in order to collect the program to collect information.

?
Then, you can go to the Opentsdb Web interface to draw, the following is the data I collected for several hours to show:

?
Put it in the Grfana, the data will show better.

?
The configuration is roughly like this:

Hbase+opentsdb stand-alone version construction