大資料：從入門到XX(六)

標籤：zookeeper3.4.8叢集部署在三台redhat虛擬機器上   同時演練leader選舉過程   

什麼是ZooKeeper，看看ZooKeeper官網怎麼說：

    Apache ZooKeeper is an effort to develop and maintain an open-source server which enables highly reliable distributed coordination.

    ZooKeeper is a centralized service for maintaining configuration information, naming, providing distributed synchronization, and providing group services.

    點此進入：ZooKeeper Overview

    ZooKeeper是分布式應用環境中的基礎組件，在hadoop叢集中，hdfs和yarn都要用到ZooKeeper來配置可自動切換的高可用（HA）系統，本文根據ZooKeeper官方文檔，選用三台虛擬機器，部署ZooKeeper叢集，用到的虛擬機器為前文提到的： hadoop01、hadoop02、hadoop03。以下為詳細安裝步驟。

1、在hadoop01機器上，建立zookeeper使用者，加入hadoop組（本系列文檔中有關於建立hadoop組的部分)

#以root使用者登入hadoop01機器 [[email protected] hadoop]# useradd -m -g hadoop zookeeper #首次啟用，設定zookeeper使用者密碼 [[email protected] hadoop]# passwd zookeeper

2、下載最新的ZooKeeper發布包：官網推薦的鏡像地址

#zookeeper-3.4.8.tar.gz拷貝至 /home/zookeeper 目錄 [[email protected] ~]$ tar zxvf zookeeper-3.4.8.tar.gz 解壓後多一個檔案夾 zookeeper-3.4.8

3、使用zookeeper使用者登入，設定環境變數

#在hadoop01機器上以zookeeper使用者身份操作 [[email protected] hadoop]# su - zookeeper [[email protected] ~]$ vi .bash_profile export PATH #add start 20160628 export ZOOKEEPER_HOME=/home/zookeeper/zookeeper-3.4.8 export PATH=$PATH:$ZOOKEEPER_HOME/bin #add end 20160628 #儲存以上環境配置，退出vi後，執行以下命令是環境變數生效 [[email protected] ~]$ source .bash_profile

4、在hadoop01機器上設定zookeeper相關參數

[[email protected] ~]$ cd zookeeper-3.4.8/conf/ #從範例複製一個設定檔範本 [[email protected] zookeeper-3.4.8]$ cp zoo_sample.cfg zoo.cfg #編輯設定檔 [[email protected] conf]$ vi zoo.cfg # The number of milliseconds of each tick tickTime=2000 # The number of ticks that the initial # synchronization phase can take initLimit=10 # The number of ticks that can pass between # sending a request and getting an acknowledgement syncLimit=5 # the directory where the snapshot is stored. # do not use /tmp for storage, /tmp here is just # example sakes. #modify start 20160628 #dataDir=/tmp/zookeeper dataDir=/home/zookeeper/zookeeper-3.4.8/zkdata #modify end 20160628 # the port at which the clients will connect clientPort=2181 # the maximum number of client connections. # increase this if you need to handle more clients #maxClientCnxns=60 # # Be sure to read the maintenance section of the # administrator guide before turning on autopurge. # # http://zookeeper.apache.org/doc/current/zookeeperAdmin.html#sc_maintenance # # The number of snapshots to retain in dataDir #autopurge.snapRetainCount=3 # Purge task interval in hours # Set to "0" to disable auto purge feature #autopurge.purgeInterval=1 #添加的配置資訊 #modify start 20160628 server.1=hadoop01:2888:3888 server.2=hadoop02:2888:3888 server.3=hadoop03:2888:3888 #modify end 20160628

5、在hadoop01機器上建立工作目錄，設定zookeeper ID值（在不同伺服器上ID不能重複）

#在hadoop01機器上，以zookeeper使用者執行以下命令 [[email protected] ~]$ cd /home/zookeeper/zookeeper-3.4.8 [[email protected] zookeeper-3.4.8]$ mkdir zkdata [[email protected] zkdata]$ cd zkdata/ [[email protected] zkdata]$ echo ‘1‘ >myid

6、在hadoop02上執行相關操作

#在hadoop02機器上，建立zookeeper使用者 [[email protected] hadoop]# useradd -m -g hadoop zookeeper #在hadoop01機器上，執行以下操作 [[email protected] ~]$ cd /home/zookeeper [[email protected] ~]$ scp -r zookeeper-3.4.8 hadoop02:$PWD #在hadoop02機器上，切換zookeeper使用者身份 [[email protected] hadoop]# su - zookeeper [[email protected] ~]$ vi .bash_profile export PATH #add start 20160628 export ZOOKEEPER_HOME=/home/zookeeper/zookeeper-3.4.8 export PATH=$PATH:$ZOOKEEPER_HOME/bin #add end 20160628 #儲存以上環境配置，退出vi後，執行以下命令使環境變數生效 [[email protected] ~]$ source .bash_profile

7、建立工作目錄，設定zookeeper ID值

#在hadoop02機器上，以zookeeper使用者執行以下命令 [[email protected] ~]$ cd /home/zookeeper/zookeeper-3.4.8 [[email protected] zookeeper-3.4.8]$ mkdir zkdata [[email protected] zkdata]$ cd zkdata/ [[email protected] zkdata]$ echo ‘2‘ >myid

8、在hadoop03上執行相關操作

#在hadoop03機器上，建立zookeeper使用者 [[email protected] hadoop]# useradd -m -g hadoop zookeeper #在hadoop01機器上，執行以下操作 [[email protected] ~]$ cd /home/zookeeper [[email protected] ~]$ scp -r zookeeper-3.4.8 hadoop03:$PWD #在hadoop03機器上，切換zookeeper使用者身份 [[email protected] hadoop]# su - zookeeper [[email protected] ~]$ vi .bash_profile export PATH #add start 20160628 export ZOOKEEPER_HOME=/home/zookeeper/zookeeper-3.4.8 export PATH=$PATH:$ZOOKEEPER_HOME/bin #add end 20160628 #儲存以上環境配置，退出vi後，執行以下命令使環境變數生效 [[email protected] ~]$ source .bash_profile

9、建立工作目錄，設定zookeeper ID值

#在hadoop03機器上，以zookeeper使用者執行以下命令 [[email protected] ~]$ cd /home/zookeeper/zookeeper-3.4.8 [[email protected] zookeeper-3.4.8]$ mkdir zkdata [[email protected] zkdata]$ cd zkdata/ [[email protected] zkdata]$ echo ‘3‘ >myid

10、啟動zookeeper叢集

#在hadoop01機器上運行 [[email protected] ~]$ zkServer.sh start ZooKeeper JMX enabled by default Using config: /home/zookeeper/zookeeper-3.4.8/bin/../conf/zoo.cfg Starting zookeeper ... STARTED #在hadoop02機器上運行 [[email protected] ~]$ zkServer.sh start ZooKeeper JMX enabled by default Using config: /home/zookeeper/zookeeper-3.4.8/bin/../conf/zoo.cfg Starting zookeeper ... STARTED #在hadoop03機器上運行 [[email protected] ~]$ zkServer.sh start ZooKeeper JMX enabled by default Using config: /home/zookeeper/zookeeper-3.4.8/bin/../conf/zoo.cfg Starting zookeeper ... STARTED

11、查看zookeeper叢集狀態

[[email protected] ~]$ zkServer.sh status ZooKeeper JMX enabled by default Using config: /home/zookeeper/zookeeper-3.4.8/bin/../conf/zoo.cfg Mode: follower [[email protected] ~]$ zkServer.sh status ZooKeeper JMX enabled by default Using config: /home/zookeeper/zookeeper-3.4.8/bin/../conf/zoo.cfg Mode: leader

12、查看執行的進程

[[email protected] ~]$ jps -l 5449 org.apache.zookeeper.server.quorum.QuorumPeerMain

13、關閉zookeeper叢集

#在hadoop01機器上運行 [[email protected] ~]$ zkServer.sh stop #在hadoop02機器上運行 [[email protected] ~]$ zkServer.sh stop #在hadoop03機器上運行 [[email protected] ~]$ zkServer.sh stop

14、示範了開啟zookeeper叢集，以及殺掉leader伺服器的進程後，自動選舉新的leader伺服器情況。

650) this.width=650;" src="http://s3.51cto.com/wyfs02/M01/84/2E/wKiom1eHoRGwlagDAAJViL9LkY0639.png-wh_500x0-wm_3-wmp_4-s_3301827932.png" title="zookeeper.png" alt="wKiom1eHoRGwlagDAAJViL9LkY0639.png-wh_50" />

本文出自 “沈進群” 部落格，謝絕轉載！

大資料：從入門到XX(六)