Zookeeper + Kafka cluster Installation

Zookeeper + Kafka cluster Installation

Record the Zookeeper + Kafka cluster installation process in detail. Prepare three virtual machines. The system is RHEL64.

1) the configuration of each machine is as follows:

$ Cat/etc/hosts

...
# Zookeeper hostnames:
192.168.8.182 zk1
192.168.8.183 zk2
192.168.8.184 zk3

2) install jdk, zookeeper, and kafka on each machine. The configuration is as follows:

$ Vi/etc/profile

...

# Jdk, zookeeper, kafka
Export KAFKA_HOME =/usr/local/lib/kafka/kafka_2.9.2-0.8.11
Export ZK_HOME =/usr/local/lib/zookeeper/zookeeper-3.4.6
Export CLASSPATH =.: $ JAVA_HOME/lib/tools. jar: $ JAVA_HOME/lib/dt. jar
Export PATH = $ JAVA_HOME/bin: $ JAVA_HOME/jre/bin: $ KAFKA_HOME/bin: $ ZK_HOME/bin: $ PATH

3) run on each machine:

$ Source/etc/profile

$ Mkdir-p/var/lib/zookeeper

$ Cd $ ZK_HOME/conf

$ Cp zoo_sample.cfg zoo. cfg

$ Vi zoo. cfg

...

DataDir =/var/lib/zookeeper

# The port at which the clients will connect
ClientPort = 2181

# Zookeeper cluster
Server.1 = zk1: 2888: 3888
Server.2 = zk2: 2888: 3888
Server.3 = zk3: 2888: 3888

4) generate the myid on each machine:

Zk1:

$ Echo "1">/var/lib/zookeeper/myid

Zk2:

$ Echo "2">/var/lib/zookeeper/myid

Zk3:

$ Echo "3">/var/lib/zookeeper/myid

5) run setup on each machine to close the Firewall

Firewall:

[] Enabled

6) Start zookeeper on each machine:

$ ZkServer. sh start

View status:

$ ZkServer. sh status

Kafka architecture design of the distributed publish/subscribe message system

Apache Kafka code example

Apache Kafka tutorial notes

Getting started with Kafka

For more details, please continue to read the highlights on the next page:

1 2 Next Page