Kafka Cluster Deployment

Three machines: Vm1, VM2, vm3

1. Deploy the zookeeper cluster assuming there is already a deployed zookeeper cluster: ZK1, ZK2, ZK3

2, download, decompression Kafka

1	tar -xzvfkafka_2.10-0.8.2.1.tgz

3, modify the vm1 above the Config/server.properties

1234	broker.id=1log.dirs=/usr/local/bigdata/tmp/kafka_logshost.name=vm1zookeeper.connect=zk1:2181,zk2:2181,zk3:2181

4. Copy Kafka to other machines

12	rsync -a/usr/local/bigdata/kafka_2.10-0.8.2.1  vm2:/usr/local/bigdata/rsync -a/usr/local/bigdata/kafka_2.10-0.8.2.1  vm3:/usr/local/bigdata/

Modify the corresponding content in the server.properties of VM2, vm3

5. Start Zookeeper clusterin Zk1, ZK2, zk3 above respectively:

1	zkServer.sh start

6. Start Kafka cluster

In Vm1, VM2, vm3 above respectively:

1	kafka-server-start.sh config/server.properties

Or

1	kafka-server-start.sh -daemonconfig/server.properties >/dev/null

7. Testing

Create a topic

1	kafka-topics.sh --create--zookeeper zk1:2181 --partitions 1 --replication-factor 1 --topictest

List all the topic

1	kafka-topics.sh --list--zookeeperzk1:2181

8. Close Kafka Cluster

In Vm1, VM2, vm3 above respectively:

1	kafka-server-stop.sh config/server.properties

  

From for notes (Wiz)

Kafka Cluster Deployment