Three machines: Vm1, VM2, vm3
1. Deploy the zookeeper cluster assuming there is already a deployed zookeeper cluster: ZK1, ZK2, ZK3
2, download, decompression Kafka
1 |
tar -xzvf kafka_2.10-0.8.2.1.tgz |
3, modify the vm1 above the Config/server.properties
1234 |
broker.id=1 log.dirs=/usr/local/bigdata/tmp/kafka_logs host.name=vm1 zookeeper.connect=zk1:2181,zk2:2181,zk3:2181 |
4. Copy Kafka to other machines
12 |
rsync -a /usr/local/bigdata/kafka_2.10-0.8.2.1 vm2:/usr/local/bigdata/ rsync -a /usr/local/bigdata/kafka_2.10-0.8.2.1 vm3:/usr/local/bigdata/ |
Modify the corresponding content in the server.properties of VM2, vm3
5. Start Zookeeper clusterin Zk1, ZK2, zk3 above respectively:
6. Start Kafka cluster
In Vm1, VM2, vm3 above respectively:
1 |
kafka -server -start .sh config/server.properties |
Or
1 |
kafka -server -start .sh -daemon config/server.properties >/dev/null |
7. Testing
Create a topic
1 |
kafka -topics .sh - -create - -zookeeper zk1:2181 - -partitions 1 - -replication -factor 1 - -topic test |
List all the topic
1 |
kafka -topics .sh - -list - -zookeeper zk1:2181 |
8. Close Kafka Cluster
In Vm1, VM2, vm3 above respectively:
1 |
kafka -server -stop .sh config/server.properties |
From for notes (Wiz)
Kafka Cluster Deployment