Build Kafka Cluster

1. Start the Zookeeper server

./zookeeper-server-start.sh/opt/cx/kafka_2.11-0.9.0.1/config/zookeeper.properties

2. Modify the Broker-1,broker-2 configuration

broker.id=1listeners=plaintext://:9093 # The port the socket server listens onport=9093log.dirs=/opt/cx/kafka/ Kafka-logs-1

broker.id=2listeners=plaintext://:9094# the port the socket server listens onport=9094log.dirs=/opt/cx/kafka/ Kafka-logs-2

3. Start the Broker-1,broker-2 server

./kafka-server-start.sh/opt/cx/kafka/config/server-1.properties &

./kafka-server-start.sh/opt/cx/kafka/config/server-2.properties &

4. View the broker cluster

Ps-ef|grep Kafka

5. Create and view topic

./kafka-topics.sh--create--zookeeper localhost:2181--replication-factor 2--partitions 1--topic my-replicated-topiccreated topic "My-replicated-topic".

View Topic

./kafka-topics.sh--describe--zookeeper localhost:2181--topic my-replicated-topictopic:my-replicated-topic Partitioncount:1 replicationfactor:2 configs:topic:my-replicated-topic partition:0 leader:1 Replicas : isr:1,2

As you can see from the console, a total of one theme is created (the first row is a summary row, and the following list shows the details).

Leader: A node that reads and writes messages from a partition. Each node becomes leader random.

Replicas: The node that replicates the log, whether it is leader or not, regardless of whether it is still available.

ISR: The replica set of the synchronization state. This replica set includes nodes that may later become leader for the active person.

Now look at the theme test that was created on a single node (if there is no better zookeeper, the previously created theme exists). We found no replicas at all (replicas:0), because replicas are created only under the cluster server.

./kafka-topics.sh--describe--zookeeper localhost:2181--topic testtopic:test partitioncount:1 ReplicationFactor:1 Configs:Topic:test partition:0 leader:0 replicas:0 isr:0

6. Send a message

./kafka-console-producer.sh--broker-list localhost:9093--topic my-replicated-topicmy Replicated Message 1My Replicated Message 2

7. Receiving Messages

./kafka-console-consumer.sh--zookeeper localhost:2181--from-beginning--topic my-replicated-topicmy Replicated Message 1My Replicated Message 2

8. Test the fault tolerance mechanism, end the server-1 process, and view the cluster theme. You can see that the leader node has been transferred, and Server-1 is not already in the synchronized replication set.

[[email protected] bin]# ps -ef|grep server-1.propertiesroot       5777 14396  0 12:07 pts/5    00:00:00 grep  server-1.propertiesroot     11351 10096  1 11:08 pts/3     00:01:01 JAVA -XMX1G -XMS1G -SERVER -XX:+USEG1GC -XX: maxgcpausemillis=20 -xx:initiatingheapoccupancypercent=35 -xx:+disableexplicitgc - djava.awt.headless=true -xloggc:/opt/cx/kafka_2.11-0.9.0.1/bin/. /logs/kafkaserver-gc.log -verbose:gc -xx:+printgcdetails -xx:+printgcdatestamps -xx:+ Printgctimestamps -dcom.sun.management.jmxremote -dcom.sun.management.jmxremote.authenticate=false  -dcom.sun.management.jmxremote.ssl=false -dkafka.logs.dir=/opt/cx/kafka_2.11-0.9.0.1/bin/. /logs -dlog4j.configuration=file:./. /config/log4j.properties -cp :/opt/cx/kafka_2.11-0.9.0.1/bin/. /libs/* kafka. kafka /opt/cx/kafka/config/server-1.properties[[email protected] bin]# kill -9  11351[[email protected] bin]# ./kafka-topics.sh  --describe --zookeeper  localhost:2181 --topic my-replicated-topictopic:my-replicated-topic     partitioncount:1    replicationfactor:2    configs:     topic: my-replicated-topic    partition: 0    leader:  2    replicas: 1,2    isr: 2

9. Rerun the consumer can still receive messages, although these messages are written by Leader-1.

[[email protected] bin]# ./kafka-console-consumer.sh  --zookeeper localhost:2181 --from-beginning --topic my-replicated-topicMy  Replicated message 1my replicated message 2 

This article is from the "This person's IT World" blog, be sure to keep this source http://favccxx.blog.51cto.com/2890523/1763046

Build Kafka Cluster