Kafka single-machine, cluster mode installation details (i.)

The environment of this article is as follows:
Operating system: CentOS 6 32-bit
JDK version: 1.8.0_77 32-bit
Kafka version: 0.9.0.1 (Scala 2.11)

1. The required environment

Kafka requires the following operating environments:
Java Installation Reference CentOS 6 install JDK8 using RPM method
Zookeeper Installation Reference: CentOS under Zookeeper standalone mode, cluster mode installation

2. Download and unzip the Kafka installation package

Kafka website Address: http://kafka.apache.org/
For example:

"http://mirrors.hust.edu.cn/apache/kafka/0.9.0.1/kafka_2.11-0.9.0.1.tgz"tar -xzvf kafka_2.11-0.9.0.1.tgzmv kafka_2.11-0.9.0.1 /opt/

3. Configure environment variables (optional)

Add Kafka_2.11-0.9.0.1/bin to path for easy access

vi /etc/profile

Add at the end:

KAFKA_HOME=/opt/kafka_2.11-0.9.0.1PATH=$PATH:$KAFKA_HOME/bin

4. Start the standalone Mode 4.1 modify the configuration file

cd /opt/kafka_2.11-0.9.0.1/configvi server.properties

Modify the following content in the configuration file:

broker.id=0        //为依次增长的：0、1、2、3、4，集群中唯一idlog.dirs=/opt/kafka_2.11-0.9.0.1/logs    //日志地址zookeeper.connect=master:2181,slave1:2181,slave2:2181//zookeeperServers列表，各节点以逗号分开

4.2 Starting the program

First make sure that zookeeper it is started and then execute it in the Kafka directory

nohup bin/kafka-server-start.sh config/server.properties&

If no error is indicated, the boot is successful. nuhop &is to implement the startup in the background.

4.3 Simple test

Open 2 terminals and execute the following commands in the Kafka directory, respectively
Start producer

bin/kafka-console-producer.--broker-list localhost:9092--topic test

Start consumer

bin/kafka-console-consumer.sh--zookeeperlocalhost:2181--topictest--from-beginning

Enter any character at the command line of producer to see if consumer is properly received.

5. Common Error 5.1 Occurs when you start Kafka

Java HotSpot (TM) Server VMSWarning: Info:os::commit_memory (0x67e00000,1073741824,0) failed;Error=' cannot allocate memory '(errno= A) # # There is insufficientMemory  forThe Java Runtime environment toContinue. # NativeMemoryAllocation (MMAP) failed to map1073741824bytes forCommitting reservedMemory. # anErrorReportfileWith more information is saved as:#/opt/kafka_2. One-0.9. 0. 1/hs_err_pid2249.Log

Cause of Error:
Kafka -Xmx1G -Xms1G The JVM memory configuration that is used by default, if the machine memory is small, you need to adjust the boot configuration.
Open/config/kafka-server-start.sh, modify
export KAFKA_HEAP_OPTS="-Xmx1G -Xms1G"
To fit the configuration of the current server, such asexport KAFKA_HEAP_OPTS="-Xmx256M -Xms128M"

Kafka single-machine, cluster mode installation details (i.)