Kafka Local stand-alone installation deployment

Kafka is a high-throughput distributed subscription messaging system that will be Kafka in one of these days, with specific project practices documenting the Kafka local installation deployment process to share with colleagues.
Preparatory work:

The above files are placed in the/usr/local/kafka directory except for the JDK.

1, the installation of Jdk,kafka used to use the JDK
First check if there are any jdk:java-version
Cd/usr/local/hadoop (in this case I put the JDK's installation package under the Hadoop folder, you can do it on your own)
Http://www.oracle.com/technetwork/java/javase/downloads/jdk7-downloads-1880260.html
Upload the downloaded installation package to the/usr/local/hadoop folder on the server
TAR-ZXVF jdk-7u79-linux-x64.tar.gz
to configure the environment variables for the JDK:
Vim/etc/profile

Make it effective immediately source/etc/profile
Check whether the installation was successful: Java–version

The JDK installation configuration was successful.

2, Installation Zookeeper
Cd/usr/local/kafka
wget https://archive.apache.org/dist/zookeeper/zookeeper-3.3.6/zookeeper-3.3.6.tar.gz
TAR-ZXVF zookeeper-3.3.6.tar.gz
Vim/etc/profile

Make it effective immediately
Source/etc/profile

Test if the zookeeper is installed successfully
Cd/usr/local/zookeeper-3.3.6/bin
./zkserver.sh Start

As shown, the configuration for the zookeeper installation is successful.

3, Installation Kafka
Cd/usr/local/kafka
wget https://archive.apache.org/dist/kafka/0.8.0/kafka_2.8.0-0.8.0.tar.gz
TAR-ZXVF kafka_2.8.0-0.8.0.tar.gz
Configure Kafka, modify file Server.properties
Cd/usr/local/kafka/kafka_2.8.0-0.8.0/config

Vim Server.properties

Write the IP address of the local server at the Host.name

Log.dirs in combination with their own built folder path to fill, no strict requirements

The default is 2181 port in the red box

Configure Kafka under Zookeeper
Mkdir/usr/local/kafka/zookeeper #创建zookeeper目录
Mkdir/usr/local/kafka/log/zookeeper #创建zookeeper日志目录
Cd/usr/local/kafka/kafka_2.8.0-0.8.0/config
Vim Zookeeper.properties

The DataDir and Datalogdir can be filled in according to the folder path that they have created.

4. Create startup and shutdown Kafka scripts
(1) Create Kafka script
Vim kafkastart.sh

(2) Create close Kafka script
Vim kafkastop.sh

(3) Add script execution permissions
chmod +x kafkastart.sh
chmod +x kafkastop.sh

(4) Set script to start automatic execution
Vim/etc/rc.d/rc.local

5. Test Kafka

(1) Create a theme
Cd/usr/local/kafka/kafka_2.8.0-0.8.0/bin
./kafka-create-topic.sh–partition 1–replica 1–zookeeper localhost:2181–topic test

Check if the theme was created successfully
./kafka-list-topic.sh–zookeeper localhost:2181

(2) Start producer
./kafka-console-producer.sh–broker-list 192.168.18.229:9092–topic Test
(192.168.18.229 is the local server IP address, the reader is filled in according to the situation)

As shown, enter the Hello kafka! after the carriage return
and then open another shell terminal .
Cd/usr/local/kafka/kafka_2.8.0-0.8.0/bin
Start consumer under this terminal
./kafka-console-consumer.sh–zookeeper Localhost:2181–topic Test

By visible, the Terminal 1 producer under the input of the Hello kafka!, in the Terminal 2 under the consumer appears, this Kafka single-machine installation of the deployment completed.

(6) Close Kafka

Cd/usr/local/kafka
./kafkastop.sh

Attached: Problems and solutions that may be encountered
Question 1:failed to acquire lock on file.

Solution:
This is because the Kafka process has been running before, and can be as follows:
View the process number with the JPS command, kill it, and then restart the Kafka that resolves the problem

Problem 2:bindexception:address already in use

Solution:
This can sometimes be ignored, as long as the last successful start Kafka can, can also be server.properties under the port number 2181, this error is because 2181 port is occupied.

Kafka Local stand-alone installation deployment