Kafka Stand-alone installation

Preface

Kafka is a distributed, multi-partition, multi-replica messaging service. With Message Queuing, producers and consumers interact asynchronously without having to wait for each other. Compared to traditional messaging services, Kafka has the following features:
Themes can be scaled horizontally by partitioning (Partition).
Partitions are distributed across multiple nodes to achieve high data availability.
Through the consumer group (Consumer Group) to support a single consumer in a queue or pub/sub form of message consumption, or multiple consumer clusters order consumer messages.

Related Configuration

1. Download

Download Kafka and Zookeeper
Https://www.apache.org/dyn/closer.cgi?path=/kafka/1.0.0/kafka_2.11-1.0.0.tgz

https://mirrors.tuna.tsinghua.edu.cn/apache/zookeeper/

Extract
# TAR-ZXVF Kafka_2.11-1.0.0.tgz

# MV kafka_2.11-1.0.0 Kafka
Decompression Zookeeper

2, Configuration Zookeeper
Kafka used to zookeeper, all start Zookper First, the following simple to enable a single-instance Zookkeeper service.
A) Create a configuration file

$ mv zoo_sample.cfg  zoo.cfg
$ VI conf/zoo.cfg

The following configuration is added:

ticktime=2000
datadir=/usr/software/zookeeper/data
clientport=2181
initlimit=5
syncLimit=2

b) Start the Zookeeper server

JMX enabled by default
Using config:/usr/software/zookeeper/zookeeper/bin/. /conf/zoo.cfg
Starting zookeeper ... STARTED

3. Start Kafka
You can start the server by giving the following command-

$./bin/kafka-server-start.sh Config/server.properties &

[2017-11-12 22:30:27,960] WARN No meta.properties file under Dir/tmp/kafka-logs/meta.properties (Kafka.server.BrokerMetadataCheckpoint)
[2017-11-12 22:30:27,988] INFO Kafka version:1.0.0 (org.apache.kafka.common.utils.AppInfoParser)
[2017-11-12 22:30:27,993] INFO Kafka commitid:aaa7af6d4a11b29d (org.apache.kafka.common.utils.AppInfoParser)
[2017-11-12 22:30:27,999] INFO [Kafkaserver Id=0] started (kafka.server.KafkaServer)

After starting Kafka broker, type command JPS on the terminal and see the following response-

[Root@centos0 bin]# JPS

2133 Kafka
1762 Quorumpeermain
2434 Jps
Now you can see that two daemons are running on the terminal, Quorumpeermain is the Zookeeper daemon, and the other is the Kafka daemon.

Stop Kafka Server
After you have performed all the actions, you can stop the server by using the following command-

$./kafka-server-stop.sh config/server.properties

4. Create Kafka Theme Topic
Single node-single agent configuration: A zookeeper and proxy ID instance, the following are the steps to configure it:
a) Create theme Test_kafka
Create Kafka topic-Kafka provides a command-line utility called kafka-topics.sh that is used to create a theme on the server.
Example

./bin/kafka-topics.sh--create--zookeeper localhost:2181--replication-factor 1--partitions 1--topic Test_kafka

Create a theme named Test_kafka topic

List of topics
To get a list of topics in the Kafka server, you can use the following command-
Grammar

./bin/kafka-topics.sh--list--zookeeper 192.168.1.114:2181

Output
Test_kafka

b) Start the producer to send a message
Grammar
./bin/kafka-console-producer.sh--broker-list 192.168.1.114:9092--topic topic-name
From the syntax above, the producer command line client requires two main parameters-
Proxy List-The list of agents we want to send messages to. In this case, we have only one agent. The Config/server.properties file contains the proxy port ID because we know that our agent is listening on port 9092, so you can specify it directly.
Subject name-The following is an example of a topic name.
Example

[Root@centos0 kafka]#./bin/kafka-console-producer.sh--broker-list 192.168.1.114:9092--topic Test_kafka

>123123123123
>12312321;
>eee
>123123123;;;
>
c) Start the consumer to receive the message
Similar to the producer, the default consumer attribute is specified in the/config/consumer.properties file.
Grammar
./bin/kafka-console-consumer.sh--zookeeper 192.168.1.114:2181-topic topic-name--from-beginning
Example

[Root@centos0 kafka]#./bin/kafka-console-consumer.sh--zookeeper localhost:2181--topic Test_kafka--from-beginning

Using the Consoleconsumer with old consumer is deprecated and'll be removed in a future major release. Consider using the new consumer by passing [Bootstrap-server] instead of [zookeeper].
123123123123
12312321;
Eee
123123123;;;

Running the consumer command line in one terminal, and the other terminal running the producer command line, you can enter the message at one terminal and the other terminal to read the message.
Both commands have their own optional parameters that can be used without any parameters to see the help information at run time.