Getting Started with Apache Kafka-basic configuration and running _kafka

Getting Started with Apache Kafka

In order to facilitate later use, the recording of their own learning process. Because there is no production link use of experience, I hope that experienced friends can leave message guidance.

The introduction of Apache Kafka is probably divided into 5 blogs, the content is basic, the plan contains the following content: Kafka basic configuration and Run Kafka command details kafka-manager basic configuration and running Kafka API simple usage Spring Boot integrated ka Fka

Kafka supports Linux and Windows environments, and this article runs the environment using Linux (CentOS).

This is the first article. Basic configuration and operation of Kafka

Kafka official https://kafka.apache.org

Kafka official documentation is very detailed, the quick Start is also very friendly, although it is in English, but look at the command can understand, Quick start https://kafka.apache.org/quickstart. The following is a basic excerpt from the official QuickStart, suggesting a better understanding of the English version. 1. Download code

First download Kafka from the following address, the current version is 0.10.2.1.

Https://kafka.apache.org/downloads

After the download is complete, unzip the compressed package and enter the Kafka directory:

> TAR-XZF kafka_2.11-0.10.2.1.tgz
> CD kafka_2.11-0.10.2.1

2. Start the server (1) Start zookeeper

Kafka uses zookeeper, so you need to first start a zookeeper server if you haven't. You can use the handy script packaged with Kafka to get a fast but rough single node zookeeper instance.

> bin/zookeeper-server-start.sh config/zookeeper.properties

There are 3 main configurations in this zookeeper:

# The directory where the snapshot is stored.
Datadir=/tmp/zookeeper
# The port at which the clients'll connect
clientport=2181
# Disable the PER-IP limit On the number of connections since a non-production config
maxclientcnxns=0

We need to remember zookeeper Port 2181, which will be used later. (2) Kafka basic Configuration

Kafka provides a basic configuration file in the Config directory. To ensure that Kafka can be accessed remotely, we need to modify both configurations.

Open the Config/server.properties file, in a very forward position with listeners and advertised.listeners two configuration comments, remove the two comments, and according to the current server's IP modified as follows:

# The address of the socket server listens on. It'll get the value returned from 
# java.net.InetAddress.getCanonicalHostName () if not configured.
#   FORMAT:
#     listeners = listener_name://host_name:port
#   EXAMPLE:
#     listeners = plaintext://your.host.name:9092
listeners=plaintext://:9092

# Hostname and port the broker would advertise to Producers and consumers. If not set, 
# It uses the ' value for ' listeners ' if configured.  Otherwise, it would use the value
# returned from Java.net.InetAddress.getCanonicalHostName ().
advertised.listeners=plaintext://192.168.16.150:9092

The current server IP is 192.168.16.150 and you need to modify the server IP that is accessible to the extranet or LAN. (3) Start Kafka

Next, start the Kafka service:

> bin/kafka-server-start.sh config/server.properties

If you appear java.lang.OutOfMemoryError:Map at startup failed

Open the bin/kafka-run-class.sh file,

Search-XX:+DISABLEEXPLICITGC,

Replace this parameter with-xx:+explicitgcinvokesconcurrent.

Specific reasons can refer to: http://blog.csdn.net/xieyuooo/article/details/7547435. Create Topic

Use the following command to create the Topic.

> bin/kafka-topics.sh--create--zookeeper localhost:2181--replication-factor 1--partitions 1--topic test

In addition to the command method, using the Kafka-manager described later can be more convenient to create. Start a consumer

Execute the following command at a new terminal.

> bin/kafka-console-consumer.sh--bootstrap-server localhost:9092--topic test--from-beginning

This consumer is simply outputting messages to the console. Start producer

> bin/kafka-console-producer.sh--broker-list localhost:9092--topic test

Once started, you can enter the content, and then return.

At this point you should be able to see the message output in the previous consumer. The display effect is as follows

The above figure is for the consumer, simply outputting the message to the console. Below is the producer, and the message is generated when carriage returns.

The basic command for Kafka is over, followed by the Kafka command line details.