Learn kafka with me (2) and learn kafka

Learn kafka with me (2) and learn kafka

Kafka is installed on a linux server in many cases, but we are learning it now, so you can try it on windows first. To learn kafk, you must install kafka first. I will describe how to install kafka in windows.

Step 1: Install jdk first. Because kafka depends on the Java environment, make sure that JDK is installed in your system. (I won't talk about jdk installation. Please share a download link here: http://pan.baidu.com/s/1i5jgMFF)

Step 2: here we need to use a full zookeeper instead of the one packaged with Kafka, because this is a single-node Zookeeper instance. You can also run Kafka with zookeeper packaged in the \ kafka \ bin \ windows library. : Bytes.

Step 3: Decompress zookerper to your specified directory. I decompress it to drive F.

After decompression, the configuration file and environment variables must be modified before they can be used!

1 ). Open the F: \ zookeeper-3.4.8 \ conf \ zoo_sample.cfg file, it is recommended not to open with notepad, with noteped ++ text editor to open, press ctrl + f to find the dataDir attribute and change the subsequent attribute to the figure shown in the figure. This attribute means the location where the log file is stored. You must create a data folder manually, not automatically created!

The port number is 2181 by default.

2 ). Add ZOOKEEPER_HOME to the system variable and set the value to the zookeeper installation path:

Modify the path Variable with the value System Variable % ZOOKEEPER_HOME % \ bin. Note that you are not allowed to clear the path value !!!, It is added at the beginning or end! Remember to separate values.

Now that the configuration is complete, let's start the service, start the zkserver command in the bin directory, hold down the shift key, right-click the machine, and you will find another option: Open the command here, select it.

If you see the text, it indicates that the installation is successful. Congratulations, Zookeeper has been completed and runs on port 2181. Otherwise, go back to Step 3 and try again!

Step 4: Decompress kafka to the directory you specified. I have to go to the f disk.

Of course, you have to modify something before you can use it.

1), enter the Kafka configuration directory, such as F: \ kafka_2.11-0.9.0.1 \ config, edit the file "server. properties"

 

Find and modify log. dirs value: f: \ kafka_2.11-0.9.0.0 \ kafka-logs, of course, this folder is also manually created! If Zookeeper runs on some other machines or clusters, you can change "zookeeper. connect: 2181" to a custom IP address and port. In this demonstration, we use the same machine, so there is no need to make changes. The Kafka port and broker. id in the file can also be configured. Other settings remain unchanged. OK. Now we start kafka. Important: Make sure that the Zookeeper instance is ready and running before starting the Kafka server.

Step 5: Go to the Kafka installation directory, press Shift + right-click, select the "open command window" option, open the command line, and enter now. \ bin \ windows \ kafka-server-start.bat. \ config \ server. properties and press ENTER

If everything is normal, you can see that the interface should be normal, otherwise please solve it on your own Baidu, note that JDK versions must be above 1.7

 

Now that Kafka is ready and running, you can create a topic to store messages. We can also generate or use data from Java/Scala code or directly from the command line.

Now create a topic named "test" and open a new command line in f: \ kafka_2.11-0.9.0.1 \ bin \ windows. Enter the following command and press Enter:

kafka-topics.bat --create --zookeeper localhost:2181 --replication-factor 1 --partitions 1 --topic test

Create a Producer and Consumer to test the server and open a new command line in f: \ kafka_2.11-0.9.0.1 \ bin \ windows. Enter the following command and press Enter:

kafka-console-producer.bat --broker-list localhost:9092 -- Topic test, open another f: \ kafka_2.11-0.9.0.1 \ bin \ windows new command line. Enter the following command and press Enter:

kafka-console-consumer.bat --zookeeper localhost:2181 --topic test


 

Input any text on the producer side, and you will find that the consumer side will receive the text.

OK, Kafka installation is complete! I will show you how to use java later. I am looking for code because the code is on the company's computer, and now I am home!