Install and run Kafka in Windows

Install and run Kafka in Windows
Introduction

This article describes how to configure and start Apache Kafka on Windows OS. This Guide will guide you to install Java and Apache Zookeeper.
Apache Kafka is a fast and scalable Message Queue that can handle heavy read/write loads, that is, I/O-related work. For detailed steps to install Kafka in Windows, you can view this video: https://youtu.be/OJKesEpO6ok

Download required files

• Download Server JREhttp: // www.oracle.com/technetwork/java/javase/downloads/jre8-downloads-%3155.html based on the system OS and CPU architecture.
• Download and install 7-ziphttp: // www.7-zip.org/download.html
• Download Zookeeper here and unzip the http://zookeeper.apache.org/releases.html with 7-zip
• Download Kafka and unzip the http://kafka.apache.org/downloads.html with 7-zip

  For this tutorial, we extract Zookeeper and Kafka to drive C, but you can also select another location. Here we need to use a full zookeeper instead of the one packaged with Kafka, because this is a single-node Zookeeper instance. You can also run Kafka with zookeeper packaged in the \ kafka \ bin \ windows library.

  Install

  A. Install JDK

  1. Start JRE installation, select the check box "modify target path", and click Install.

  2. modify the installation directory. The folder name cannot contain spaces, for example, C: \ Java \ jre1.8.0 _ xx \ (C: \ Program Files \ Java \ jre1.8.0 _ xx by default ), click Next.
  3. Click Control Panel> system> advanced system Settings> environment variables to open the system environment variables dialog box.
  4. Click the new user variable button in the user variable, enter JAVA_HOME in the variable name, and fill in the Custom jre path with the variable value. As shown in:

  The Java path and version may change according to the Kafka version used.

   

  5. Click OK now.
  6. In the "environment variables" dialog box that you just opened, you can find the PATH variables in the system variables column.
  7. Edit the path and type "; % JAVA_HOME % \ bin", for example: <喎?http: www.bkjia.com kf ware vc " target="_blank" class="keylink"> VcD4NCjxwPjxpbWcgYWx0PQ = "picture description" src = "http://www.bkjia.com/uploads/allimg/160220/040PL540-2.png" title = "\"/>

  8. Confirm the Java installation. Open cmd and enter the type "java-version". You should be able to see the installed java version.

  If the command line prompt is similar, continue. Otherwise, you need to re-check whether the installed version matches the OS architecture (x86, x64) and whether the environment variable path is correct.

  B. Install Zookeeper

  1. Enter the Zookeeper setting directory, such as C: \ zookeeper-3.4.7 \ conf
  2. Rename "zoo_sample.cfg" to "zoo. cfg ".
  3. Open zoo. cfg in any text editor (such as notepad). I personally prefer notepad ++.
  4. Locate and edit dataDir =/tmp/zookeeper to: \ zookeeper-3.4.7 \ data
  5. Similar to the practice in Java, we add the following in the system environment variables:
  A. Add ZOOKEEPER_HOME = C: \ zookeeper-3.4.7 to the system variable
  B. Edit the System Variable and name it System Variable % ZOOKEEPER_HOME % \ bin;
  6. Modify the default Zookeeper port (default port 2181) in the zoo. cfg file ).
  7. Open the new cmd, enter zkserver, and run Zookeeper.
  8. The command line prompt is as follows:

  Congratulations, Zookeeper has been completed and runs on port 2181.

  C. Install Kafka

  1. Enter the Kafka configuration directory, such as C: \ kafka_2.11-0.9.0.0 \ config
  2. edit the file "server. properties"
  3. Locate and edit "log. dirs =/tmp/kafka-logs" to "log. dir = C: \ kafka_2.11-0.9.0.0 \ kafka-logs"
  4. If Zookeeper runs on some other machines or clusters, you can change "zookeeper. connect: 2181" to a custom IP address and port. In this demonstration, we use the same machine, so there is no need to make changes. The Kafka port and broker. id in the file can also be configured. Other settings remain unchanged.
  5. Kafka runs on port 9092 by default and connects to the default port 2181 of zookeeper.

  D. Run the Kafka Server

  Important: Make sure that the Zookeeper instance is ready and running before starting the Kafka server.

  1. Go to the Kafka installation directory C: \ kafka_2.11-0.9.0.0 \
  2. Right-click Shift + and select the "open command window" option to open the command line.
  3. Enter. \ bin \ windows \ kafka-server-start.bat. \ config \ server. properties and press Enter.

  .\bin\windows\kafka-server-start.bat .\config\server.properties

  4. If everything is normal, the command line should be like this:

  5. Now that Kafka is ready and running, you can create a topic to store messages. We can also generate or use data from Java/Scala code or directly from the command line.

  E. Create a topic

  1. Now create a topic named "test" and replication factor = 1 (because only one Kafka server is running ). If more than one Kafka server is running in the cluster, you can add replication-factor to improve data availability and system fault tolerance.
  2. Open a new command line in C: \ kafka_2.11-0.9.0.0 \ bin \ windows.
  3. Enter the following command and press Enter:

  kafka-topics.bat --create --zookeeper localhost:2181 --replication-factor 1 --partitions 1 --topic test

  F. Create a Producer and a Consumer to test the server.

  1. Open a new command line in C: \ kafka_2.11-0.9.0.0 \ bin \ windows.
  2. Enter the following command to start producer:

  kafka-console-producer.bat --broker-list localhost:9092 --topic test

  3. Open the new command line again in the same location C: \ kafka_2.11-0.9.0.0 \ bin \ windows.
  4. Enter the following command to start consumer:

  kafka-console-consumer.bat --zookeeper localhost:2181 --topic test

  5. Now there are two command line windows, such:

  6. input any content in the producer command line and press Enter. The corresponding message can be seen in other consumer command lines.

  7. If the message can be pushed to the consumer end and displayed, the Kafka installation is complete.

  Some useful commands

  1. list topics: kafka-topics.bat-list-zookeeper localhost: 2181
  2. Description topic: kafka-topics.bat-describe-zookeeper localhost: 2181-Topic [topic Name]
  3. read messages from the beginning: kafka-console-consumer.bat-zookeeper localhost: 2181-topic [Topic Name]-from-beginning
  4. delete topic: kafka-run-class.bat kafka. admin. TopicCommand-delete-topic [topic_to_delete]-zookeeper localhost: 2181