Under Windows Kafka Source reading environment construction

1. Tool Preparation: Jdk1.8,scala-2.11.11,gradle-3.1,zookeeper-3.4.5,kafka-0.10.0.1-src.tgz, kafka_2.11-0.10.0.1.tgz

2. Installing the JDK

3. Install Scala

4. Build Zookeeper

5. Kafka Source Code Construction

   Unzip kafka-0.10.0.1-src.tgz, command line kafka-0.10.0.1-src, execute gradle idea

   
   

   650) this.width=650; "Src=" Https://s5.51cto.com/oss/201711/05/e7b90ed54a50810c84f71b7b03bc4dbc.png-wh_500x0-wm_3 -wmp_4-s_3555752997.png "title=" 1.png "alt=" E7b90ed54a50810c84f71b7b03bc4dbc.png-wh_ "/>

Note:

1) Gradle idea, with foreign libraries to build the download speed is very slow, enter the project, found Build.gradle file modified library for Ari Image Library

Buildscript {

Repositories {

maven{url ' http://maven.aliyun.com/nexus/content/groups/public/'}

}

Apply From:file (' Gradle/buildscript.gradle '), To:buildscript

dependencies {

For Apache Rat plugin to ignore non-git files

Classpath "org.ajoberstar:grgit:1.5.0"

Classpath ' com.github.ben-manes:gradle-versions-plugin:0.12.0 '

Classpath ' org.scoverage:gradle-scoverage:2.0.1 '

}

}

allprojects {

Apply plugin: ' Idea '

Apply plugin: ' Eclipse '

Apply plugin: "Jacoco"

allprojects {

Repositories {

maven{url ' http://maven.aliyun.com/nexus/content/groups/public/'}

}

}

2) error when executing gradle idea

* What went wrong:

A problem occurred evaluating root project ' KAFKA-0.10.0.1-SRC '.

> Failed to apply plugin [class ' Org.gradle.api.plugins.scala.ScalaBasePlugin ']

> No such property:useant for class:org.gradle.api.tasks.scala.ScalaCompileOptions

* Try:

Run with--STACKTRACE option to get the stack trace. Run with--info or--debug option to get more log output.

BUILD FAILED

Workaround: Open the KAFKA-0.10.0.1-SRC directory under the Build.gradle file

Add as Downstream

[Plain] View plain copy

ScalaCompileOptions.metaClass.daemonServer = True

ScalaCompileOptions.metaClass.fork = True

ScalaCompileOptions.metaClass.useAnt = False

ScalaCompileOptions.metaClass.useCompileDaemon = False

7. Open KAFKA-0.10.0.1-SRC with IntelliJ idea

650) this.width=650; "Src=" Https://s4.51cto.com/oss/201711/05/2e54b08ee3d7b2412c280f36f31cd0bd.png-wh_500x0-wm_3 -wmp_4-s_2374849679.png "title=" 2.png "alt=" 2e54b08ee3d7b2412c280f36f31cd0bd.png-wh_ "/>

8. Copy the Log4j.properties file into the Src/main/scala path of the core module so that the log can be displayed properly

650) this.width=650; "Src=" Https://s4.51cto.com/oss/201711/05/2ed43d098b7cebdf2faa21876ad0dac4.png-wh_500x0-wm_3 -wmp_4-s_3605728480.png "title=" 3.png "alt=" 2ed43d098b7cebdf2faa21876ad0dac4.png-wh_ "/>

9. Increase SCALA-SDK

650) this.width=650; "Src=" Https://s1.51cto.com/oss/201711/05/0c3b177e184622b505534f8879c596fc.png-wh_500x0-wm_3 -wmp_4-s_88991643.png "title=" 4.png "alt=" 0c3b177e184622b505534f8879c596fc.png-wh_ "/>

10. Configure Startup Parameters

650) this.width=650; "Src=" Https://s4.51cto.com/oss/201711/05/12e18887649e11e1f9db69d044850dc6.png-wh_500x0-wm_3 -wmp_4-s_1139839183.png "title=" 5.png "alt=" 12e18887649e11e1f9db69d044850dc6.png-wh_ "/>

11. Start the project

650) this.width=650; "Src=" Https://s3.51cto.com/oss/201711/05/ce3716f9bb285fd7a644b66b2441c228.png-wh_500x0-wm_3 -wmp_4-s_475031007.png "title=" 6.png "alt=" Ce3716f9bb285fd7a644b66b2441c228.png-wh_ "/>

12. Verification

Unzip kafka_2.11-0.10.0.1.tgz file, command line into Bin/windows

1) Create topic

C:\webserver\kafka_2.11-0.10.0.1\bin\windows>kafka-topics.bat--create--zookeeper localhost:2181-- Replication-factor 1--

Partitions 1--topic testmsg

Created topic "Testmsg".

The console logs are as follows:

650) this.width=650; "Src=" Https://s5.51cto.com/oss/201711/05/38fb278207d888d2f482d0fced205b5d.png-wh_500x0-wm_3 -wmp_4-s_3427900052.png "title=" 7.png "alt=" 38fb278207d888d2f482d0fced205b5d.png-wh_ "/>

2) Executing producer commands, generating messages:

650) this.width=650; "Src=" Https://s5.51cto.com/oss/201711/05/d95d2efbc0fb9428444026348165dd22.png-wh_500x0-wm_3 -wmp_4-s_3822918000.png "title=" 8.png "alt=" D95d2efbc0fb9428444026348165dd22.png-wh_ "/>

3) Execution of consumer orders, consumer messages:

650) this.width=650; "Src=" Https://s1.51cto.com/oss/201711/05/c81d90a057bfd0c7c3221db7726f6f21.png-wh_500x0-wm_3 -wmp_4-s_2851271549.png "title=" 9.png "alt=" C81d90a057bfd0c7c3221db7726f6f21.png-wh_ "/>

Problem:

ERROR Processor got uncaught exception. (Kafka.network.Processor)
Java.nio.BufferUnderflowException

Solution:

This is the client's jar package and the cluster version number is inconsistent with the minor version, keeping the version number consistent.

Scala-2.11.11,kafka-0.10.0.1-src.tgz, kafka_2.11-0.10.0.1.tgz

This article is from the "Coho" blog, please be sure to keep this source http://gauyanm.blog.51cto.com/629619/1979198

Under Windows Kafka Source reading environment construction