Under Windows Kafka Source reading environment construction

Source: Internet
Author: User
Tags stack trace zookeeper aliyun

    1. Tool Preparation: Jdk1.8,scala-2.11.11,gradle-3.1,zookeeper-3.4.5,kafka-0.10.0.1-src.tgz, kafka_2.11-0.10.0.1.tgz

    2. Installing the JDK

    3. Install Scala

    4. Build Zookeeper

    5. Kafka Source Code Construction

      Unzip kafka-0.10.0.1-src.tgz, command line kafka-0.10.0.1-src, execute gradle idea


      650) this.width=650; "Src=" Https://s5.51cto.com/oss/201711/05/e7b90ed54a50810c84f71b7b03bc4dbc.png-wh_500x0-wm_3 -wmp_4-s_3555752997.png "title=" 1.png "alt=" E7b90ed54a50810c84f71b7b03bc4dbc.png-wh_ "/>

Note:

1) Gradle idea, with foreign libraries to build the download speed is very slow, enter the project, found Build.gradle file modified library for Ari Image Library

Buildscript {

Repositories {

maven{url ' http://maven.aliyun.com/nexus/content/groups/public/'}

}

Apply From:file (' Gradle/buildscript.gradle '), To:buildscript


dependencies {

For Apache Rat plugin to ignore non-git files

Classpath "org.ajoberstar:grgit:1.5.0"

Classpath ' com.github.ben-manes:gradle-versions-plugin:0.12.0 '

Classpath ' org.scoverage:gradle-scoverage:2.0.1 '

}

}


allprojects {

Apply plugin: ' Idea '

Apply plugin: ' Eclipse '

Apply plugin: "Jacoco"


allprojects {

Repositories {

maven{url ' http://maven.aliyun.com/nexus/content/groups/public/'}

}

}

2) error when executing gradle idea


* What went wrong:

A problem occurred evaluating root project ' KAFKA-0.10.0.1-SRC '.

> Failed to apply plugin [class ' Org.gradle.api.plugins.scala.ScalaBasePlugin ']

> No such property:useant for class:org.gradle.api.tasks.scala.ScalaCompileOptions

* Try:

Run with--STACKTRACE option to get the stack trace. Run with--info or--debug option to get more log output.

BUILD FAILED


Workaround: Open the KAFKA-0.10.0.1-SRC directory under the Build.gradle file

Add as Downstream

[Plain] View plain copy

ScalaCompileOptions.metaClass.daemonServer = True

ScalaCompileOptions.metaClass.fork = True

ScalaCompileOptions.metaClass.useAnt = False

ScalaCompileOptions.metaClass.useCompileDaemon = False


7. Open KAFKA-0.10.0.1-SRC with IntelliJ idea

650) this.width=650; "Src=" Https://s4.51cto.com/oss/201711/05/2e54b08ee3d7b2412c280f36f31cd0bd.png-wh_500x0-wm_3 -wmp_4-s_2374849679.png "title=" 2.png "alt=" 2e54b08ee3d7b2412c280f36f31cd0bd.png-wh_ "/>

8. Copy the Log4j.properties file into the Src/main/scala path of the core module so that the log can be displayed properly

650) this.width=650; "Src=" Https://s4.51cto.com/oss/201711/05/2ed43d098b7cebdf2faa21876ad0dac4.png-wh_500x0-wm_3 -wmp_4-s_3605728480.png "title=" 3.png "alt=" 2ed43d098b7cebdf2faa21876ad0dac4.png-wh_ "/>

9. Increase SCALA-SDK

650) this.width=650; "Src=" Https://s1.51cto.com/oss/201711/05/0c3b177e184622b505534f8879c596fc.png-wh_500x0-wm_3 -wmp_4-s_88991643.png "title=" 4.png "alt=" 0c3b177e184622b505534f8879c596fc.png-wh_ "/>

10. Configure Startup Parameters

650) this.width=650; "Src=" Https://s4.51cto.com/oss/201711/05/12e18887649e11e1f9db69d044850dc6.png-wh_500x0-wm_3 -wmp_4-s_1139839183.png "title=" 5.png "alt=" 12e18887649e11e1f9db69d044850dc6.png-wh_ "/>

11. Start the project

650) this.width=650; "Src=" Https://s3.51cto.com/oss/201711/05/ce3716f9bb285fd7a644b66b2441c228.png-wh_500x0-wm_3 -wmp_4-s_475031007.png "title=" 6.png "alt=" Ce3716f9bb285fd7a644b66b2441c228.png-wh_ "/>

12. Verification

Unzip kafka_2.11-0.10.0.1.tgz file, command line into Bin/windows

1) Create topic

C:\webserver\kafka_2.11-0.10.0.1\bin\windows>kafka-topics.bat--create--zookeeper localhost:2181-- Replication-factor 1--

Partitions 1--topic testmsg

Created topic "Testmsg".


The console logs are as follows:

650) this.width=650; "Src=" Https://s5.51cto.com/oss/201711/05/38fb278207d888d2f482d0fced205b5d.png-wh_500x0-wm_3 -wmp_4-s_3427900052.png "title=" 7.png "alt=" 38fb278207d888d2f482d0fced205b5d.png-wh_ "/>

2) Executing producer commands, generating messages:

650) this.width=650; "Src=" Https://s5.51cto.com/oss/201711/05/d95d2efbc0fb9428444026348165dd22.png-wh_500x0-wm_3 -wmp_4-s_3822918000.png "title=" 8.png "alt=" D95d2efbc0fb9428444026348165dd22.png-wh_ "/>

3) Execution of consumer orders, consumer messages:

650) this.width=650; "Src=" Https://s1.51cto.com/oss/201711/05/c81d90a057bfd0c7c3221db7726f6f21.png-wh_500x0-wm_3 -wmp_4-s_2851271549.png "title=" 9.png "alt=" C81d90a057bfd0c7c3221db7726f6f21.png-wh_ "/>

Problem:

ERROR Processor got uncaught exception. (Kafka.network.Processor)
Java.nio.BufferUnderflowException

Solution:

This is the client's jar package and the cluster version number is inconsistent with the minor version, keeping the version number consistent.

Scala-2.11.11,kafka-0.10.0.1-src.tgz, kafka_2.11-0.10.0.1.tgz


This article is from the "Coho" blog, please be sure to keep this source http://gauyanm.blog.51cto.com/629619/1979198

Under Windows Kafka Source reading environment construction

Contact Us

The content source of this page is from Internet, which doesn't represent Alibaba Cloud's opinion; products and services mentioned on that page don't have any relationship with Alibaba Cloud. If the content of the page makes you feel confusing, please write us an email, we will handle the problem within 5 days after receiving your email.

If you find any instances of plagiarism from the community, please send an email to: info-contact@alibabacloud.com and provide relevant evidence. A staff member will contact you within 5 working days.

A Free Trial That Lets You Build Big!

Start building with 50+ products and up to 12 months usage for Elastic Compute Service

  • Sales Support

    1 on 1 presale consultation

  • After-Sales Support

    24/7 Technical Support 6 Free Tickets per Quarter Faster Response

  • Alibaba Cloud offers highly flexible support services tailored to meet your exact needs.