Eclipse安裝Hadoop外掛程式

標籤：style   blog   http   color   io   使用   ar   java   for   

首先說明一下配置的環境

系統：Ubuntu14.0.4

IDE：eclipse 4.4.1

Hadoop：Hadoop 2.2.0

        對於Hadoop的舊版本，可以直接複製 hadoop安裝目錄/contrib/eclipse-plugin/hadoop-0.20.203.0-eclipse-plugin.jar 到 eclipse安裝目錄/plugins/ 下（並沒有親自驗證）。對於Hadoop2，需要上https://github.com/winghc/hadoop2x-eclipse-plugin下載外掛程式的源碼自己進行編譯產生jar檔案。開始為了方便，直接下載別人編譯後的jar檔案，複製到eclipse的plugins檔案夾下，但是重啟eclipse後，無法找到Open PersPective中找到Map/Reduce外掛程式。於是猜測外掛程式的編譯需要依賴，進而決定自行編譯。

      下載外掛程式的源碼，上面有清楚的編譯方式

How to build$cd src/contrib/eclipse-plugin$ant jar -Dversion=2.2.0 -Declipse.home=/opt/eclipse -Dhadoop.home=/usr/share/hadoopfinal jar will be genrated at directory$root/build/contrib/eclipse-plugin/hadoop-eclipse-plugin-2.2.0.jar

　　由上可知，需要指定hadoop和eclipse的home目錄，一下是我首次編譯的結果

[email protected]:~/下載/hadoop2x-eclipse-plugin-master/src/contrib/eclipse-plugin$ ant jar -Dversion=2.2.0 -Declipse.home=/usr/local/eclipse -Dhadoop.home=/usr/local/hadoop/hadoop-2.2.0Buildfile: /home/hadoop/下載/hadoop2x-eclipse-plugin-master/src/contrib/eclipse-plugin/build.xmlcheck-contrib:init:     [echo] contrib: eclipse-plugininit-contrib:ivy-download:      [get] Getting: http://repo2.maven.org/maven2/org/apache/ivy/ivy/2.1.0/ivy-2.1.0.jar      [get] To: /home/hadoop/下載/hadoop2x-eclipse-plugin-master/ivy/ivy-2.1.0.jar      [get] Not modified - so not downloadedivy-probe-antlib:ivy-init-antlib:CLASSPATH element /home/hadoop/下載/hadoop2x-eclipse-plugin-master/ivy/ivy-2.1.0.jar is not a JAR.  [typedef] CLASSPATH element /home/hadoop/下載/hadoop2x-eclipse-plugin-master/ivy/ivy-2.1.0.jar is not a JAR.  [typedef] Could not load definitions from resource org/apache/ivy/ant/antlib.xml. It could not be found.BUILD FAILED/home/hadoop/下載/hadoop2x-eclipse-plugin-master/src/contrib/build-contrib.xml:476: You need Apache Ivy 2.0 or later from http://ant.apache.org/      It could not be loaded from http://repo2.maven.org/maven2/org/apache/ivy/ivy/2.1.0/ivy-2.1.0.jarTotal time: 1 second

　　編譯失敗，此時如果電腦是連網狀態，會自動下載ivy-2.1.0.jar檔案，把下載下來的檔案拷貝到上訴的目錄中，重新編譯

[email protected]:~/下載/hadoop2x-eclipse-plugin-master/src/contrib/eclipse-plugin$ ant jar -Dversion=2.2.0 -Declipse.me=/usr/local/eclipse -Dhadoop.home=/usr/local/hadoop/hadoop-2.2.0Buildfile: /home/hadoop/下載/hadoop2x-eclipse-plugin-master/src/contrib/eclipse-plugin/build.xmlcheck-contrib:init:     [echo] contrib: eclipse-plugininit-contrib:ivy-download:      [get] Getting: http://repo2.maven.org/maven2/org/apache/ivy/ivy/2.1.0/ivy-2.1.0.jar      [get] To: /home/hadoop/下載/hadoop2x-eclipse-plugin-master/ivy/ivy-2.1.0.jar      [get] Not modified - so not downloadedivy-probe-antlib:ivy-init-antlib:ivy-init:[ivy:configure] :: Ivy 2.1.0 - 20090925235825 :: http://ant.apache.org/ivy/ ::[ivy:configure] :: loading settings :: file = /home/hadoop/下載/hadoop2x-eclipse-plugin-master/ivy/ivysettings.xmlivy-resolve-common:ivy-retrieve-common:[ivy:cachepath] DEPRECATED: ‘ivy.conf.file‘ is deprecated, use ‘ivy.settings.file‘ instead[ivy:cachepath] :: loading settings :: file = /home/hadoop/下載/hadoop2x-eclipse-plugin-master/ivy/ivysettings.xmlcompile:     [echo] contrib: eclipse-plugin    [javac] /home/hadoop/下載/hadoop2x-eclipse-plugin-master/src/contrib/eclipse-plugin/build.xml:76: warning: ‘includeantruntime‘ was not set, defaulting to build.sysclasspath=last; set to false for repeatable builds    [javac] Compiling 45 source files to /home/hadoop/下載/hadoop2x-eclipse-plugin-master/build/contrib/eclipse-plugin/classes    [javac] /usr/local/hadoop/hadoop-2.2.0/share/hadoop/common/hadoop-common-2.2.0.jar(org/apache/hadoop/fs/Path.class): 警告: 無法找到類型 ‘LimitedPrivate‘ 的注釋方法 ‘value()‘: 找不到org.apache.hadoop.classification.InterfaceAudience的類檔案    [javac] /usr/local/hadoop/hadoop-2.2.0/share/hadoop/hdfs/hadoop-hdfs-2.2.0.jar(org/apache/hadoop/hdfs/DistributedFileSystem.class): 警告: 無法找到類型 ‘LimitedPrivate‘ 的注釋方法 ‘value()‘    [javac] /usr/local/hadoop/hadoop-2.2.0/share/hadoop/common/hadoop-common-2.2.0.jar(org/apache/hadoop/fs/FileSystem.class): 警告: 無法找到類型 ‘LimitedPrivate‘ 的注釋方法 ‘value()‘    [javac] /usr/local/hadoop/hadoop-2.2.0/share/hadoop/common/hadoop-common-2.2.0.jar(org/apache/hadoop/fs/FileSystem.class): 警告: 無法找到類型 ‘LimitedPrivate‘ 的注釋方法 ‘value()‘    [javac] /usr/local/hadoop/hadoop-2.2.0/share/hadoop/common/hadoop-common-2.2.0.jar(org/apache/hadoop/fs/FileSystem.class): 警告: 無法找到類型 ‘LimitedPrivate‘ 的注釋方法 ‘value()‘    [javac] /usr/local/hadoop/hadoop-2.2.0/share/hadoop/common/hadoop-common-2.2.0.jar(org/apache/hadoop/fs/FSDataInputStream.class): 警告: 無法找到類型 ‘LimitedPrivate‘ 的注釋方法 ‘value()‘    [javac] /usr/local/hadoop/hadoop-2.2.0/share/hadoop/common/hadoop-common-2.2.0.jar(org/apache/hadoop/fs/FSDataOutputStream.class): 警告: 無法找到類型 ‘LimitedPrivate‘ 的注釋方法 ‘value()‘    [javac] 注: 某些輸入檔案使用或覆蓋了已淘汰的 API。    [javac] 注: 有關詳細資料, 請使用 -Xlint:deprecation 重新編譯。    [javac] 注: 某些輸入檔案使用了未經檢查或不安全的操作。    [javac] 注: 有關詳細資料, 請使用 -Xlint:unchecked 重新編譯。    [javac] 7 個警告jar:    [mkdir] Created dir: /home/hadoop/下載/hadoop2x-eclipse-plugin-master/build/contrib/eclipse-plugin/lib     [copy] Copying 9 files to /home/hadoop/下載/hadoop2x-eclipse-plugin-master/build/contrib/eclipse-plugin/lib     [copy] Copying /usr/local/hadoop/hadoop-2.2.0/share/hadoop/mapreduce/hadoop-mapreduce-client-app-2.2.0.jar to /home/hadoop/下載/hadoop2x-eclipse-plugin-master/build/contrib/eclipse-plugin/lib/hadoop-mapreduce-client-app-2.2.0.jar     [copy] Copying /usr/local/hadoop/hadoop-2.2.0/share/hadoop/mapreduce/hadoop-mapreduce-client-common-2.2.0.jar to /home/hadoop/下載/hadoop2x-eclipse-plugin-master/build/contrib/eclipse-plugin/lib/hadoop-mapreduce-client-common-2.2.0.jar     [copy] Copying /usr/local/hadoop/hadoop-2.2.0/share/hadoop/mapreduce/hadoop-mapreduce-client-core-2.2.0.jar to /home/hadoop/下載/hadoop2x-eclipse-plugin-master/build/contrib/eclipse-plugin/lib/hadoop-mapreduce-client-core-2.2.0.jar     [copy] Copying /usr/local/hadoop/hadoop-2.2.0/share/hadoop/mapreduce/hadoop-mapreduce-client-hs-2.2.0.jar to /home/hadoop/下載/hadoop2x-eclipse-plugin-master/build/contrib/eclipse-plugin/lib/hadoop-mapreduce-client-hs-2.2.0.jar     [copy] Copying /usr/local/hadoop/hadoop-2.2.0/share/hadoop/mapreduce/hadoop-mapreduce-client-hs-plugins-2.2.0.jar to /home/hadoop/下載/hadoop2x-eclipse-plugin-master/build/contrib/eclipse-plugin/lib/hadoop-mapreduce-client-hs-plugins-2.2.0.jar     [copy] Copying /usr/local/hadoop/hadoop-2.2.0/share/hadoop/mapreduce/hadoop-mapreduce-client-jobclient-2.2.0-tests.jar to /home/hadoop/下載/hadoop2x-eclipse-plugin-master/build/contrib/eclipse-plugin/lib/hadoop-mapreduce-client-jobclient-2.2.0-tests.jar     [copy] Copying /usr/local/hadoop/hadoop-2.2.0/share/hadoop/mapreduce/hadoop-mapreduce-client-jobclient-2.2.0.jar to /home/hadoop/下載/hadoop2x-eclipse-plugin-master/build/contrib/eclipse-plugin/lib/hadoop-mapreduce-client-jobclient-2.2.0.jar     [copy] Copying /usr/local/hadoop/hadoop-2.2.0/share/hadoop/mapreduce/hadoop-mapreduce-client-shuffle-2.2.0.jar to /home/hadoop/下載/hadoop2x-eclipse-plugin-master/build/contrib/eclipse-plugin/lib/hadoop-mapreduce-client-shuffle-2.2.0.jar     [copy] Copying /usr/local/hadoop/hadoop-2.2.0/share/hadoop/mapreduce/hadoop-mapreduce-examples-2.2.0.jar to /home/hadoop/下載/hadoop2x-eclipse-plugin-master/build/contrib/eclipse-plugin/lib/hadoop-mapreduce-examples-2.2.0.jar     [copy] Copying 3 files to /home/hadoop/下載/hadoop2x-eclipse-plugin-master/build/contrib/eclipse-plugin/lib     [copy] Copying /usr/local/hadoop/hadoop-2.2.0/share/hadoop/common/hadoop-common-2.2.0-tests.jar to /home/hadoop/下載/hadoop2x-eclipse-plugin-master/build/contrib/eclipse-plugin/lib/hadoop-common-2.2.0-tests.jar     [copy] Copying /usr/local/hadoop/hadoop-2.2.0/share/hadoop/common/hadoop-common-2.2.0.jar to /home/hadoop/下載/hadoop2x-eclipse-plugin-master/build/contrib/eclipse-plugin/lib/hadoop-common-2.2.0.jar     [copy] Copying /usr/local/hadoop/hadoop-2.2.0/share/hadoop/common/hadoop-nfs-2.2.0.jar to /home/hadoop/下載/hadoop2x-eclipse-plugin-master/build/contrib/eclipse-plugin/lib/hadoop-nfs-2.2.0.jar     [copy] Copying 3 files to /home/hadoop/下載/hadoop2x-eclipse-plugin-master/build/contrib/eclipse-plugin/lib     [copy] Copying /usr/local/hadoop/hadoop-2.2.0/share/hadoop/hdfs/hadoop-hdfs-2.2.0-tests.jar to /home/hadoop/下載/hadoop2x-eclipse-plugin-master/build/contrib/eclipse-plugin/lib/hadoop-hdfs-2.2.0-tests.jar     [copy] Copying /usr/local/hadoop/hadoop-2.2.0/share/hadoop/hdfs/hadoop-hdfs-2.2.0.jar to /home/hadoop/下載/hadoop2x-eclipse-plugin-master/build/contrib/eclipse-plugin/lib/hadoop-hdfs-2.2.0.jar     [copy] Copying /usr/local/hadoop/hadoop-2.2.0/share/hadoop/hdfs/hadoop-hdfs-nfs-2.2.0.jar to /home/hadoop/下載/hadoop2x-eclipse-plugin-master/build/contrib/eclipse-plugin/lib/hadoop-hdfs-nfs-2.2.0.jar     [copy] Copying 11 files to /home/hadoop/下載/hadoop2x-eclipse-plugin-master/build/contrib/eclipse-plugin/lib     [copy] Copying /usr/local/hadoop/hadoop-2.2.0/share/hadoop/yarn/hadoop-yarn-api-2.2.0.jar to /home/hadoop/下載/hadoop2x-eclipse-plugin-master/build/contrib/eclipse-plugin/lib/hadoop-yarn-api-2.2.0.jar     [copy] Copying /usr/local/hadoop/hadoop-2.2.0/share/hadoop/yarn/hadoop-yarn-applications-distributedshell-2.2.0.jar to /home/hadoop/下載/hadoop2x-eclipse-plugin-master/build/contrib/eclipse-plugin/lib/hadoop-yarn-applications-distributedshell-2.2.0.jar     [copy] Copying /usr/local/hadoop/hadoop-2.2.0/share/hadoop/yarn/hadoop-yarn-applications-unmanaged-am-launcher-2.2.0.jar to /home/hadoop/下載/hadoop2x-eclipse-plugin-master/build/contrib/eclipse-plugin/lib/hadoop-yarn-applications-unmanaged-am-launcher-2.2.0.jar     [copy] Copying /usr/local/hadoop/hadoop-2.2.0/share/hadoop/yarn/hadoop-yarn-client-2.2.0.jar to /home/hadoop/下載/hadoop2x-eclipse-plugin-master/build/contrib/eclipse-plugin/lib/hadoop-yarn-client-2.2.0.jar     [copy] Copying /usr/local/hadoop/hadoop-2.2.0/share/hadoop/yarn/hadoop-yarn-common-2.2.0.jar to /home/hadoop/下載/hadoop2x-eclipse-plugin-master/build/contrib/eclipse-plugin/lib/hadoop-yarn-common-2.2.0.jar     [copy] Copying /usr/local/hadoop/hadoop-2.2.0/share/hadoop/yarn/hadoop-yarn-server-common-2.2.0.jar to /home/hadoop/下載/hadoop2x-eclipse-plugin-master/build/contrib/eclipse-plugin/lib/hadoop-yarn-server-common-2.2.0.jar     [copy] Copying /usr/local/hadoop/hadoop-2.2.0/share/hadoop/yarn/hadoop-yarn-server-nodemanager-2.2.0.jar to /home/hadoop/下載/hadoop2x-eclipse-plugin-master/build/contrib/eclipse-plugin/lib/hadoop-yarn-server-nodemanager-2.2.0.jar     [copy] Copying /usr/local/hadoop/hadoop-2.2.0/share/hadoop/yarn/hadoop-yarn-server-resourcemanager-2.2.0.jar to /home/hadoop/下載/hadoop2x-eclipse-plugin-master/build/contrib/eclipse-plugin/lib/hadoop-yarn-server-resourcemanager-2.2.0.jar     [copy] Copying /usr/local/hadoop/hadoop-2.2.0/share/hadoop/yarn/hadoop-yarn-server-tests-2.2.0.jar to /home/hadoop/下載/hadoop2x-eclipse-plugin-master/build/contrib/eclipse-plugin/lib/hadoop-yarn-server-tests-2.2.0.jar     [copy] Copying /usr/local/hadoop/hadoop-2.2.0/share/hadoop/yarn/hadoop-yarn-server-web-proxy-2.2.0.jar to /home/hadoop/下載/hadoop2x-eclipse-plugin-master/build/contrib/eclipse-plugin/lib/hadoop-yarn-server-web-proxy-2.2.0.jar     [copy] Copying /usr/local/hadoop/hadoop-2.2.0/share/hadoop/yarn/hadoop-yarn-site-2.2.0.jar to /home/hadoop/下載/hadoop2x-eclipse-plugin-master/build/contrib/eclipse-plugin/lib/hadoop-yarn-site-2.2.0.jar     [copy] Copying 1 file to /home/hadoop/下載/hadoop2x-eclipse-plugin-master/build/contrib/eclipse-plugin/lib     [copy] Copying /usr/local/hadoop/hadoop-2.2.0/share/hadoop/common/lib/protobuf-java-2.5.0.jar to /home/hadoop/下載/hadoop2x-eclipse-plugin-master/build/contrib/eclipse-plugin/lib/protobuf-java-2.5.0.jar     [copy] Copying 1 file to /home/hadoop/下載/hadoop2x-eclipse-plugin-master/build/contrib/eclipse-plugin/lib     [copy] Copying /usr/local/hadoop/hadoop-2.2.0/share/hadoop/common/lib/log4j-1.2.17.jar to /home/hadoop/下載/hadoop2x-eclipse-plugin-master/build/contrib/eclipse-plugin/lib/log4j-1.2.17.jar     [copy] Copying 1 file to /home/hadoop/下載/hadoop2x-eclipse-plugin-master/build/contrib/eclipse-plugin/lib     [copy] Copying /usr/local/hadoop/hadoop-2.2.0/share/hadoop/common/lib/commons-cli-1.2.jar to /home/hadoop/下載/hadoop2x-eclipse-plugin-master/build/contrib/eclipse-plugin/lib/commons-cli-1.2.jar     [copy] Copying 1 file to /home/hadoop/下載/hadoop2x-eclipse-plugin-master/build/contrib/eclipse-plugin/lib     [copy] Copying /usr/local/hadoop/hadoop-2.2.0/share/hadoop/common/lib/commons-configuration-1.6.jar to /home/hadoop/下載/hadoop2x-eclipse-plugin-master/build/contrib/eclipse-plugin/lib/commons-configuration-1.6.jar     [copy] Copying 1 file to /home/hadoop/下載/hadoop2x-eclipse-plugin-master/build/contrib/eclipse-plugin/lib     [copy] Copying /usr/local/hadoop/hadoop-2.2.0/share/hadoop/common/lib/commons-lang-2.5.jar to /home/hadoop/下載/hadoop2x-eclipse-plugin-master/build/contrib/eclipse-plugin/lib/commons-lang-2.5.jar     [copy] Copying 1 file to /home/hadoop/下載/hadoop2x-eclipse-plugin-master/build/contrib/eclipse-plugin/lib     [copy] Copying /usr/local/hadoop/hadoop-2.2.0/share/hadoop/common/lib/jackson-core-asl-1.8.8.jar to /home/hadoop/下載/hadoop2x-eclipse-plugin-master/build/contrib/eclipse-plugin/lib/jackson-core-asl-1.8.8.jar     [copy] Copying 1 file to /home/hadoop/下載/hadoop2x-eclipse-plugin-master/build/contrib/eclipse-plugin/lib     [copy] Copying /usr/local/hadoop/hadoop-2.2.0/share/hadoop/common/lib/jackson-mapper-asl-1.8.8.jar to /home/hadoop/下載/hadoop2x-eclipse-plugin-master/build/contrib/eclipse-plugin/lib/jackson-mapper-asl-1.8.8.jar     [copy] Copying 1 file to /home/hadoop/下載/hadoop2x-eclipse-plugin-master/build/contrib/eclipse-plugin/lib     [copy] Copying /usr/local/hadoop/hadoop-2.2.0/share/hadoop/common/lib/slf4j-log4j12-1.7.5.jar to /home/hadoop/下載/hadoop2x-eclipse-plugin-master/build/contrib/eclipse-plugin/lib/slf4j-log4j12-1.7.5.jar     [copy] Copying 1 file to /home/hadoop/下載/hadoop2x-eclipse-plugin-master/build/contrib/eclipse-plugin/lib     [copy] Copying /usr/local/hadoop/hadoop-2.2.0/share/hadoop/common/lib/slf4j-api-1.7.5.jar to /home/hadoop/下載/hadoop2x-eclipse-plugin-master/build/contrib/eclipse-plugin/lib/slf4j-api-1.7.5.jar     [copy] Copying 1 file to /home/hadoop/下載/hadoop2x-eclipse-plugin-master/build/contrib/eclipse-plugin/lib     [copy] Copying /usr/local/hadoop/hadoop-2.2.0/share/hadoop/common/lib/guava-11.0.2.jar to /home/hadoop/下載/hadoop2x-eclipse-plugin-master/build/contrib/eclipse-plugin/lib/guava-11.0.2.jar     [copy] Copying 1 file to /home/hadoop/下載/hadoop2x-eclipse-plugin-master/build/contrib/eclipse-plugin/lib     [copy] Copying /usr/local/hadoop/hadoop-2.2.0/share/hadoop/common/lib/hadoop-auth-2.2.0.jar to /home/hadoop/下載/hadoop2x-eclipse-plugin-master/build/contrib/eclipse-plugin/lib/hadoop-auth-2.2.0.jar     [copy] Copying 1 file to /home/hadoop/下載/hadoop2x-eclipse-plugin-master/build/contrib/eclipse-plugin/lib     [copy] Copying /usr/local/hadoop/hadoop-2.2.0/share/hadoop/common/lib/netty-3.6.2.Final.jar to /home/hadoop/下載/hadoop2x-eclipse-plugin-master/build/contrib/eclipse-plugin/lib/netty-3.6.2.Final.jar      [jar] Building jar: /home/hadoop/下載/hadoop2x-eclipse-plugin-master/build/contrib/eclipse-plugin/hadoop-eclipse-plugin-2.2.0.jarBUILD SUCCESSFULTotal time: 36 seconds

ivy-resolve-common的位置可能會等待一段時間。之後將產生的hadoop-eclipse-plugin-2.2.0.jar拷貝到eclipse的plugins檔案夾下，重啟eclipse，看見Hdoop外掛程式已經安裝成功

上文記錄自己的安裝過程，如果有所遺漏或錯誤，不吝賜教

Eclipse安裝Hadoop外掛程式