Hadoop Learning Note (iii)--win7+eclipse+hadoop2.5.2 deployment

The last half of the night. The deployment succeeded, slightly more complicated than on Linux, with the following steps:

1) JDK, ant, Hadoop environment variable configuration

2) Extract hadoop-2.5.2.tar.gz, hadoop-2.5.2-src.tar.gz, Hadoop2x-eclipse-plugin, Hadoop-common-2.2.0-bin and download to d:\ respectively Under the Profession\hadoop directory

3) Modify the Hadoop-eclipse-plugin-2.5.2.jar configuration

Modify D:\profession\hadoop\hadoop2x-eclipse-plugin-master\ivy\libraries.properties

hadoop.version=2.5.2

jackson.version=1.9.13

4) Ant compiler Hadoop-eclipse-plugin-2.5.2.jar

D:\profession\hadoop\hadoop2x-eclipse-plugin-master\src\contrib\eclipse-plugin>
Ant Jar-dversion=2.5.2-declipse.home= d:\profession\eclipse-dhadoop.home= D:\profession\hadoop\hadoop-2.5.2

After compiling, Hadoop-eclipse-plugin-2.5.2.jar will be in D:\profession\hadoop\hadoop2x-eclipse-plugin-master\build\contrib\. Under the Eclipse-plugin directory

(Note: There will be a warning message indicating that the jar package is missing and can be imported)

5) Copy the compiled Hadoop-eclipse-plugin-2.5.2.jar to Eclipse's plugins directory and restart Eclipse.

(Note: If you see DFS Locations in Project on the left, that means Eclipse is recognized successfully, some versions of Eclipse must be recognized under the Java EE version)

6) Window--preference--hadoop Map/reduce is configured,

7) Display Hadoop Connection Configuration window: Window--showview--other-mapreduce Tools

8) Configure the connection to Hadoop as shown in:

9) Click on the Dfs Locations on the left, if you can see the user, the installation is successful

Hadoop cluster environment to add test files using the Input.txt file above

11) New Item File--new--other--map/reduce project named MR1,

Then create the Org.apache.hadoop.examples.WordCount, copy the overlay from the HADOOP-2.5.2-SRC (\hadoop-2.5.2-src\hadoop-mapreduce-project\ Hadoop-mapreduce-examples\src\main\java\org\apache\hadoop\examples)

12) Configure the operating environment under Windows,

Need Hadoop.dll,winutils.exe
Direct copy of the D:\profession\hadoop\hadoop-common-2.2.0-bin-master\bin directory this 2 items overwrite D:\profession\hadoop\hadoop-2.5.2\bin

13) Configure the operating parameters

Click Wordcount.java, right-click, runas->run configurations, configure run parameters, i.e. input and output folders

hdfs://hadoop:9000/user/hadoop/inputhdfs://hadoop:9000/user/hadoop/output02

14) Running the program

Run as application/run as Hadoop

15) Error Collection

1. Q:an internal error occurred during: "Map/reduce locationstatus Updater".

Java.lang.NullPointerException

A: Switch eclipse to Java EE version

2. Q:warn HDFs. Dfsclient:datastreamer exception:org.apache.hadoop.ipc.remoteexception:java.io.ioexception:file/user/root/ INPUT01/FILE01 could only be replicated to0 nodes, instead of 1

A: This is because the node was not added because the Datanode did not start successfully

3. Q:java.io.ioexception:cannot Run Program "D:/java" (indirectory "c:/users/administrator/.net.sf.jadclipse/ 1372683520891 "):

A:jadclipse not Configured, download Jad.exe,windows->preferences, Java-jadclipse option, write todecomplier in path d:/java/: text box Jad.exe (the path of the previously downloaded Jad.exe)

4. Q:JAVA.LANG.UNSATISFIEDLINKERROR:ORG.APACHE.HADOOP.IO.NATIVEIO.NATIVEIO$WINDOWS.ACCESS0 (Ljava/lang/String;I)

A:hadoop-common-2.2.0-bin-master under the bin Hadoop.dll put under C:\Windows\System32, and then restart the computer, perhaps not so simple, or there is such a problem. Windows is the only way to check the current process of the request, in the given path of access, so we give access first, we first modify the source code, return True to allow access. We download the corresponding Hadoop source code, hadoop-2.5.2-src.tar.gz decompression, hadoop-2.5.2-src\hadoop-common-project\hadoop-common\src\main\java\ Org\apache\hadoop\io\nativeio under Nativeio.java Copy to the corresponding eclipse of project, and then modify the 570 behavior to return true:

16) Impressions

Encounter abnormal problems, not impatient, careful analysis, if time permitting, can be re-smoothed or temporarily to do something else, perhaps suddenly you know where you are wrong

Hadoop Learning Note (iii)--win7+eclipse+hadoop2.5.2 deployment