Hadoop Learning Note (iii)--win7+eclipse+hadoop2.5.2 deployment

For the most part of the night. The final deployment succeeded, slightly more complex than on Linux, such as the following:

1) JDK, ant, Hadoop environment variable configuration

2) Extract hadoop-2.5.2.tar.gz, hadoop-2.5.2-src.tar.gz, Hadoop2x-eclipse-plugin, Hadoop-common-2.2.0-bin and download to d:\ respectively Under the Profession\hadoop folder

3) Change Hadoop-eclipse-plugin-2.5.2.jar configuration

Change D:\profession\hadoop\hadoop2x-eclipse-plugin-master\ivy\libraries.properties

hadoop.version=2.5.2

jackson.version=1.9.13

4) Ant compiler Hadoop-eclipse-plugin-2.5.2.jar

D:\profession\hadoop\hadoop2x-eclipse-plugin-master\src\contrib\eclipse-plugin>
Ant Jar-dversion=2.5.2-declipse.home= d:\profession\eclipse-dhadoop.home= D:\profession\hadoop\hadoop-2.5.2

After compiling, Hadoop-eclipse-plugin-2.5.2.jar will be in D:\profession\hadoop\hadoop2x-eclipse-plugin-master\build\contrib\. Under the Eclipse-plugin folder

(Note: A warning message indicates that the jar package is missing.) Import is possible)

5) Copy the compiled Hadoop-eclipse-plugin-2.5.2.jar to the Eclipse's plugins folder. Then restart Eclipse.

(Note: If you see DFS Locations in Project on the left, that means Eclipse is recognized successfully, some Eclipse version numbers must be recognized under the Java EE version number)

6) Window--preference--hadoop Map/reduce is configured,

7) Display the Hadoop connection configuration form: Window--showview--other-mapreduce Tools

8) Configure the connection to Hadoop, for example, as seen in:

9) Click on the left side of Dfs Locations. If you can see the user, the installation is successful

The Hadoop cluster environment joins the test file, using the Input.txt file above

11) New Item File--new--other--map/reduce project named MR1,

Then create the Org.apache.hadoop.examples.WordCount. Copy overwrite from HADOOP-2.5.2-SRC (\hadoop-2.5.2-src\hadoop-mapreduce-project\hadoop-mapreduce-examples\src\main\java\org\ Apache\hadoop\examples)

12) Configure the execution environment under Windows.

Need Hadoop.dll,winutils.exe
Directly copy the D:\profession\hadoop\hadoop-common-2.2.0-bin-master\bin folder under this 2 items overlay D:\profession\hadoop\hadoop-2.5.2\bin

13) configuration of the execution parameters

Click Wordcount.java. Right-click Runas->run configurations, configure execution parameters, input and output directories

hdfs://hadoop:9000/user/hadoop/inputhdfs://hadoop:9000/user/hadoop/output02

14) Execution procedure

Run as application/run as Hadoop

15) Error Collection

1. Q:an internal error occurred during: "Map/reduce locationstatus Updater".

Java.lang.NullPointerException

A: Switch eclipse to Java EE version number to

2. Q:warn HDFs. Dfsclient:datastreamer exception:org.apache.hadoop.ipc.remoteexception:java.io.ioexception:file/user/root/ INPUT01/FILE01 could only be replicated to0 nodes, instead of 1

A: This is because there is no reason to join the node. That Datanode did not start successfully

3. Q:java.io.ioexception:cannot Run Program "D:/java" (indirectory "c:/users/administrator/.net.sf.jadclipse/ 1372683520891 "):

A:jadclipse not Configured, download Jad.exe,windows->preferences, Java-jadclipse option, write todecomplier in path d:/java/: text box Jad.exe (the path of the previously downloaded Jad.exe)

4. Q:JAVA.LANG.UNSATISFIEDLINKERROR:ORG.APACHE.HADOOP.IO.NATIVEIO.NATIVEIO$WINDOWS.ACCESS0 (Ljava/lang/String;I)

A:hadoop-common-2.2.0-bin-master The bin under the Hadoop.dll C:\Windows\System32, and then restart the computer, perhaps not so simple, or this problem occurs. Windows is the only way to check the current process request, in a given path access permission, so we first give the ability to visit, we first change the source code, return True to agree to visit. We download the corresponding Hadoop source code, hadoop-2.5.2-src.tar.gz decompression, hadoop-2.5.2-src\hadoop-common-project\hadoop-common\src\main\java\ Org\apache\hadoop\io\nativeio under Nativeio.java Copy to the corresponding Eclipse project, and then change the 570 behavior to return true to see:

16) Impressions

Encounter abnormal problems, not impatient, careful analysis, assuming the time to agree, can once again or temporarily to do something else, perhaps suddenly you know where you are wrong

Hadoop Learning Note (iii)--win7+eclipse+hadoop2.5.2 deployment