Hadoop: Windows 7 32 Bit Compile and Run

1.Windows 7 32 Bit OS (you know)

2. Apache Hadoop 2.2.0-bin (hadoop-2.2.0.tar.gz)

3. Apache Hadoop 2.2.0-src (hadoop-2.2.0-src.tar.gz)

3.JDK 1.7

4.Maven 3.2.1 (apache-maven-3.2.1-bin.zip)

5.Protocol Buffers 2.5.0

6.Unix command-line tool Cygwin (Setup-x86.exe)

7.Microsoft Windows SDK v7.1

Apache Hadoop 2.X recommended operating environment is 64Bit machine, because the actual operation of more than 4GB of memory it! 32Bit suitable for personal development and debugging learning to use.

Preface: Since the Apache Hadoop 2.2.0 distribution uses the dynamic link library (bin \ hadoop.dll, libwinutils.lib, winutils.exe) on Windows 64bit, you need to recompile it to 32bit using the source code Back to the hadoop-2.2.0.tar.gz distribution does not have these Windows dynamic library). If your system disk is SSD, it is recommended to install the C root directory.

(Tools: 3, 4, 5, 6, 7) The installation process is no longer detailed, all the way Next on the line (JavaSE recommended root directory: c: \ java). Focus on the configuration of the environment variable, add the following environment variable to the "System Variables" (Java path can not have spaces):

JAVA_HOME = C: \ Java \ jdk1.7.0_45

Platform = Win32

M2_HOME = C: \ apache-maven-3.2.1

Path =; C: \ cygwin \ bin; C: \ apache-maven-3.2.1 \ bin; C: \ protoc-2.5.0-win32;

Configuration example (do not forget to set Path Oh):

Hadoop-2.2.0-src.tar.gz source code to the root directory of D, it looks like the following path: D: \ hadoop-2.2.0 \

Apache Hadoop svn Codebase Address: http://svn.apache.org/repos/asf/hadoop/common/tags/release-2.2.0

Need to manually correct the source code of several compilation errors:

Modify the file: \ hadoop-common-project \ hadoop-auth \ pom.xml

Modification: Add an Xml configuration node at about 56 lines.

<dependency>
<groupId> org.mortbay.jetty </ groupId>
<artifactId> jetty-util </ artifactId>
<scope> test </ scope>
</ dependency>

Modify the example:

Modify the file: hadoop-common-project \ hadoop-common \ src \ main \ native \ native.sln

Modify content: Use Notepad to open the file.

Replacement content:

GlobalSection (ProjectConfigurationPlatforms) = postSolution
{4C0C12D2-3CB0-47F8-BCD0-55BD5732DFA7} .Debug | Mixed Platforms.ActiveCfg = Release | x64
{4C0C12D2-3CB0-47F8-BCD0-55BD5732DFA7} .Debug | Mixed Platforms.Build.0 = Release | x64
{4C0C12D2-3CB0-47F8-BCD0-55BD5732DFA7} .Debug | Win32.ActiveCfg = Release | x64
{4C0C12D2-3CB0-47F8-BCD0-55BD5732DFA7} .Debug | Win32.Build.0 = Release | x64
{4C0C12D2-3CB0-47F8-BCD0-55BD5732DFA7} .Debug | x64.ActiveCfg = Release | x64
{4C0C12D2-3CB0-47F8-BCD0-55BD5732DFA7} .Debug | x64.Build.0 = Release | x64
{4C0C12D2-3CB0-47F8-BCD0-55BD5732DFA7}. Release | Mixed Platforms.ActiveCfg = Release | x64
{4C0C12D2-3CB0-47F8-BCD0-55BD5732DFA7}. Release | Mixed Platforms.Build.0 = Release | x64
{4C0C12D2-3CB0-47F8-BCD0-55BD5732DFA7}. Release | Win32.ActiveCfg = Release | x64
{4C0C12D2-3CB0-47F8-BCD0-55BD5732DFA7}. Release | Win32.Build.0 = Release | x64
{4C0C12D2-3CB0-47F8-BCD0-55BD5732DFA7}. Release | x64.ActiveCfg = Release | x64
{4C0C12D2-3CB0-47F8-BCD0-55BD5732DFA7}. Release | x64.Build.0 = Release | x64
EndGlobalSection

New content:

GlobalSection (ProjectConfigurationPlatforms) = postSolution
{4C0C12D2-3CB0-47F8-BCD0-55BD5732DFA7} .Debug | Win32.ActiveCfg = Release | Win32
{4C0C12D2-3CB0-47F8-BCD0-55BD5732DFA7} .Debug | Win32.Build.0 = Release | Win32
{4C0C12D2-3CB0-47F8-BCD0-55BD5732DFA7} .Debug | x64.ActiveCfg = Release | x64
{4C0C12D2-3CB0-47F8-BCD0-55BD5732DFA7} .Debug | x64.Build.0 = Release | x64
{4C0C12D2-3CB0-47F8-BCD0-55BD5732DFA7}. Release | Win32.ActiveCfg = Release | Win32
{4C0C12D2-3CB0-47F8-BCD0-55BD5732DFA7}. Release | Win32.Build.0 = Release | Win32
{4C0C12D2-3CB0-47F8-BCD0-55BD5732DFA7}. Release | x64.ActiveCfg = Release | x64
{4C0C12D2-3CB0-47F8-BCD0-55BD5732DFA7}. Release | x64.Build.0 = Release | x64
EndGlobalSection

Modify the example:

Modify the file: hadoop-common-project \ hadoop-common \ src \ main \ native \ native.vcxproj

Modify the content:

Find Replace "Release | x64" to "Release | Win32"

Look for the replacement "<Platform> x64 </ Platform>" for "<Platform> Win32 </ Platform>"

Modify the example:

Right-click the "D: \ hadoop-2.2.0" folder, select "administrator to obtain ownership." Otherwise, the compilation process may occur "refused to access" error (right button does not display the menu, find your own online registry changes).

Open the "Start" - "All Programs" - "Microsoft Windows SDK v7.1" - "Windows SDK 7.1 Command Prompt" into the command line VC + + tools (must enter here to successfully compile the Hadoop source code, Remember to run as an administrator). The order is as follows:

Switch to the source root directory, execute the compile command: mvn package -Pdist, native-win -DskipTests -Dtar

Example results of the operation

Setting SDK environment relative to C: \ Program Files \ Microsoft SDKs \ Windows \ v7.1
\.
Targeting Windows 7 x86 Debug

C: \ Windows \ system32> d:

D: \> cd D: \ hadoop-2.2.0

D: \ hadoop-2.2.0> mvn package -Pdist, native-win -DskipTests -Dtar

................................ 2.2.1 version Compile completed about 16 minutes or so

................................ 2.4.0 version compilation completed about 22 minutes or so (211.html "> feel Slow translation process: yourself configure the mirror server)