Hadoop: Windows 7 32 Bit Compile and Run

Source: Internet
Author: User
Keywords Compile run source code
Tags .gz apache code compile configuration content debug directory
Required tools

1.Windows 7 32 Bit OS (you know)

2. Apache Hadoop 2.2.0-bin (hadoop-2.2.0.tar.gz)

3. Apache Hadoop 2.2.0-src (hadoop-2.2.0-src.tar.gz)

3.JDK 1.7

4.Maven 3.2.1 (apache-maven-3.2.1-bin.zip)

5.Protocol Buffers 2.5.0

6.Unix command-line tool Cygwin (Setup-x86.exe)

7.Microsoft Windows SDK v7.1

Apache Hadoop 2.X recommended operating environment is 64Bit machine, because the actual operation of more than 4GB of memory it! 32Bit suitable for personal development and debugging learning to use.

Environment configuration

Preface: Since the Apache Hadoop 2.2.0 distribution uses the dynamic link library (bin \ hadoop.dll, libwinutils.lib, winutils.exe) on Windows 64bit, you need to recompile it to 32bit using the source code Back to the hadoop-2.2.0.tar.gz distribution does not have these Windows dynamic library). If your system disk is SSD, it is recommended to install the C root directory.

Environment variables

(Tools: 3, 4, 5, 6, 7) The installation process is no longer detailed, all the way Next on the line (JavaSE recommended root directory: c: \ java). Focus on the configuration of the environment variable, add the following environment variable to the "System Variables" (Java path can not have spaces):

JAVA_HOME = C: \ Java \ jdk1.7.0_45

Platform = Win32

M2_HOME = C: \ apache-maven-3.2.1

Path =; C: \ cygwin \ bin; C: \ apache-maven-3.2.1 \ bin; C: \ protoc-2.5.0-win32;

Configuration example (do not forget to set Path Oh):

Unzip the source code

Hadoop-2.2.0-src.tar.gz source code to the root directory of D, it looks like the following path: D: \ hadoop-2.2.0 \

Apache Hadoop svn Codebase Address: http://svn.apache.org/repos/asf/hadoop/common/tags/release-2.2.0

Need to manually correct the source code of several compilation errors:

First place:

Modify the file: \ hadoop-common-project \ hadoop-auth \ pom.xml

Modification: Add an Xml configuration node at about 56 lines.

<dependency>
<groupId> org.mortbay.jetty </ groupId>
<artifactId> jetty-util </ artifactId>
<scope> test </ scope>
</ dependency>

Modify the example:

Second place:

Modify the file: hadoop-common-project \ hadoop-common \ src \ main \ native \ native.sln

Modify content: Use Notepad to open the file.

Replacement content:

GlobalSection (ProjectConfigurationPlatforms) = postSolution
{4C0C12D2-3CB0-47F8-BCD0-55BD5732DFA7} .Debug | Mixed Platforms.ActiveCfg = Release | x64
{4C0C12D2-3CB0-47F8-BCD0-55BD5732DFA7} .Debug | Mixed Platforms.Build.0 = Release | x64
{4C0C12D2-3CB0-47F8-BCD0-55BD5732DFA7} .Debug | Win32.ActiveCfg = Release | x64
{4C0C12D2-3CB0-47F8-BCD0-55BD5732DFA7} .Debug | Win32.Build.0 = Release | x64
{4C0C12D2-3CB0-47F8-BCD0-55BD5732DFA7} .Debug | x64.ActiveCfg = Release | x64
{4C0C12D2-3CB0-47F8-BCD0-55BD5732DFA7} .Debug | x64.Build.0 = Release | x64
{4C0C12D2-3CB0-47F8-BCD0-55BD5732DFA7}. Release | Mixed Platforms.ActiveCfg = Release | x64
{4C0C12D2-3CB0-47F8-BCD0-55BD5732DFA7}. Release | Mixed Platforms.Build.0 = Release | x64
{4C0C12D2-3CB0-47F8-BCD0-55BD5732DFA7}. Release | Win32.ActiveCfg = Release | x64
{4C0C12D2-3CB0-47F8-BCD0-55BD5732DFA7}. Release | Win32.Build.0 = Release | x64
{4C0C12D2-3CB0-47F8-BCD0-55BD5732DFA7}. Release | x64.ActiveCfg = Release | x64
{4C0C12D2-3CB0-47F8-BCD0-55BD5732DFA7}. Release | x64.Build.0 = Release | x64
EndGlobalSection

New content:

GlobalSection (ProjectConfigurationPlatforms) = postSolution
{4C0C12D2-3CB0-47F8-BCD0-55BD5732DFA7} .Debug | Win32.ActiveCfg = Release | Win32
{4C0C12D2-3CB0-47F8-BCD0-55BD5732DFA7} .Debug | Win32.Build.0 = Release | Win32
{4C0C12D2-3CB0-47F8-BCD0-55BD5732DFA7} .Debug | x64.ActiveCfg = Release | x64
{4C0C12D2-3CB0-47F8-BCD0-55BD5732DFA7} .Debug | x64.Build.0 = Release | x64
{4C0C12D2-3CB0-47F8-BCD0-55BD5732DFA7}. Release | Win32.ActiveCfg = Release | Win32
{4C0C12D2-3CB0-47F8-BCD0-55BD5732DFA7}. Release | Win32.Build.0 = Release | Win32
{4C0C12D2-3CB0-47F8-BCD0-55BD5732DFA7}. Release | x64.ActiveCfg = Release | x64
{4C0C12D2-3CB0-47F8-BCD0-55BD5732DFA7}. Release | x64.Build.0 = Release | x64
EndGlobalSection

Modify the example:

Third place:

Modify the file: hadoop-common-project \ hadoop-common \ src \ main \ native \ native.vcxproj

Modify the content:

Find Replace "Release | x64" to "Release | Win32"

Look for the replacement "<Platform> x64 </ Platform>" for "<Platform> Win32 </ Platform>"

Modify the example:

Fourth place:

Right-click the "D: \ hadoop-2.2.0" folder, select "administrator to obtain ownership." Otherwise, the compilation process may occur "refused to access" error (right button does not display the menu, find your own online registry changes).

Compile process

Open the "Start" - "All Programs" - "Microsoft Windows SDK v7.1" - "Windows SDK 7.1 Command Prompt" into the command line VC + + tools (must enter here to successfully compile the Hadoop source code, Remember to run as an administrator). The order is as follows:

Switch to the source root directory, execute the compile command: mvn package -Pdist, native-win -DskipTests -Dtar

Example results of the operation

Setting SDK environment relative to C: \ Program Files \ Microsoft SDKs \ Windows \ v7.1
\.
Targeting Windows 7 x86 Debug

C: \ Windows \ system32> d:

D: \> cd D: \ hadoop-2.2.0

D: \ hadoop-2.2.0> mvn package -Pdist, native-win -DskipTests -Dtar

................................ 2.2.1 version Compile completed about 16 minutes or so

................................ 2.4.0 version compilation completed about 22 minutes or so (211.html "> feel Slow translation process: yourself configure the mirror server)

Configure the operation
Related Article

Contact Us

The content source of this page is from Internet, which doesn't represent Alibaba Cloud's opinion; products and services mentioned on that page don't have any relationship with Alibaba Cloud. If the content of the page makes you feel confusing, please write us an email, we will handle the problem within 5 days after receiving your email.

If you find any instances of plagiarism from the community, please send an email to: info-contact@alibabacloud.com and provide relevant evidence. A staff member will contact you within 5 working days.

A Free Trial That Lets You Build Big!

Start building with 50+ products and up to 12 months usage for Elastic Compute Service

  • Sales Support

    1 on 1 presale consultation

  • After-Sales Support

    24/7 Technical Support 6 Free Tickets per Quarter Faster Response

  • Alibaba Cloud offers highly flexible support services tailored to meet your exact needs.