Environment
Windows 7 x64 bit, Visual Studio Professional
Hadoop Source Version 2.2.0
Step (from the book "Pro Apache Hadoop, Second Edition" slightly modified.
- Ensure that JDK, 1.6 is, or higher is installed. We assume that it's installed in thec:/myapps/jdkl6/ folder, which should has a bin subfolder.
Download the hadoop-2.2.x-src.tar.gz files (2.2.0 at the time of this writing) from the Download section of the A Pache Web site for Hadoop. The download link is as follows:http://www.apache.org/dist/hadoop/core/hadoop-2.2.0/hadoop-2.2.0-src.tar.gz.
Explode the tar file into a directory. For the purpose of this section, we assume the directory is calledc:/myapps/hadoop/.
If using Visual Studio, use Visual Studio-Professional (not 2012). Do not use the Visual Studio Express because it does not the compiling for 64-bit, which presents problems if running on a 64-bit System. Alternatively, download the Microsoft Windows SDK v7.1. At the time of writing, the link is the following:
http://www.microsoft.com/en-us/download/confirmation.aspx?id=8279.
Place the SDK in the folder with full privileges. For this section, we assume the folder isc:/myapps/winsdk71/.
Download and install the Linux-like environment for Windows named Cygwin. We assume that Cygwin home folder isc:/myapps/cygwin/. Ensure that the following UNIX command-line tools is installed:sh, mkdir, RM, CP, Tar, and gzip. The download location for Cygwin ishttp://cygwin.com/install.html.
Download and install Maven 3.1.1 or a compatible version. Installation consists of simply exploding thetar bundle into a Windows folder. We assume the home folder for Maven download isc:/myapps/apache-maven-3.1.1/. Verify that this folder has the bin subfolder.
Download and install Protocol buffers 2.5.0. At the time of writing, the link is http://protobuf.googlecode.com/files/protoc-2.5.0-win32.zip.
Simply unzip the zip file into a folder (assumed to be c:/myapps/protobuf/). Verify the installation is correct by ensuring the folder contains theprotoc.exe file.
Install ZLIB from Http://zlib.net/zlib128-dll.zip and explode the zip file in the c:/zlib-1.2.8 folder. At runtime, this folder must is accessible from thePATH variable.
ADD the environment variables shown in Table A-1.
Table A-1: Environment Variables for Windows
Environment Variable |
Value |
Java_home |
c:/myapps/jdk16/ |
M2_home |
C:/myapps/apache-maven-3.1.1/ |
Platform |
x64 (or Win32 when building on the 32-bit system) |
Cygwin_home |
c:/myapps/cygwin/ |
Protobuf_home |
c:/myapps/protobuf/ |
Zlib_home |
C :/zlib-1.2.8 12. Add the following to the Windows PATH variable:
%java_home%/bin/
%cygwin_ home%/bin/
%m2_home%/bin/
%protobuf_home%
%zlib_home
|
To start compiling, perform the following steps:
Hadoop 2.2.0 has a bug at compile time, refer to https://issues.apache.org/jira/browse/HADOOP-10110 description, then download https://issues.apache.org/ Jira/secure/attachment/12614482/hadoop-10110.patch This patch, modify it according to this patch hadoop-common-project/hadoop-auth/ Pom.xml the file, and then compile it, or it will go wrong.
Microsoft Windows SDK v7.1 and start the Windows SDK 7.1 command prompt in Run as Administrator mode.
C:/myapps/hadoop. Recall that's the folder in which, the Hadoop source was placed.
Execute the following command MVN package with the -pdist,native-win-dskiptests-dtar options. The exact command is this:
MVN Package-pdist,native-win-dskiptests–dtar
If There is no errors, the hadoop-2.2.0.tar.gz file was created in the c:/myapps/hadoop/hadoop-dist/hadoop-2 .2.0/ folder.
After compiling, if there are no errors, the hadoop-2.2.0.tar.gz file will be generated under the Hadoop-2.2.0-src\hadoop-dist\target directory.
The first step is done!
Getting started with Hadoop-compiling x64-bit Hadoop on Windows