Recompile Hadoop 2.2.0 in a 64-bit operating system

recompile Hadoop 2.2.0 in a 64-bit operating system

Recently, in order to do some testing, deploy Hadoop 2.2.0 in CentOS6.5. Download the hadoop-2.2.0.tar.gz installation package from the official website, and you will not be able to run the prompt after deployment installation:

WARNutil.NativeCodeLoader:Unable to load Native-hadoop Library for Yourplatform ... using Builtin-java classes where appli Cable error.

View local library by command the libhadoop.so.1.0.0 in the software is 32-bit, while the operating system is 64-bit. So it is very sad to "" "can only compile the Hadoop installation package on its own.

The following command can be used to view the situation of libhadoop.so.1.0.0:

#file ${hadoop_home}/lib/native/libhadoop.so.1.0.0

/hadoop/hadoop-2.2.0/lib/native/libhadoop.so.1.0.0:elf 32-bit LSB Shared Object, Intel 80386,version 1 (SYSV), Dynamically linked, not stripped

WORKAROUND: Recompile the installation package

Make sure the machine can connect to the network, if not connected, download the corresponding installation package and install it manually.

A Download the source program for Hadoop 2.2.0 HADOOP-2.2.0-SRC. tar.gz (recommended)

Or you can download it via SVN: (this way is slower)

# SVN co ' http://svn.apache.org/repos/asf/hadoop/common/tags/release-2.2.0 '

Two Compiled development environment Installation

necessary to have JDK,SVN,CMAKE,NCURSES,OPENSSL,GCC,MAVEN,PROTOBUF

1. #yum Install SVN

#yum Install Autoconfautomake Libtool cmake

#yum Install Openssl-devel

#yum Install Ncurses-devel

#yum Install gcc*

2. Install Maven

Download and unzip maven:http://maven.apache.org/download.cgi

#mvapache-maven-3.1.1/usr/local

Loading/usr/local/mvapache-maven-3.1.1/bin into an environment variable

3. Installing PROTOBUF

From Https://code.google.com/p/protobuf/downloads/list

Download protobuf-2.5.0.tar.gz, unzip the installation package, go to the Protobuf root directory, execute the following command:

#.configure

#make

#make Check

#make Install

#protoc--version

Libprotoc 2.5.0

Three Compiling Hadoop

After installing the above compiled development environment, you can compile Hadoop, go to the root directory of Hadoop, and execute the following command:

$ MVN Package-pdist,native-dskiptests-dtar

The whole compilation time is long, wait patiently ~~~~~~

Finally, the entire compilation situation is indicated:

[INFO] Apache Hadoop mini-cluster ......... ......... SUCCESS [2.151s]

[INFO]------------------------------------------------------------------------

[INFO] BUILD SUCCESS

[INFO]------------------------------------------------------------------------

[INFO] Total time:31:17.81s

[INFO] Finished At:tue Mar 19:23:10 CST2014

[INFO] Final memory:78m/239m

[INFO]------------------------------------------------------------------------

Finally, a compiled installation package can be found through the following path:

hadoop-2.2.0-src/hadoop-dist/target/hadoop-2.2.0/

The entire compilation is complete.

Redeploying a clustered environment, deploying a cluster online is a lot more than repeating.

You can also download the compiled 64-bit installation package from the following path: Http://pan.baidu.com/s/1ntlupaL