Compiling hadoop2.4.1 with 64-bit centos

1. Install JDK
A. Decompress JDK

[[email protected] software]$ tar -xvzf jdk-7u60-linux-x64.tar.gz

B. Set Environment Variables

PATH=$PATH:$HOME/binexport JAVA_HOME=/home/bigdata001/BigDataPlatform/jdk1.7.0_60export JAVA_BIN=$JAVA_HOME/binexport CLASSPATH=.:$JAVA_HOME/lib/dt.jar:$JAVA_HOME/lib/tools.jarexport JRE_HOME=$JAVA_HOME/jreexport PATH=$PATH:$JAVA_HOME/binexport PATH

C. Make it take effect

[[email protected] ~]$ source .bash_profile

D. Verify

[[email protected] ~]$ java -version

java version "1.7.0_60"Java(TM) SE Runtime Environment (build 1.7.0_60-b19)Java HotSpot(TM) 64-Bit Server VM (build 24.60-b09, mixed mode)

2. Install mavena. Unzip

[[email protected] software]$ tar -xvzf apache-maven-3.2.3-bin.tar.gz

B. Configure Environment Variables

You can modify the system environment variables in either of the following ways:/etc/profile, or in/etc/profile. d/to add a customized shell file. In view of the importance of the profile file, do not add content to the profile file as much as possible. The second method is recommended officially to ensure the absolute security of the profile file.
The second method is used below:

[[email protected] etc]# cd /etc/profile.d/[[email protected] profile.d]# touch maven.sh

Add the following content to maven. sh:

export MAVEN_HOME='/home/bigdata001/BigDataPlatform/apache-maven-3.2.3'export PATH=$MAVEN_HOME/bin:$PATH

C. Make it take effect

[[email protected] profile.d]# source /etc/profile

D. Verify Installation

Apache Maven 3.2.3 (33f8c3e1027c3ddde99d3cdebad2656a31e8fdf4; 2014-08-12T04:58:10+08:00)Maven home: /home/bigdata001/BigDataPlatform/apache-maven-3.2.3Java version: 1.7.0_60, vendor: Oracle CorporationJava home: /home/bigdata001/BigDataPlatform/jdk1.7.0_60/jreDefault locale: zh_CN, platform encoding: UTF-8OS name: "linux", version: "2.6.32-431.el6.x86_64", arch: "amd64", family: "unix"

3. Install protobuf
A. Install dependent Software

[[email protected] ~]# yum install gcc*

B. decompress the package (note the suffix of the compressed file. The Maven installation package is a-GZIP file. Add-Z to the package)

[[email protected] software]$ tar -xvf protobuf-2.5.0.tar.bz2

C. Installation

[[email protected] BigDataPlatform]$ cd protobuf-2.5.0/[[email protected] protobuf-2.5.0]$ ./configure[[email protected] protobuf-2.5.0]$ make[[email protected] protobuf-2.5.0]$ make check[[email protected] protobuf-2.5.0]# make install

D. Verify Installation

[[email protected] protobuf-2.5.0]# protoc --version

libprotoc 2.5.0

4. Compile hadoopa. Decompress

[[email protected] software]$ tar -xvzf hadoop-2.4.1-src.tar.gz

B. Modify the maven configuration file

Switch to the root permission and modify the file

[[email protected] BigDataPlatform]# gedit apache-maven-3.2.3/conf/settings.xml

Add profile content

<profile>      <id>jdk-1.7</id>      <activation>        <jdk>1.7</jdk>      </activation>      <repositories>        <repository>          <id>nexus</id>          <name>local private nexus</name>          <url>http://maven.oschina.net/content/groups/public/</url>          <releases>            <enabled>true</enabled>          </releases>          <snapshots>            <enabled>false</enabled>          </snapshots>        </repository>      </repositories>      <pluginRepositories>        <pluginRepository>          <id>nexus</id>          <name>local private nexus</name>          <url>http://maven.oschina.net/content/groups/public/</url>          <releases>            <enabled>true</enabled>          </releases>          <snapshots>            <enabled>false</enabled>          </snapshots>        </pluginRepository>      </pluginRepositories>    </profile>

C. Install the dependency package

[[email protected] hadoop-2.4.1-src]# yum install ncurses-devel[[email protected] hadoop-2.4.1-src]# yum install openssl-devel[[email protected] hadoop-2.4.1-src]# yum install cmake

D. Because the compilation dependency component has been installed, you can directly execute the compilation command.

[[email protected] hadoop-2.4.1-src]# mvn package -Pdist,native -DskipTests -Dtar

After a long wait, hadoop compilation is complete, and the results are

E. Verify that the compiled file exists in

/home/bigdata001/BigDataPlatform/hadoop-2.4.1-src/hadoop-dist/target

F. Verify the version number.

[[email protected] hadoop-2.4.1]$ bin/hadoop version

Hadoop 2.4.1Subversion Unknown -r UnknownCompiled by root on 2014-08-18T10:51ZCompiled with protoc 2.5.0From source with checksum bb7ac0a3c73dc131f4844b873c74b630This command was run using /home/bigdata001/BigDataPlatform/hadoop-2.4.1-src/hadoop-dist/target/hadoop-2.4.1/share/hadoop/common/hadoop-common-2.4.1.jar

G. Verify the number of digits of hadoop lib

[[email protected] hadoop-2.4.1]$ file lib//native/*

lib//native/libhadoop.a:        current ar archivelib//native/libhadooppipes.a:   current ar archivelib//native/libhadoop.so:       symbolic link to `libhadoop.so.1.0.0'lib//native/libhadoop.so.1.0.0: ELF 64-bit LSB shared object, x86-64, version 1 (SYSV), dynamically linked, not strippedlib//native/libhadooputils.a:   current ar archivelib//native/libhdfs.a:          current ar archivelib//native/libhdfs.so:         symbolic link to `libhdfs.so.0.0.0'lib//native/libhdfs.so.0.0.0:   ELF 64-bit LSB shared object, x86-64, version 1 (SYSV), dynamically linked, not stripped