centOS 64位下編譯hadoop2.4.1

標籤：64位   centos   hadoop   bigdata   maven   

1. 安裝JDK
a.      解壓jdk

[[email protected] software]$ tar -xvzf jdk-7u60-linux-x64.tar.gz

b.      設定環境變數

PATH=$PATH:$HOME/binexport JAVA_HOME=/home/bigdata001/BigDataPlatform/jdk1.7.0_60export JAVA_BIN=$JAVA_HOME/binexport CLASSPATH=.:$JAVA_HOME/lib/dt.jar:$JAVA_HOME/lib/tools.jarexport JRE_HOME=$JAVA_HOME/jreexport PATH=$PATH:$JAVA_HOME/binexport PATH

c.      使之生效

[[email protected] ~]$ source .bash_profile

d.      驗證

[[email protected] ~]$ java -version

java version "1.7.0_60"Java(TM) SE Runtime Environment (build 1.7.0_60-b19)Java HotSpot(TM) 64-Bit Server VM (build 24.60-b09, mixed mode)

2.   安裝Mavena.      解壓

[[email protected] software]$ tar -xvzf apache-maven-3.2.3-bin.tar.gz

b.      配置環境變數

修改系統內容變數有兩種方式，修改/etc/profile，或者在/etc/profile.d/下添加定製的shell檔案，鑒於profile檔案的重要性，盡量不要在profile檔案裡新增內容，官方建議採用第二種，以保證profile檔案的絕對安全。
下面採用第二種方式：

[[email protected] etc]# cd /etc/profile.d/[[email protected] profile.d]# touch maven.sh

    向maven.sh中添加如下內容

export MAVEN_HOME='/home/bigdata001/BigDataPlatform/apache-maven-3.2.3'export PATH=$MAVEN_HOME/bin:$PATH

c.      使之生效

[[email protected] profile.d]# source /etc/profile

d.      驗證安裝

Apache Maven 3.2.3 (33f8c3e1027c3ddde99d3cdebad2656a31e8fdf4; 2014-08-12T04:58:10+08:00)Maven home: /home/bigdata001/BigDataPlatform/apache-maven-3.2.3Java version: 1.7.0_60, vendor: Oracle CorporationJava home: /home/bigdata001/BigDataPlatform/jdk1.7.0_60/jreDefault locale: zh_CN, platform encoding: UTF-8OS name: "linux", version: "2.6.32-431.el6.x86_64", arch: "amd64", family: "unix"

3.   安裝Protobuf
a.      安裝依賴軟體

[[email protected] ~]# yum install gcc*

b.      解壓縮（注意壓縮檔尾碼， maven安裝包是—gzip檔案，解壓時需加–z）

[[email protected] software]$ tar -xvf protobuf-2.5.0.tar.bz2

c.      安裝

[[email protected] BigDataPlatform]$ cd protobuf-2.5.0/[[email protected] protobuf-2.5.0]$ ./configure[[email protected] protobuf-2.5.0]$ make[[email protected] protobuf-2.5.0]$ make check[[email protected] protobuf-2.5.0]# make install

d.      驗證安裝

[[email protected] protobuf-2.5.0]# protoc --version

libprotoc 2.5.0

4.    編譯Hadoopa.      解壓縮

[[email protected] software]$ tar -xvzf hadoop-2.4.1-src.tar.gz

b.      修改Maven設定檔

    切換為root許可權並修改檔案

[[email protected] BigDataPlatform]# gedit apache-maven-3.2.3/conf/settings.xml

     增加profile內容

<profile>      <id>jdk-1.7</id>      <activation>        <jdk>1.7</jdk>      </activation>      <repositories>        <repository>          <id>nexus</id>          <name>local private nexus</name>          <url>http://maven.oschina.net/content/groups/public/</url>          <releases>            <enabled>true</enabled>          </releases>          <snapshots>            <enabled>false</enabled>          </snapshots>        </repository>      </repositories>      <pluginRepositories>        <pluginRepository>          <id>nexus</id>          <name>local private nexus</name>          <url>http://maven.oschina.net/content/groups/public/</url>          <releases>            <enabled>true</enabled>          </releases>          <snapshots>            <enabled>false</enabled>          </snapshots>        </pluginRepository>      </pluginRepositories>    </profile>

c.      安裝依賴包

[[email protected] hadoop-2.4.1-src]# yum install ncurses-devel[[email protected] hadoop-2.4.1-src]# yum install openssl-devel[[email protected] hadoop-2.4.1-src]# yum install cmake

d.      由於編譯依賴的組件已經安裝完成，所以直接執行編譯命令即可

[[email protected] hadoop-2.4.1-src]# mvn package -Pdist,native -DskipTests -Dtar

    經過的漫長等待，hadoop編譯結束，效果

e.      驗證編譯    編譯後的檔案存在於

/home/bigdata001/BigDataPlatform/hadoop-2.4.1-src/hadoop-dist/target

f.      驗證版本號碼

[[email protected] hadoop-2.4.1]$ bin/hadoop version

Hadoop 2.4.1Subversion Unknown -r UnknownCompiled by root on 2014-08-18T10:51ZCompiled with protoc 2.5.0From source with checksum bb7ac0a3c73dc131f4844b873c74b630This command was run using /home/bigdata001/BigDataPlatform/hadoop-2.4.1-src/hadoop-dist/target/hadoop-2.4.1/share/hadoop/common/hadoop-common-2.4.1.jar

g.      驗證hadoop lib 的位元

[[email protected] hadoop-2.4.1]$ file lib//native/*

lib//native/libhadoop.a:        current ar archivelib//native/libhadooppipes.a:   current ar archivelib//native/libhadoop.so:       symbolic link to `libhadoop.so.1.0.0'lib//native/libhadoop.so.1.0.0: ELF 64-bit LSB shared object, x86-64, version 1 (SYSV), dynamically linked, not strippedlib//native/libhadooputils.a:   current ar archivelib//native/libhdfs.a:          current ar archivelib//native/libhdfs.so:         symbolic link to `libhdfs.so.0.0.0'lib//native/libhdfs.so.0.0.0:   ELF 64-bit LSB shared object, x86-64, version 1 (SYSV), dynamically linked, not stripped