標籤:
1. 下載Hadoop 2.2.0 源碼包,並解壓
$ wget http://mirrors.hust.edu.cn/apache/hadoop/common/hadoop-2.2.0/hadoop-2.2.0-src.tar.gz$ tar zxf hadoop-2.2.0-src.tar.gz
2. 安裝下面的軟體
$ sudo yum install cmake lzo-devel zlib-devel gcc autoconf automake libtool ncurses-devel openssl-devel
3. 安裝Maven
$ wget http://mirror.esocc.com/apache/maven/maven-3/3.1.4/binaries/apache-maven-3.1.4-bin.tar.gz$ sudo tar zxf apache-maven-3.1.4-bin.tar.gz -C /opt$ sudo vim /etc/profileexport MAVEN_HOME=/opt/apache-maven-3.1.4export PATH=$PATH:$MAVEN_HOME/bin
使之生效 source /etc/profile
4. 安裝Ant
$ wget http://apache.dataguru.cn//ant/binaries/apache-ant-1.9.4-bin.tar.gz$ sudo tar zxf apache-ant-1.9.4-bin.tar.gz -C /opt$ sudo vim /etc/profileexport ANT_HOME=/opt/apache-ant-1.9.4export PATH=$PATH:$ANT_HOME/bin
5. 安裝Findbugs
$ wget http://prdownloads.sourceforge.net/findbugs/findbugs-2.0.3.tar.gz?download$ sudo tar zxf findbugs-2.0.3.tar.gz -C /opt$ sudo vim /etc/profileexport FINDBUGS_HOME=/opt/findbugs-2.0.3export PATH=$PATH:$FINDBUGS_HOME/bin
6. 安裝protobuf
$ wget https://protobuf.googlecode.com/files/protobuf-2.5.0.tar.gz$ tar zxf protobuf-2.5.0.tar.gz$ cd protobuf-2.5.0$ ./configure$ make$ sudo make install
7. 給Hadoop源碼打一個patch
最新的Hadoop 2.2.0 的Source Code 壓縮包解壓出來的code有個bug 需要patch後才能編譯。否則編譯hadoop-auth 會提示下面錯誤:
[ERROR] Failed to execute goal org.apache.maven.plugins:maven-compiler-plugin:2.5.1:testCompile (default-testCompile) on project hadoop-auth: Compilation failure: Compilation failure:[ERROR] /home/chuan/trunk/hadoop-common-project/hadoop-auth/src/test/java/org/apache/hadoop/security/authentication/client/AuthenticatorTestCase.java:[84,13] cannot access org.mortbay.component.AbstractLifeCycle[ERROR] class file for org.mortbay.component.AbstractLifeCycle not found
Patch: https://issues.apache.org/jira/browse/HADOOP-10110
wget https://issues.apache.org/jira/secure/attachment/12614482/HADOOP-10110.patch; patch -p0 < HADOOP-10110.patch
8. 編譯 Hadoop
cd hadoop-2.2.0-srcmvn package -DskipTests -Pdist,native -Dtar
9. 替換掉32位的native庫
用 hadoop-2.2.0-src/hadoop-dist/target/hadoop-2.2.0/lib/native
替換掉 hadoop-2.2.0/lib/native
。
rm -rf ~/local/opt/hadoop-2.2.0/lib/nativecp ./hadoop-dist/target/hadoop-2.2.0/lib/native ~/local/opt/hadoop-2.2.0/lib/
hadoop namenode -format
start-all.sh
jps
64位CentOS上編譯 Hadoop 2.2.0