64-bit Ubuntu compiled Hadoop source

I downloaded the latest version of the Ubuntu64 (14.04) Desktop version of the system, in the installation of hadoop2.6.0, because the official Hadoop is on the 32-bit machine compiled, it is necessary to download the Hadoop source code to compile.

Preparation: HADOOP-2.6.0-SRC

Jdk1.7.0_75 (because the latest version of the JDK is the 1.8.0_31 version, and I am using this version to compile Hadoop problems, and later found that 1.8.0 after Hadoop came out, so there may be some do not support, it is recommended to use 1.7.0 version)

1. Installing the JDK

How to install the JDK you can search online, or through this link http://www.cnblogs.com/yulijunzj/p/4253748.html view, the way is the same

2. Install Maven

sudo Install maven

You can view Maven version information after installation

MVN--version

3, Installation OpenSSH

sudo Install Openssh-server

4. Install dependent libraries

sudo Install pkg-config Libssl-dev

5, Installation Protoc

sudo Install Protobuf-compiler

You can view the version information when you are finished installing

Protoc--version

6, start compiling, enter the Hadoop source code directory HADOOP-2.6.0-SRC, execute:

MVN Clean Package-pdist,native-dskiptests-dtar

7. Successful compilation, the results are as follows:

[INFO]------------------------------------------------------------------------[INFO] Reactor summary:[info] [info] Apache Hadoop Main ....... ..... ............ SUCCESS [1. 205s] [INFO] Apache Hadoop Project POM ......... ......... SUCCESS [1. 187s] [INFO] Apache Hadoop Annotations ......... .......... SUCCESS [2. 856s] [INFO] Apache Hadoop assemblies ......... .......... SUCCESS [0. 256s] [INFO] Apache Hadoop Project Dist POM ......... ..... SUCCESS [1. 726s] [INFO] Apache Hadoop Maven Plugins ........ ......... SUCCESS [2. 863s] [INFO] Apache Hadoop minikdc ......... ............. SUCCESS [3. 064s] [INFO] Apache Hadoop Auth ......... ............... SUCCESS [4. 360s] [INFO] Apache Hadoop Auth Examples ........ ......... SUCCESS [3. 036s] [INFO] Apache Hadoop Common ......... ............. SUCCESS [1: *. 206s] [INFO] Apache Hadoop NFS ......... ................ SUCCESS [7. 385s] [INFO] Apache Hadoop KMS ....... .................. SUCCESS [ One. 518s] [INFO] Apache Hadoop Common Project ......... ....... SUCCESS [0. 037s] [INFO] Apache Hadoop HDFS ......... ............... SUCCESS [2: -. 988s] [INFO] Apache Hadoop Httpfs ......... ............. SUCCESS [ the. 917s] [INFO] Apache Hadoop HDFS bookkeeper Journal .... ..... SUCCESS [8: Wu. 814s] [INFO] Apache Hadoop HDFS-nfs ......... ............ SUCCESS [3. 273s] [INFO] Apache Hadoop HDFS Project ......... ......... SUCCESS [0. 039s] [INFO] Hadoop-yarn ....... ....................... SUCCESS [0. 029s] [INFO] Hadoop-yarn-api ........ ................... SUCCESS [1: +. 677s] [INFO] Hadoop-yarn-common ......... ............... SUCCESS [2:Geneva. 882s] [INFO] Hadoop-yarn-server ......... ............... SUCCESS [0. 040s] [INFO] Hadoop-yarn-server-common ......... .......... SUCCESS [ -. 159s] [INFO] Hadoop-yarn-server-nodemanager ....... ........ SUCCESS [2: the. 110s] [INFO] Hadoop-yarn-server-web-proxy ........ ......... SUCCESS [2. 738s] [INFO] Hadoop-yarn-server-applicationhistoryservice ... SUCCESS [5. 060s] [INFO] Hadoop-yarn-server-resourcemanager .......... SUCCESS [ -. 724s] [INFO] Hadoop-yarn-server-tests ......... ........... SUCCESS [4. 537s] [INFO] Hadoop-yarn-client ......... ............... SUCCESS [6. 404s] [INFO] Hadoop-yarn-applications ......... ........... SUCCESS [0. 031s] [INFO] Hadoop-yarn-applications-distributedshell ..... SUCCESS [2. 486s] [INFO] Hadoop-yarn-applications-unmanaged-am-launcher .... SUCCESS [1. 715s] [INFO] Hadoop-yarn-site ......... ................. SUCCESS [0. 060s] [INFO] Hadoop-yarn-registry ......... .............. SUCCESS [4. 597s] [INFO] Hadoop-yarn-project ......... ............... SUCCESS [4. 524s] [INFO] Hadoop-mapreduce-client ......... ............ SUCCESS [0. 053s] [INFO] Hadoop-mapreduce-client-core ........ ......... SUCCESS [ -. 050s] [INFO] Hadoop-mapreduce-client-common ....... ........ SUCCESS [ the. 055s] [INFO] Hadoop-mapreduce-client-shuffle ....... ........ SUCCESS [3. 437s] [INFO] Hadoop-mapreduce-client-app ........ ......... SUCCESS [8. 152s] [INFO] Hadoop-mapreduce-client-hs ......... ......... SUCCESS [6. 760s] [INFO] Hadoop-mapreduce-client-jobclient ........... SUCCESS [ Wu. 547s] [INFO] Hadoop-mapreduce-client-hs-plugins .......... SUCCESS [1. 622s] [INFO] Apache Hadoop MapReduce Examples ........ ..... SUCCESS [4. 799s] [INFO] Hadoop-mapreduce ......... ................. SUCCESS [4. 062s] [INFO] Apache Hadoop MapReduce streaming ....... ..... SUCCESS [ the. 407s] [INFO] Apache Hadoop distributed Copy ......... ..... SUCCESS [ $. 599s] [INFO] Apache Hadoop Archives ......... ............ SUCCESS [2. 880s] [INFO] Apache Hadoop rumen ......... .............. SUCCESS [5. 047s] [INFO] Apache Hadoop gridmix ......... ............. SUCCESS [4. 041s] [INFO] Apache Hadoop Data Join ......... ........... SUCCESS [2. 439s] [INFO] Apache Hadoop Ant Tasks ......... ........... SUCCESS [2. 007s] [INFO] Apache Hadoop Extras ......... ............. SUCCESS [2. 676s] [INFO] Apache Hadoop Pipes ......... .............. SUCCESS [9. 342s] [INFO] Apache Hadoop OpenStack support ....... ....... SUCCESS [4. 124s] [INFO] Apache Hadoop Amazon Web Services support ... SUCCESS [ the: About. 705s] [INFO] Apache Hadoop Client ......... ............. SUCCESS [6. 766s] [INFO] Apache Hadoop Mini-cluster ......... ......... SUCCESS [0. 111s] [INFO] Apache Hadoop Scheduler Load Simulator ...... SUCCESS [5. 307s] [INFO] Apache Hadoop Tools Dist ......... .......... SUCCESS [ +. 215s] [INFO] Apache Hadoop Tools ......... .............. SUCCESS [0. 024s] [INFO] Apache Hadoop distribution ......... ......... SUCCESS [ +. 475s] [INFO]------------------------------------------------------------------------[INFO] BUILD Success[info]------------------------------------------------------------------------[INFO] Total time: the: -. 388s[info] finished At:fri Jan -  -: -: theCst -[INFO] Final memory:148m/691m[info]------------------------------------------------------------------------

Problem:

I encountered three major problems in compiling Hadoop, which resulted in unsuccessful compilation.

1) When installing MAVEN before compiling his default Java is not pre-loaded, and I previously loaded the JDK version is 1.8.0_31, found incompatible with hadoop2.6.0, need to go to the Hadoop code, more trouble.

Workaround: To use the jdk1.7.0_65 version, after changing the version, I need to change the default version:

    sudo update-alternatives--config java      sudo update-alternatives--config javac     sudo update- Alternatives--config javadoc      sudo update-alternatives--config Javah

2) Error encountered unable to delete target folder

Workaround:

Chown User name folder/-R

3) is because the speed is too slow, from time to time directly failed

Workaround:

Repeat the sixth step of the execution.

64-bit Ubuntu compiled Hadoop source