Hadoop-2.7.2 Package 64-bit compilation tutorial

Many tutorials on the web about Hadoop-2.4 package 64-bit encoding tutorial, the latest version 2.7.2 almost the same, here for everyone to retell.

Share two more authoritative attached links:

Ubuntu User Recommended Reference: http://www.aboutyun.com/forum.php?mod=viewthread&tid=8130&extra=page%3D1&page=1

CentOS Series User reference: Http://www.cnblogs.com/hadoop2015/p/4259899.html

1, the early tool preparation:

(1) CentOS and successful networking (Note: It is best to use root role compilation , reduce folder permissions problems.) Novice to pay attention to the command problem: such as Ubuntu use Apt-get, Cenos use yum difference. )

(2) Source code, JDK7, maven, Ant, Protobuf (the role of each plug-in self-access information)

650) this.width=650; "Src=" Http://s2.51cto.com/wyfs02/M00/83/0B/wKioL1dpQZSS0q-hAABDK5Z10Yw631.jpg-wh_500x0-wm_3 -wmp_4-s_3816518479.jpg "title=" qq20160621160431.jpg "alt=" Wkiol1dpqzss0q-haabdk5z10yw631.jpg-wh_50 "/>

First, install the plug-in step: (All under the root identity operation, environment variable configuration in/etc/profile)

1, JDK Decompression, configuration environment variables java_home and path, verify the Java-version (the following need to verify whether the configuration is successful)

2, MAVEN decompression, configuration maven_home and path.

3. Ant extract, configure Ant _home and path.

4. Install the glibc-headers and g++ commands as follows:

$yum Install Glibc-headers

$ yum Install gcc-c++

5. Install Make and CMake

$yum Install make
$yum Install CMake

6, decompression protobuf, into the Protobuf home directory after decompression,

Then execute the command successively:

$./configure
$ make
$ make Check
$ make Install
$ ldconfig

Modify the environment variable ld_library_path. Authentication ( command:$protoc--version)

To refresh the configuration/etc/profile file for each last configuration:

$source/etc/profile

After configuring the profile to be attached to the configuration file:

650) this.width=650; "Src=" Http://s3.51cto.com/wyfs02/M00/83/0C/wKiom1dpQaaiPjVUAABm-R0nKNE382.jpg-wh_500x0-wm_3 -wmp_4-s_3412243603.jpg "title=" qq20160621162332.jpg "alt=" Wkiom1dpqaaipjvuaabm-r0nkne382.jpg-wh_50 "/>

7. Installing the OpenSSL Library

$ yum Install Openssl-devel

8, install Ncurses-devel Library:

$yum Install Ncurses-devel

To this, the compilation tool installation is basically complete.

Second, the compilation phase:

Start the MAVEN command and start compiling Hadoop 64-bit.

Jump to the extracted Hadoop source folder inside the main directory (src directory), execute:

$ MVN Package-pdist,native-dskiptests-dtar

The waiting time is long and the final success is all success:

650) this.width=650; "Src=" Http://s1.51cto.com/wyfs02/M01/83/0B/wKioL1dpQbfwiFJfAAENvS7ilxc255.png-wh_500x0-wm_3 -wmp_4-s_1956470154.png "title=" qq picture 20160621125704.png "alt=" Wkiol1dpqbfwifjfaaenvs7ilxc255.png-wh_50 "/>

The successful 64-bit Hadoop package is under Scr/hadoop-dist/target.

3, encountered problems:

1. JVM Memory overflow when MAVEN install

Processing: The heap size of the maven_opt can be adjusted in both the environment profile and the Maven executable file. (see Maven compiled JVM tuning issues for details, such as: Http://outofmemory.cn/code-snippet/12652/maven-outofmemoryerror-method)

2. Maven error during compilation. Possible network blocking issues cause dependent library downloads to be incomplete and execute commands multiple times (one pass is more difficult):

$ MVN Package-pdist,native-dskiptests-dtar

3, reported Ant, Protobuf and other errors, plug-in download incomplete or plug-in version of the problem, the first link has more special cases, and recommended

2.7.0 version of the issue rollup post HTTP://WWW.TUICOOL.COM/ARTICLES/IBN63QF

I have less to write a blog, thanks to the cloud community, I hope to help you.

This article is from the "It_hadoop" blog, make sure to keep this source http://quanxiaofei.blog.51cto.com/8911472/1791513

Hadoop-2.7.2 Package 64-bit compilation tutorial