Compiling hadoop2.6.0 CDH 5.4.5 Integrated Snappy Compression

Original address: http://www.cnblogs.com/qiaoyihang/p/6995146.html

1. Download Source: http://archive-primary.cloudera.com/cdh5/cdh/5/

2. Prepare the Environment for compilation
A. Install Maven
B, Installation Protobuffer

./configure--prefix=/usr/local/protobuf

Note the dependency packages required to install the compilation
sudo yum install gcc-c++

sudo yum-y install CMake

sudo yum-y install zlib

sudo yum-y install Openssl-devel

Old process:

Config > Make >make check >make install

C, configure environment variables:
#protobuf
Export path= $PATH:/usr/local/protobuf/bin
Export Ld_library_path= $LD _library_path:/usr/local/protobuf/lib

3, installation Snappy

./configure--prefix=/usr/local/snappy

Config > Make >make check >make install

4. Compiling Hadoop integrated Snappy

Export maven_opts= "-xms256m-xmx512m" MVN package-pdist,native-dskiptests-dtar-dbundle.snappy-dsnappy.lib=/usr/ Local/snappy/lib

Attention:

Compiling 2.6 requires 1.7 jdk

Compile completion location:/home/hadoop/hadoop-2.6.0-cdh5.4.5/hadoop-dist/target

The local library integrates snappy

Check local library load

Hadoop checknative-a

View basic information about the installation

File./*

A small problem encountered in the compilation:

Problem: The remote repository is configured at compile time, but the central repository is still read

<!--Configure an image -<Mirrors> <Mirror><!--This sends everything else to/public -<ID>Nexus</ID><mirrorof>*</mirrorof> <URL>http://maven.aliyun.com/nexus/content/groups/public/</URL></Mirror><Mirror><Mirrors> 

In this example, the value of <mirrorOf> is an asterisk, which means that the configuration is a mirror image of all Maven repositories, and any requests for remote warehouses will be forwarded to

http://maven.aliyun.com/nexus/content/groups/public/

If the image warehouse requires authentication, configure an authentication message with the Nexus ID.

It is important to note that because the mirrored warehouse is completely shielded from the mirrored warehouse, MAVEN will not be able to download artifacts when the mirrored warehouse is unstable or the service is stopped.

WORKAROUND: Change * to Central

Compiling hadoop2.6.0 CDH 5.4.5 Integrated Snappy Compression