Hadoop 2.3.0-cdh5.1.0 re-compiling

There are many versions of Hadoop, and here I choose the CDH version. CDH is the Cloudera company in Apache original base processed things. The specific CHD is:

http://archive-primary.cloudera.com/cdh5/cdh/5/

The version information is as follows:
Hadoop:hadoop 2.3.0-cdh5.1.0
jdk:1.7.0_79
maven:apache-maven-3.2.5 (3.3.1 and later must be above JDK1.7)
protobuf:protobuf-2.5.0
ant:1.7.1

1. Install Maven
Maven can download it on the MAVEN website (http://maven.apache.org/download.cgi). It is important to note that if the JDK is 1.6 and below, maven can only download up to 3.2.5, if you download 3.3.1 and its later version you must have 1.7 jdk.

2. Installing PROTOBUF
PROTOBUF installation See: HTTP://WWW.TUICOOL.COM/ARTICLES/JM7NN2, the installation process is also very simple.

3. Install other required components
You also need to install ant, gcc and other components, run the following command, you will need to download the installation on-line.

Yum install autoconf automake libtool ant gcc gcc-c++ make cmake openssl-devel ncurses-devel

If you fail, try a few more times. If you have failed, consider yourself downloading the Internet and installing it manually. The specific process is: Windows Internet download ftp passed to Linux on Linux decompression installation.
Where Ant's installation steps are visible: http://blog.csdn.net/zj7758/article/details/8437739

4. Download the Hadoop source code and unzip it
SOURCE is: http://archive-primary.cloudera.com/cdh5/cdh/5/

5. Compiling the source code
Run:

MVN Package-dskiptests-pdist,native-dtar

If an error occurs, you can add-X to display the error details:

Mvn-x Package-dskiptests-pdist,native–dtar

After the recompilation is complete, the native package is typically in the following directory:

.. /hadoop-2.3.0-cdh5.1.0/src/hadoop-common-project/hadoop-common/target/native/target/usr/local/lib/

Such as:

Common errors during the installation process:

1. The jar package download failed in maven.

downloading:https://repository.cloudera.com/artifactory/cloudera-repos/com/cloudera/cdh/cdh-root/5.1.0/ cdh-root-5.1.0.pomdownloading:http://repository.jboss.org/nexus/content/groups/public/com/cloudera/cdh/ cdh-root/5.1.0/cdh-root-5.1.0.pomdownloading:https://repo.maven.apache.org/maven2/com/cloudera/cdh/cdh-root/ 5.1.0/cdh-root-5.1.0.pom[error] The build could not read 1 project---[Help 1][error][error] the project org.apache.h     adoop:hadoop-main:2.3.0-cdh5.1.0 (/home/hbase/install/hadoop-2.3.0-cdh5.1.0/src/pom.xml) has 1 Error[ERROR] Non-resolvable parent Pom:could not transfer artifact com.cloudera.cdh:cdh-root:pom:5.1.0 from/to Cdh.repo (https:// Repository.cloudera.com/artifactory/cloudera-repos): repository.cloudera.com and ' Parent.relativepath ' points at Wrong local POM @ line, column 11:unknown host repository.cloudera.com-"help 2][error][error" to see the full St Ack trace of the errors, re-run Maven with THE-E switch. [ERROR] Re-run Maven using The-x switch To enable full debug logging. [ERROR] [ERROR] For more information on the errors and possible solutions, please read the following articles:[error] [Help 1] http://c Wiki.apache.org/confluence/display/maven/projectbuildingexception[error] [Help 2] http://cwiki.apache.org/ Confluence/display/maven/unresolvablemodelexception[error] unresolveable build extension:plugin Org.apache.felix: maven-bundle-plugin:2.4.0 or one of its dependencies could is resolved:failed to read artifact descriptor for Org.apa che.felix:maven-bundle-plugin:jar:2.4.0 @ at Org.apache.maven.project.DefaultProjectBuilder.build ( defaultprojectbuilder.java:416) at Org.apache.maven.DefaultMaven.collectProjects (defaultmaven.java:672) at Org.apache.maven.DefaultMaven.getProjectsForMavenReactor (defaultmaven.java:663) at Org.apache.maven.DefaultMaven.doExecute (defaultmaven.java:253) at Org.apache.maven.DefaultMaven.execute ( defaultmaven.java:155) at Org.apache.maven.cli.MavenCli.execute (mavencli.java:584) at org.apache.maven.clI.mavencli.domain (mavencli.java:216) at Org.apache.maven.cli.MavenCli.main (mavencli.java:160) at SUN.REFLECT.NATIVEMETHODACCESSORIMPL.INVOKE0 (Native Method) at Sun.reflect.NativeMethodAccessorImpl.invoke ( nativemethodaccessorimpl.java:39) at Sun.reflect.DelegatingMethodAccessorImpl.invoke ( DELEGATINGMETHODACCESSORIMPL.JAVA:25) at Java.lang.reflect.Method.invoke (method.java:597) at org.codehaus.plexus.classworlds.launcher.Launcher.launchEnhanced (launcher.java:289) at Org.codehaus.plexus.classworlds.launcher.Launcher.launch (launcher.java:229) at Org.codehaus.plexus.classworlds.launcher.Launcher.mainWithExitCode (launcher.java:415) at Org.codehaus.plexus.classworlds.launcher.Launcher.main (launcher.java:356) [ERROR] [ERROR] the project org.apache.hadoop:hadoop-main:2.3.0 (/usr/local/release-2.3.0/pom.xml) has 1 error

Reason to find out, engaged for a day, and finally found that the Linux cluster is unable to surf the internet (through Ping www.baidu.com). Finally modified a bit of DNS to be able to surf the Internet.

2. Compilation failed

The MVN package always fails and can be considered via MVN compile, as follows:

MVN Compile-pdist,native-dskiptests-dtar

Knowledge Tip: Turn on and off Hadoop debugging

Open: Export hadoop_root_logger=debug,console off: Export hadoop_root_logger=info,console

Full text, reproduced please specify the source: http://blog.csdn.net/ghuil/article/details/45371977

Hadoop 2.3.0-cdh5.1.0 re-compiling