Build a Hadoop project using Maven

Hadoop's MapReduce environment is a complex programming environment, so we should try to simplify the process of building a MapReduce project. Maven is a very good automated project construction tool. Maven helps us to get rid of complicated environment configurations and standardize the development process. Therefore, before writing MapReduce, let's spend some time grinding the knife !! Of course, in addition to Maven, there are other options, such as Gradle (recommended) and Ivy.

1. Introduction to Maven

Apache Maven is a Java project management and automatic building tool provided by the Apache Software Foundation. Based on the concept of the project Object Model (POM), Maven uses a central information segment to manage steps such as building, reporting, and documentation of a project. I used to be a subproject of the Jakarta project and is now an independent Apache project.

Maven developers pointed out on their development websites that the goal of maven is to make project construction easier, it organically concatenates different links in the development process, such as compilation, packaging, testing, and release, and generates consistent and high-quality project information, so that project members can get feedback in a timely manner. Maven effectively supports test priority and continuous integration, reflecting the software development philosophy of encouraging communication and timely feedback. If Ant reuse is based on "copy-paste", Maven implements real reuse of the project construction logic through the plug-in mechanism.

 

Development Environment:

Win2008 64bit
Java-1.6.0_30
Maven-3.1.1
Hadoop-1.2.1

Eclipse Juno Service Release 2

 

2. Maven installation (win)

1) download Maven: http://maven.apache.org/download.cgi

Download the latest xxx-bin.zip file and decompress it on winto download the latest xxx-bin.zip file:

 

2) Set Environment Variables

Decompress E to E: \ software \ share \ apache-maven-3.1.1 on win

Set the maven/bin directory to the environment variable PATH:

Set the JAVA_HOME system variable to be added:

3) run the test

Then, open the command line and enter mvn. The running effect of the mvn command is displayed.

C: \ Users \ Administrator> MVN

[INFO] Scanning for projects...

[INFO] ------------------------------------------------------------------------

[INFO] BUILD FAILURE

[INFO] ------------------------------------------------------------------------

[INFO] Total time: 0.208 s

[INFO] Finished at: Thu Dec 12 11:06:32 CST 2013

[INFO] Final Memory: 4 M/490 M

[INFO] ------------------------------------------------------------------------

[ERROR] No goals have been specified for this build. You must specify a valid li

Fecyclephase or a goal in the format <plugin-prefix >:< goal> or <plugin-group-id

>:< Plugin-artifact-id> [: <plugin-version>]: <goal>. Available lifecycle phases are

: Validate, initialize, generate-sources, process-sources, generate-resources, p

Rocess-resources, compile, process-classes, generate-test-sources, process-test-

Sources, generate-test-resources, process-test-resources, test-compile, process-

Test-classes, test, prepare-package, package, pre-integration-test, integration-

Test, post-integration-test, verify, install, deploy, pre-clean, clean, post-cle

An, pre-site, site, post-site, site-deploy.-> [Help 1]

[ERROR]

[ERROR] To see the full stack trace of the errors, re-run Maven with the-e swit

Ch.

[ERROR] Re-run Maven using the-X switch to enable full debug logging.

[ERROR]

[ERROR] For more information about the errors and possible solutions, please rea

Dthe following articles:

[ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/NoGoalSpecifie

DException

 

4) install the Maven plug-in of Eclipse: MavenIntegration for Eclipse

Go:

Copy http://download.eclipse.org/technology/m2e/releases

Go to the eclipse help menu-> Install New Software for Installation

The installation takes about half a few minutes. (Successful installation only after several attempts due to unstable network conditions)

 

 

 

5) Maven Eclipse plug-in configuration

Windows> Preference

 

Maven details: click here
Maven: click here

Add the Oracle JDBC driver to the Maven Repository

Maven 3.1.0 release, Project Build Tool

Install Maven in Linux

Maven3.0 configuration and simple use

Set up sun-jdk and Maven2 in Ubuntu

Get started with Maven

1 2 3 4 5 Next Page