Configuring the Tomcat Hadoop Eclipse environment under Ubuntu14.04lts

Under the relevant software:

(depending on whether the system is a 32-bit or 64-bit software version: JDK and Eclipse.) I machine memory 4G, install 64-bit virtual machine system very card, recommended virtual machine installation 32-bit version, physical machine can install 64-bit)

hadoop:http://mirrors.hust.edu.cn/apache/hadoop/common/

Jdk:http://www.oracle.com/technetwork/java/javase/downloads/jdk7-downloads-1880260.html

tomcat:http://tomcat.apache.org/download-80.cgi

eclipse:http://www.eclipse.org/downloads/

(JSE is the standard version, JEE is Enterprise Edition, eclipse download jee,jdk download JSE)

Installing virtual machines

Vmware+ubuntu14.04-32

User name: Sunny

Machine Name: SUNNYPC

Hard disk Size: 128G (at least greater than 20G)

After the installation is complete,

Update source sudo apt-get update

Install sudo apt-get install vim

The following assumes the computer user name: Sunny

Copy the downloaded software to the/home/sunny/downloads directory:

Plan to install the JDK into the system directory:/usr/local/lib

Install Hadoop, Tomcat, and eclipse into the user directory:/home/sunny/usr/

[Description: The installation is very simple to extract to the appropriate directory, mainly to set directory permissions and configuration files]

Add root User:

sudo passwd root

You can enter your password.

Modify Machine Name:

sudo gedit/etc/hostname

Instead: MASTERPC

sudo gedit/etc/hosts

Change Sunnypc to: MASTERPC

Restart takes effect.

Installing the JDK

Cd/usr/local/lib/su roottar-zxvf/home/sunny/downloads/jdk-7u71-linux-x64.tar.gz

(Note: Here Sunny user's directory to write complete, ~ will run to the root user's directory)

Ls-l

Modify the folder's owning user and user group (root:root), folder permissions (755root user-readable writable executable, other user-readable executable), and configure the JKD path to the system environment variable.

Chown root:root-r jdk1.7.0_71/chmod 755-r jdk1.7.0_71/gedit/etc/profile

Export java_home=/usr/local/lib/jdk1.7.0_671
Export classpath=.: $JAVA _home/jre/lib/rt.jar: $JAVA _home/lib/dt.jar: $JAVA _home/lib/tools.jar
Export path= $PATH: $JAVA _home/bin

. /etc/profilesu Sunny Java-version

(Note: JDK version update is faster, the path of the JDK in the configuration command above must be changed according to its own folder name, including the following java_home in Hadoop)

If the java-version is not displayed correctly, first check whether. /etc/profile, the path above is configured correctly, and if multiple versions of the JDK are already installed on the system

Configure the default JDK:

sudo update-alternatives--install/usr/bin/java java/usr/local/lib/jdk1.7.0_71/bin/java   (default Java, execution Javasudo Update-alternatives--install/usr/bin/javac Javac/usr/local/lib/jdk1.7.0_71/bin/javac 300 (default Javac, compiling Java) sudo Update-alternatives--list Java  (JDK in the display system)

Install Hadoop, Tomcat, eclipse

CD ~/USRTAR-ZXVF. /DOWNLOADS/HADOOP-1.2.1.TAR.GZTAR-ZXVF. /DOWNLOADS/ECLIPSE-JEE-LUNA-SR1-LINUX-GTK.TAR.GZTAR-ZXVF. /downloads/apache-tomcat-8.0.14.tar.gz

You can find the permissions on the folder above and the user groups are correct, or you can manually modify them by command.

Configure Hadoop:

-Install SSH

sudo apt-gt install SSHCD ~ssh-keygen-t rsa-p ""-F ~/.ssh/id_rsacat ~/.ssh/id_rsa.pub >> ~/.ssh/authorized_keysss h localhost

All the way to enter the password, SSH will be able to automatically log in later.

-Configure Hadoop

Enter to ~/usr/hadoop-1.2.1/conf

Core-site.xml:<configuration> <property> <name>fs.default.name</name> <value>hdfs://localhost:9000</value> </property></configuration>---------------------------------------------------- --Mapred-site.xml:<configuration> <property> <name>mapred.job.tracker</name> <value>localhost:9001</value> <property></configuration>----------------------------------------------------- --Hdfs-site.xml:<configuration> <property> <name>dfs.replication</name> <value>1</value> </property></configuration>-------------------------------------------------------hadoop-env.sh:Export java_home=/usr/local/lib/jdk1.7.0_71

When configured as a single-machine pseudo-distributed, the host name is localhost and cannot be masterpc because there is no MASTERPC loopback address through ifconfig visible 127.0.1.1 only localhost 127.0.0.1.

localhost or masterpc changes according to their actual situation

Format the Namenode file system

CD ~/USR/HADOOP-1.2.1/

Bin/hadoop Namenode-format

Open service:

CD ~/usr/hadoop-1.2.1/bin

./start-all.sh

To view the nodes that are started:

JPs

To view the system running:

Enter localhost:50070 in the browser

Test examples:

CD ~/usr/hadoop-1.2.1

./bin/hadoop Fs-put README. TXT Readme.txt

You can find that the system automatically creates folders in the HDFs file system:/usr/sunny, and generates files/usr/sunny/readme.txt

(You can view it in the browser by clicking Browse the filesystem, or you can reconnect Hadoop localhost:50070 directly in Eclipse)

./bin/hadoop jar Hadoop-examples-1.2.1.jar wordcount readme.txt Output
The/home/sunny/output/part-r-00000 file for HDFs is automatically generated when execution is complete

Use the following command to view the results:

./bin/hadoop Fs-cat output/part-r-00000

To configure Tomcat:

Cond

To configure eclipse:

Create the Workspace/eclipse directory under the/home/sunny directory as the working directory for Eclipse.

To/home/sunny/usr/eclipse double-click Open Eclipse, select the working directory just now.

-for Hadoop

Configure the Hadoop application development environment for eclipse:

1. The Eclipse plugin in Hadoop needs to be placed in the plugins directory under Eclipse, but the new version of Hadoop will need to compile the plugin itself, or it can be downloaded from the Web and compiled by others.
2. Windows--Show view--Other select Map/reduce Tools, click the blue Elephant on the lower right ecilpse tab, edit the location of the Hadoop-site file system
3. In the editing interface, the location name is random, and the left Mr and DFS on the right are used to configure the host and port of the job Tacker and Distributed File system, which should correspond to the Hadoop configuration file.

->->->->

After the configuration is complete, you can view the files in the Dfs file system in eclipse by turning on the Hadoop service.

-

Test Hadoop:

Create a new Map/reduce Project and click Configure Hadoop installation path to the right of Use default Hadoop.

In the Project Explore window, in the SRC right-click New WordCount class, package name Org.apache.hadoop.examples, enter the following code:

(See also:/home/sunny/usr/hadoop-1.2.1/src/examples/org/apache/hadoop/examples/wordcount.java)

After the compilation succeeds, run the code, because main runtime requires parameters, so right-click->run as->run Configration->java application->wordcount->arguments

(Delete the Hdfs://user/sunny/output directory before running the program, if this exists)

After the configuration is complete, click the Run button under the Triangle, select Run on Hadoop,ok, after running, the DFS icon right reconnect can see the output generated under the file.

-for Tomcat

Configure the Ecilpse Web development environment:

Server version and Path configuration:

Winow->preference->server->run Time Environment->add-> (select Tomcat version)--(The Tomcat root path you installed) + (Own JDK version)

Go back to the Eclipse interface, find the Servers tab in the lower right corner, click New Server, (modify hostname, select version)->finish

This allows the Server tab to be used, you can start to turn off Tomcat, right-click Add Project, and so on.

New Web project:

File->new->dynamic web-> (project name, tomcat version, Dynamic Web module sersion=2.5)->finish

In the Servers tab, right-click Add and remove to create your own Web project.

(Sometimes the program that is copied from elsewhere configures the Tomcat target version to be different from its own environment, and in the project right-click->properties->server Select the Tomcat target version)

In the project->webcontent right-click Add HTML file:

Input:

<! DOCTYPE html>
Click Run, running on server, will automatically start Tomcat, open the browser:
This is the end of the configuration of a virtual machine.
--------------------------------------------------------------------------------------------------------------- ----------------------------------------
(Pending verification)
For multiple machine configuration, copy the virtual machine to multiple machines, the network mode is set to Bridge, will be renamed from the host name of the node in the form of Slavepc+num.

 
 

  
  
 
   
Configure the IP address of each machine, and then write the machine name and corresponding IP to the/etc/hosts file, sudo gedit/etc/hosts (each machine synchronously modified, add new slave later)
 
   
Modify the files in the hadoop-1.2.1/conf directory to change the previous localhost to masterpc,master file: masterpc,slaves file: All slave machines added by themselves (synchronized modification)
 
   
The public key of the SSH key generated in Masterpc is copied to the slave machine (Authorized_keys)
 
   
Each time you change the composition of the node, you need to namenode-format a bit more.
 
  
 
 
Configuring the Tomcat Hadoop Eclipse environment under Ubuntu14.04lts