Ubuntu 14.04 Hadoop Eclipse Primary Environment configuration

Source: Internet
Author: User

The next day of contact with Hadoop, the configuration of Hadoop to the environment also took two days, the process of their own configuration is written here, I hope to help you!


I will use the text to share all the resources here, click to download, do not need to find a!

Among them is "the Hadoop Technology Insider" This book, the first chapter describes this configuration process, but not detailed ~


---------------Installing the JDK-------------------------------

1.
Download jdk1.6.0_45

2. Unzip to the OPT folder, configure/etc/profile, add the end of the file
#set Java Environment
Java_home=/opt/jdk1.6.0_45
Export JRE_HOME=/OPT/JDK1.6.0_45/JRE
Export classpath= $JAVA _home/lib: $JRE _home/lib: $CLASSPATH
Export path= $JAVA _home/bin: $JRE _home/bin: $PATH

3. Then use Source/etc/profile to re-execute the newly modified initialization file (profile)

4. Configure Default Programs
Update-alternatives--install/usr/bin/java Java/opt/jdk1.6.0_45/bin/java 300
Update-alternatives--install/usr/bin/java Java/opt/jdk1.6.0_45/bin/javac 300
Update-alternatives--install/usr/bin/java Java/opt/jdk1.6.0_45/bin/jar 300
Update-alternatives--install/usr/bin/java Java/opt/jdk1.6.0_45/bin/javah 300
Update-alternatives--install/usr/bin/java JAVA/OPT/JDK1.6.0_45/BIN/JAVAP 300

Then execute the following code to select the version I installed to the JDK:
Update-alternatives--config Java
5. You can then use Java-version to view the Java version.

---------------Install Eclipse-------------------------------
1. Download Java version to eclipse from official website
Http://mirror.neu.edu.cn/eclipse/technology/epp/downloads/release/kepler/SR2/eclipse-java-kepler-SR2-linux-gtk.tar.gz
2. Unzip to the/home/simon folder
3. Create a shell script with VI named Eclipse
Vi/usr/local/bin/eclipse
The contents are as follows:
/home/simon/eclipse/eclipse
4. Add executable permissions for script eclipse: chmod +x/usr/local/bin/eclipse
5. Enter eclipse directly to launch it.

---------------Install ant-------------------------------
1. Download the ant
Http://mirror.esocc.com/apache//ant/binaries/apache-ant-1.9.4-bin.tar.gz
2. Unzip the copy to the/home/simon folder
3. Modify the/etc/profile file
Export ant_home=/home/simon/apache-ant-1.9.4
Export path= $PATH $: $ANT _home/bin
4. Then use Source/etc/profile to re-execute the newly modified
5. Enter Ant-version to verify installation success
Apache Ant (TM) version 1.9.4 compiled on April 29 2014
---------------Installing Hadoop-------------------------------
1. Modify the machine name, edit/etc/hostname to localhost
2. Configure SSH login without password
SSH-KEYGEN-T RSA
CD ~/.ssh
Cat Id_rsa.pub >> Authorized_keys
Apt-get Install Openssh-server
3. If the command ssh localhost is unsuccessful, you need to start the SSH service
Start the SSH service with the following command
Service SSH Start
/etc/init.d/ssh start
If it fails to start, restart it, it works.

3. Configure Hadoop
(1) Edit conf/hadoop-env.sh, modify the value of Java_home:
Export java_home=/opt/jdk1.6.0_45
(2) Edit Conf/mapred-site.xml, add content:
<property>
<name>mapred.job.tracker</name>
<value>http://localhost:9001</value>
</property>
(3) Edit Conf/hdfs-site.xml, add content:
<property>
<name>dfs.name.dir</name>
<value>/home/simon/name</value>
</property>

<property>
<name>dfs.data.dir</name>
<value>/home/simon/data</value>
</property>
<property>
<name>dfs.permissions</name>
<value>false</value>
</property>
<property>
<name>dfs.replication</name>
<value>1</value>
</property>
(4) Edit Conf/core-site.xml, add content:
<property>
<name>fs.default.name</name>
<value>hdfs://localhost:9000</value>
</property>

<property>
<name>hadoop.tmp.dir</name>
<value>/home/hadoop/hadoop-1.0.0/tmp</value>
</property>
(5)
Formatted Hdfs:bin/hadoop Namenode-format
Start hadoop:bin/start-all.sh
If no permissions are displayed, the file may not have permissions, or the file to the user is not the current user (root)
You can try chmod +x file name
Chown Root:root bin/*
-------------------Configuring the Eclipse plug-in---------------
1. Copy the Hadoop-eclipse-plugin-1.0.0.jar to the Eclipse directory under the Plugins folder
2. Open Eclipse
Window-showview-other ... dialog box, select MapReduce tools-map/reduce Locations
If the dialog box does not, then:%eclispe_dir%/configration/config.ini file, found inside there is a org.eclipse.update.reconcile=false configuration, Re-enter eclipse after changing to true
3. DFS locations can be seen in Project Explorer, if you can open a few folders down to indicate the configuration is successful

Start Eclipse:
Env ubuntu_menuproxy=/home/simon/eclipse/eclipse start Eclipse, note that there is a space between the equals sign and the eclipse path

------------------Running Java programs--------------------
1. Configure input/output to path
In the program, right-click--run as--run Configurations. --argument
Fill in
Hdfs://localhost:9000/test/input Hdfs://localhost:9000/test/output
Intermediate spacing between input and output to path
2. Import Hadoop into the jar package, right-click on the project--properties--left to select Java Build path--Right-click on the right side of the libraries--to add External JARs ...
In hadoop/lib/this path to select the jar package, if you do not know which to choose, then select All! ~ (helpless)
3.in the program, right-click--run as--Run on Hadoop running program
Related Article

Contact Us

The content source of this page is from Internet, which doesn't represent Alibaba Cloud's opinion; products and services mentioned on that page don't have any relationship with Alibaba Cloud. If the content of the page makes you feel confusing, please write us an email, we will handle the problem within 5 days after receiving your email.

If you find any instances of plagiarism from the community, please send an email to: info-contact@alibabacloud.com and provide relevant evidence. A staff member will contact you within 5 working days.

A Free Trial That Lets You Build Big!

Start building with 50+ products and up to 12 months usage for Elastic Compute Service

  • Sales Support

    1 on 1 presale consultation

  • After-Sales Support

    24/7 Technical Support 6 Free Tickets per Quarter Faster Response

  • Alibaba Cloud offers highly flexible support services tailored to meet your exact needs.