MyEclipse configuration in Linux Hadoop

I. Required Software

Note: After installing MyEclipse, I can not find myeclipse in the software, so I start myeclipse with command line under root, it seems to avoid many permissions problems.

sudo su input password CD usr/myeclipse./myeclipse

two. Steps

1. First copy the Hadoop-eclipse-plugin-1.2.1.jar to Ubuntu desktop (I can not directly copy to MyEclipse Dropins directory, my classmate's ability, really strange ah), and then on the desktop.

Or first enter the ROOTCD CD. Back to the root directory CD home/hxsyl/desktop//find real Desktop//mobile files need in the directory of the file//put in the Dropins directory, not plugins directory, originally I thought there is no plugins directory, just put in Dropins, Not so. Sudo mv Hadoop-eclipse-plugin-1.2.1.jar/usr/myeclipse/dropins

Note: There is a desktop directly under the places under two folders on the left, But can not find just copied over the hadoop-eclipse-plugin-1.2.1.jar,devices under the computer, then home, and then desktop to see the plug-in, remember this path, after the need to copy things.

2. Enter the configuration under the same directory as Dropins, delete org.eclipse.update (this is a folder, but I did not delete).

3. Restart the MYECLIPSE4. Add the MapReduce View "Windows", "open Perspective" and "other", you will see a small elephant map/reduce, if none, please tick "show All", You can open the view (normal project in Java view, this step is to open an attempt to write code). At this point, the project browser on the left side of MyEclipse will show Dfs Locations, there is a myhadoopserver (my Hadoop server name), so I would like to our usual JSP server.

4. Configure the Hadoop server "Widows", "Show views", "mapReduce tool"-& Gt "Map/reduce locations", open, (in the following console near the map/reduce locations), right-click in the blank, select New Service "New Hadoop location, Go to the General tab.

1) "Location name": Fill in the new Hadoop service name, I wrote Myhadoopserver
2) "Map/reduce Master": This jobtracker that corresponds to the Hadoop server
A) Host: For the IP with Hadoop jobtracker, the standalone is localhost and can be changed
B) port: The port corresponding to the Jobtracker, the port configured in Mapred-site.xml, typically 9001, the default is 50400
3) "DFS Master": This corresponds to the port of HDFs, corresponding to the Core-site.xml
A) Host: The IP of the HDFs with Hadoop, which is localhost, this cannot be changed
B) port: Port corresponding to HDFs, port configured in Mapred-site.xml, typically 9000, default is 50400

Note: There is also a user name below, the default root I did not change.

Note: Open perspective, on the right side of MyEclipse, is an edit view; Show views open things under MyEclipse, and console together.

5. Open the Advanced Parameters tab on the same level as the General tab, find the Hadoop.tmp.dir (easy to find, sort by dictionary order), and then locate the corresponding content in Core-site.xml, mine is home/hxsyl/ Tmp.

6. At this point you can see the project browser on the left under the DFS locations myhadoopserver two folders, a named Home/hxsyl/tmp, this is the 5th step configuration of the folder, and the other is user/root/in and out.

This root should be the 4th step of the user name, in and out is like to run WordCount input and output files, run WordCount need command line parameters, specifically the mail run-as, run configurations, parameters And then input (note: You must run Wordcount.java first, then prompt for fewer parameter errors, then Run-as, run configurations, parameters, direct words do not show parameters, it is strange)

Hdfs://localhost:9000/user/root/in Hdfs://localhost:9000/user/root/out

Note: The in file needs to be uploaded to HDFs first, specifically: I am in the root directory (and USR sibling) Testhadoop folder

sudo mkdir testhadoop

Then go to the folder

Test1 and 2 will automatically generate the echo "Hello World" >test1echo "Hello Hadoop" in the Testhadoop folder >test2

and upload the file to HDFs.

At this time in the Testhadoop folder, but we need to go to the Hadoop folder, mainly to use the Hadoop command//But do not need a CD, must not go into the Hadoop folder, because the file will be uploaded in the directory//because we directly. Testhadoop, did not write absolute path (own understanding). Usr/hadoop1.2.1file/hadoop-1.2.1/bin/hadoop fs-put./testhadoop in//Note:./testhadoop, if you press TAB only to generate./testHadoop, He will be multiple/, be sure to delete, in is to upload to the corresponding folder on the HDFs//want to view in content, two methods, one is to use the following command Usr/hadoop1.2.1file/hadoop-1.2.1/bin/hadoop Fs-ls. Testhadoop///Another method is (already uploaded HDFs at this time), run the code * (whether or not the error is OK, personal understanding is the resume and HDFs link, then see the contents of HDFs), refresh Dfs Locations, in user/root/ In below you will see

7. After configuration, run, the result is under the DFS Locations user/root/out under the part-r-0000 file, the results are as follows.

Hadoop1hello2world1

Note: If you need to run WordCount at the command line (which is now uploaded to HDFs), you need to go to Hadoop

CD usr/hadoop1.2.1file/hadoop-1.2.1

Then run the code as follows

I haven't tried wordcount. command line case insensitive Bin/hadoop jar Hadoop-examples-1.2.1.jar WordCount in Out

Note: The problem here is that you must not be able to enter the bin directory, I have tried to enter the words can not run, it is really strange.

Then view the results of the run.

Bin/hadoop Fs-cat./out/part-r-00000

three. Concluding remarks and remaining issues3.1 Legacy Issues

1. How to copy directly from win to Linux corresponding folder, without first copying to the left, in MV

2.xml class files, sometimes can be opened directly, edit, save, but sometimes after this, the save is not available, it is really strange, must

This is the JDK environment variable sudo gedit/etc/profile

Is it because the command line terminal is not root, it should not be the problem, every time I enter Ubuntu, I first open the terminal, and then into root.

3.2 Concluding remarks

This is easy to understand against the command line and DFS Locations.

3.3 Codeforfuture

　　Codeforfuture,dotaforfun ... The group focuses on the Internet, e-commerce and data mining,members of the group are from major universities and graduate students (such as Tsinghua University, Peking University, Zhongshan University, Beijing United University, South China University of Technology, CAS, frozen, Shandong University, PingdingshanCollege, Nanyang Polytechnic, Shanghai University, Network Research Institute ... no longer listed) and the major companiesemployees (such as Baidu, Sina, Jinshan, fun games and so on), as well as headhunting puppet ... Look forward to your additioninto, let us go from excellence to excellence ...

Group number: 163354117, the left can be directly less than two-dimensional code addition group.

MyEclipse configuration in Linux Hadoop