Use elasticsearch in Windows to connect to a hadoop cluster in Linux.

Source: http://suxain.iteye.com/blog/1748356

Hadoop is a distributed system that works in Linux. As a developer, it has limited resources and has to use terminal-only virtual machines to run hadoop clusters. However, in this environment, development and debugging become so difficult. So, is there a way to issue debugging in windows. The answer is yes.

Hadoop provides us with an elasticsearch plug-in that we can develop and debug hadoop In the eclipse environment.ProgramThen, how should we install the eclipse-hadoop plug-in. I will share my research results with you.(Attach the plug-in compiled by myself).

1. Compile eclipse-hadoop-plugins

Find/src/contrib/eclipse-plugin in the hadoop directory and use eclipse to import the project,

 

 

 

As the plug-in requires some jar files of hadoop, right-click the project and choose build path> configuretion.
Build path-> libraies-> Add external jars
. Find the following jar in the hadoop directory and Its Lin directory, and add

 

Then openBuild-contrib.xml

Modify the following

 

<Property name = "hadoop. Root" location = "E: \ boolbean \ eclipseworkspaces \ hadoop-1.1.0"/>

<Property name = "Eclipse. Home" location = "D:/program files/Eclipse"/>

<Property name = "version" value = "1.1.0"/>

 

 

Right-click build. XML-> Run as-> ant build...

After compilation is successful, go to $ {hadoop. Root}/build/contrib/eclipse-plugin/to find the hadoop-eclipse-plugin-1.1.0.jar,

Copy the JAR file to the Plugins directory of the eclipse installation directory and restart eclipse.

After eclipse is restarted, select Window> preferences. The installation is successful.

 

And you can see DFS location in projects Explorer

 

Click window-> show views-> others and find MAP/reduce locations. The mapreduce view is displayed.

 

 

 

Right-click and select Edit hadoop location...

Location name, customized, not required

MAP/reduce master corresponds to IP and port set in mapred-site.xml

IP and port in DFS master core-site.xml

In advanced, there are various set parameters, inside, on the core-site.xml, hdfs-site.xml, mapred-site.xml in the parameters set, here also make the corresponding changes.

Then finish

 

If the operation succeeds, the HDFS directory structure appears in the DFS location.

 

When connecting eclipse to hadoop, You need to disable the Linux firewall or edit iptables excessive rules.

 

The above is the result of my struggle for many days. During compilation and installation, new users like me may encounter many problems, such as Google and Baidu. In addition, when connecting to hadoop
Eclipse Error Log
The view shows the detailed error description. In order to find the correct direction to solve the problem. When the problem arises, let alone the assumption that the problem will occur.

 

 

• Hadoop-eclipse-plugin-1.1.0.jar (6.1 MB)
• Downloads: 0