Eclipse Remote Debugging MapReduce

Working environment: Centos6.5+eclipse (4.4.2) +hadoop2.7.0

1. Download the Eclipse Hadoop plugin Hadoop-eclipse-plugin-2.7.0.jar paste into the Eclipse directory under plugins to restart Eclipse

2. Choose Windows-->showview-->other-->mapreduce Tool-->map/reduce in eclipse locations

3. After selecting the previous step, eclipse will have a map/reduce locations tab in the lower right corner, right-click on the-->new Hadoop locations and configure as shown:

4, configure Hadoop/etc/hadoop/mapred-site.xml, add the following information in <configuration></configuration>

<property>      <name>mapred.map.child.java.opts</name>       <value>-xmx1024m-xdebug- xrunjdwp:transport=dt_socket,server=y,suspend=y,address=8883</value>  </property>  <property>      <name>mapred.tasktracker.map.tasks.maximum</name>      <value>1 </value>  </property>  

Mapred-site.xml complete information after the Append

<?xml version="1.0"? ><?xml-stylesheet type="text/xsl"href="configuration.xsl"?><!--Put Site-specific property overridesinch  ThisFile. --><configuration><property> <name>mapreduce.framework.name</name> <value>y               arn</value> </property> <property> <name>mapred.map.child.java.opts</name> <value>-xmx1024m-xdebug-xrunjdwp:transport=dt_socket,server=y,suspend=y,address=8883</value> </property> <property> <name>mapred.tasktracker.map.tasks.maximum </name> <value>1</value> </property> </configuration>

Mapred-site.xml complete information after the Append

<?xml version="1.0"? ><?xml-stylesheet type="text/xsl"href="configuration.xsl"?><!--Put Site-specific property overridesinch  ThisFile. --><configuration>
<property> <name>mapreduce.framework.name</name> <value>yarn</value> </p Roperty>
<property> <name>mapred.map.child.java.opts</name> <value>-xmx1024m-xdebug -xrunjdwp:transport=dt_socket,server=y,suspend=y,address=8883</value> </property>
<property> <name>mapred.tasktracker.map.tasks.maximum</name> <value>1</value> </property> <property> <name>mapred.reduce.child.java.opts</name> <value>-xmx1024m-xdebug-xrunjdwp:transport=dt_socket,server=y,suspend=y,address=8884</value> </property>
<property> <name>mapred.tasktracker.reduce.tasks.maximum</name> <value>1</value> </property>
</configuration>

Execution: Hadoop jar Xxx.jar Words.txt/wordsout

After submission, the job will stop at the time of map 0% reduce 0% and wait for the debugger of remote debugging.

at this time in Eclipse-->debug as-->remote Java appplicetion:contention proteties ip:127.0.0.1[According to the situationfill in the actual ip],port: 8883[the port configured above] apply debug

You can then go inside the map function.

5. Debugging the Reduce task

Configure Hadoop/etc/hadoop/mapred-site.xml, add the following information to <configuration></configuration>

<property>      <name>mapred.reduce.child.java.opts</name>      <value>-xmx1024m-xdebug- xrunjdwp:transport=dt_socket,server=y,suspend=y,address=8884</value>  </property>  <property>      <name>mapred.tasktracker.reduce.tasks.maximum</name>      <value>1 </value>  

Execution: Hadoop jar Xxx.jar Words.txt/wordsout

Submitting Job,job later will stop at the time of map 100% reduce 0% and wait for the debugger of remote debugging.

At this point in Eclipse-->debug as-->remote Java appplicetion:contention proteties ip:127.0.0.1[fill in the actual ip],port as appropriate:8884[ Ports configured above] apply debug

You can then enter the reduce function.

This article is basically configured based on the Hadoop pseudo-deployment

Reference article: http://blog.csdn.net/gjt19910817/article/details/30384685

Eclipse Remote Debugging MapReduce