eclipse debug調試mapreduce程式

標籤：io   os   ar   java   sp   檔案   on   ad   ef   

1、將mapred-site.xml檔案拷貝一份到項目中

<configuration>    <property>        <name>mapreduce.framework.name</name>            <value>yarn</value>    </property>    <property>         <name>mapred.child.java.opts</name>         <value>-Xmx800m -Xdebug -Xrunjdwp:transport=dt_socket,server=y,suspend=y,address=8000</value>     </property>    <property>    <name>mapreduce.jobtracker.staging.root.dir</name>    <value>/tmp</value>  </property>   <property>    <name>yarn.app.mapreduce.am.staging-dir</name>    <value>/tmp</value>  </property>  <property>    <name>mapreduce.framework.name</name>    <value>local</value>  </property>  <property>    <name>mapreduce.jobtracker.address</name>    <value>local</value>  </property><property><name>mapred.job.tracker</name><value>local</value></property></configuration>

2、在項目中加入本地mapred-site.xml配置，從本地項目中讀取設定檔運行，在reduce中debug

 Job job = new Job(conf, "word count");     conf.addResource("classpath:/Hadoop01/mapred-site.xml");     conf.set("fs.defaultFS", "hdfs://192.168.1.10:9008");     conf.set("mapreduce.framework.name", "yarn");     conf.set("yarn.resourcemanager.address", "192.168.1.10:8032");     conf.set("mapred.remote.os", "Linux");     conf.set("hadoop.job.ugi", "hadoop,hadoop");

eclipse debug調試mapreduce程式