Symptom:
After a job is submitted, it is always run on the local machine and cannot be submitted to Hadoop job tracker.
Http: // 192.168.1.130: 50030/jobtracker. jsp. The running job cannot be viewed.
In the console output:
14/02/15 00:04:20 INFO mapred. LocalJobRunner: reduce> sort
Cause analysis:
First Attempt:
Replace the original default new Configuration () with the one obtained from the Configuration file, and use the following class
Public class ConfigurationUtil {
Private static Configuration conf = null;
Public static final String USER_DIR = System. getProperty ("user. dir ");
Public static final String FILE_SEPARATOR = System
. GetProperty ("file. separator ");
Public static Configuration getConf (){
If (conf = null ){
Conf = new Configuration ();
String pathCore = USER_DIR + FILE_SEPARATOR + "config. xml ";
System. out. println (pathCore );
Conf. addResource (new Path (pathCore ));
// String hostIp = "192.168.1.130 ";
// Conf. set ("hadoop. job. ugi", "hadoop ");
// Conf. set ("hadoop. job. user", "hadoop ");
// Conf. set ("mapred. job. tracker", hostIp + ": 9001 ");
// Conf. set ("fs. default. name", "hdfs: // namenode.meritit.com: 9000 ");
// Conf. set ("mapred. map. tasks", "2 ");
}
Return conf;
}
Public static void main (String args []) {
System. out. println (getConf (). get ("hadoop. job. user "));
}
}
Solve the problem after completion!
Build a Hadoop environment on Ubuntu 13.04
Cluster configuration for Ubuntu 12.10 + Hadoop 1.2.1
Build a Hadoop environment on Ubuntu (standalone mode + pseudo Distribution Mode)
Configuration of Hadoop environment in Ubuntu
Detailed tutorial on creating a Hadoop environment for standalone Edition
Build a Hadoop environment (using virtual machines to build two Ubuntu systems in a Winodws environment)