Process performed by Hadoop jar Xxxx.jar

Figuring out the process of Hadoop jar Xxxx.jar execution: How the jar files are distributed

jobclient (not necessarily a node of the Hadoop cluster) run the jar with the Bin/hadoop scriptPackage, to Hadoop-0.20.2-examples.jarFor example:
The essence of Hadoop jar Hadoop-0.20.2-examples.jar [class name] is:
1. Use the Hadoop script start a JVM process;
2. The JVM process goes to Run Org.apache.hadoop.util.RunJarThis Java class;
3.org.apache.hadoop.util.runjar Unzip hAdoop-0.20.2-examples.jar to hadoop.tmp.dir/hadoop-unjar*/Directory under ;
4.org.apache.hadoop.util.runjar Dynamic load and run Main-class or the specified class;
5.main-class or in the specified class set the properties of the job
6. Submit JobTo the Jobtracker and monitor the operating conditions.

Note: All of the above are performed on Jobclient.

When running the jar file, the jar is extracted to the hadoop.tmp.dir/hadoop-unjar*/directory (e.g./home/hadoop/hadoop-fs/dfs/temp/hadoop- unjar693919842639653083, Note: This directory is a directory of jobclient, not a jobtracker directory）。 The extracted files are:
Drwxr-xr-x 2 Hadoop hadoop 4096 Jul 15:40 Meta-inf
Drwxr-xr-x 3 Hadoop hadoop 4096 Jul 15:40 org
There is a picture of the truth:



the essence of submitting a job is:
generate ${job-id}/job.xml file to Hdfs://${mapred.system.dir}/(e.g. Hdfs://bcn152:9990/home/hadoop/hadoop-fs/dfs/temp/mapred/system/job_201007301137_0012/job.xml), the job description includes path to the jar file， Map|reduce class PathWait a minute.
Upload ${job-id}/job.jar file to Hdfs://${mapred.system.dir}/(e.g. Hdfs://bcn152:9990/home/hadoop/hadoop-fs/dfs/temp/mapred/system/job_201007301137_0012/job.jar)
There is a picture of the truth:


After the job is generated, the job is submitted to Jobtracker via static jobclient.runjob ():
Jobclient JC = new Jobclient (job);
Runningjob RJ = jc.submitjob (Job);
Jobtracker will then dispatch this job,

After the job is submitted, use the following code to get the job's progress:
try {
if (!jc.monitorandprintjob (Job, RJ)) {
throw new IOException ("Job failed!");
}
} catch (Interruptedexception IE) {
Thread.CurrentThread (). interrupt ();
}