Apache Oozie Coordinator Job Custom Configuration timer task

<span id="Label3"></p><p><p><strong>first, introduce</strong></p></p><p><p>Oozie is a workflow system for hadoop, and if you use Oozie to submit a mapreduce job (Oozie supports not only MapReduce jobs, but also other types of jobs), you can implement timed runs with Oozie Coordinator Jobs.</p></p><p><p>For a oozie job, it needs to be deployed first before it is submitted to Hadoop. That is, the configuration file (the timed job is the Coordinator.xml,workflow job is workflow.xml), the executable jar file, and the input data that remains to be processed are uploaded to Hdfs.</p></p><p><p>A typical workflow job is configured as Follows:</p></p><p><p></p></p><p><p>Where the corresponding configuration file workflow.xml is as Follows:</p></p><p><p>Among them, Samplemapper and Samplereducer are the mapper classes and reducer classes that will write the MapReduce Program.</p></p><p><p>then, use oozie-job on the command line .... You can submit your job to Hadoop to run it.</p></p><p><p></p></p><p><p><strong>second, the scheduled operation of the configuration and submit the operation</strong></p></p><p><p>This is the focus of this Article.</p></p><p><p>Oozie uses coordinator jobs to implement timing functions, as well as configuration files for a timed job. Namely: Coordinator.xml. A simple example is as Follows:</p></p><p><p></p></p><p><p>frequency, start, and end respectively indicate how often the job is run, the start time, and the ending Time.</p></p><p><p>This is all used instead of: ${frequency}, ${start}, ${end} so that when you submit a job, you only need to assign values to those variables, and you can reuse the same coordinator configuration File.</p></p><p><p>Note: the path to the <app-path> executable jar File. It can be understood that, for a timed job, it is essentially a workflow job, but only in the workflow job to add a timing function. therefore, it needs the <app-path> tag to indicate the jar file HDFs address of the deployment of the workflow job.</p></p><p><p>You only need to change frequency, start, end, and Workflow App-path ${workflowappuri} each time you submit your job. You can ensure that the basic timing job uses the same template Coordinator.xml configuration File.</p></p><pre><span style="color: #008080;"><span style="color: #008080;">1</span></span> <span style="color: #0000ff;"><span style="color: #0000ff;"></span> public</span> <span style="color: #0000ff;"><span style="color: #0000ff;">Static</span></span><span style="color: #000000;"><span style="color: #000000;">string getxmldataforcoordinatorfreq (string appPath, string frequency, string start, string End) {</span></span><span style="color: #008080;"><span style="color: #008080;">2</span></span>StringBuilder SB =<span style="color: #0000ff;"><span style="color: #0000ff;">New</span></span><span style="color: #000000;"><span style="color: #000000;">StringBuilder ();</span></span><span style="color: #008080;"><span style="color: #008080;">3</span></span> <span style="color: #008080;"><span style="color: #008080;">4</span></span>Sb.append ("<?xml version=\" 1.0\ "encoding=\" utf-8\ "?>"<span style="color: #000000;"><span style="color: #000000;">);</span></span><span style="color: #008080;"><span style="color: #008080;">5</span></span>Sb.append ("<configuration>"<span style="color: #000000;"><span style="color: #000000;">);</span></span><span style="color: #008080;"><span style="color: #008080;">6</span></span>Sb.append ("<property>"<span style="color: #000000;"><span style="color: #000000;">);</span></span><span style="color: #008080;"><span style="color: #008080;">7</span></span>Sb.append ("<name>user.name</name>"<span style="color: #000000;"><span style="color: #000000;">);</span></span><span style="color: #008080;"><span style="color: #008080;">8</span></span>Sb.append ("<value>" + user_name + "</value>"<span style="color: #000000;"><span style="color: #000000;">);</span></span><span style="color: #008080;"><span style="color: #008080;">9</span></span>Sb.append ("</property>"<span style="color: #000000;"><span style="color: #000000;">);</span></span><span style="color: #008080;"><span style="color: #008080;">Ten</span></span>Sb.append ("<property>"<span style="color: #000000;"><span style="color: #000000;">);</span></span><span style="color: #008080;"><span style="color: #008080;"></span> one</span>Sb.append ("<name>jobTracker</name>"<span style="color: #000000;"><span style="color: #000000;">);</span></span><span style="color: #008080;"><span style="color: #008080;"></span> a</span>Sb.append ("<value>" + job_tracker + "</value>"<span style="color: #000000;"><span style="color: #000000;">);</span></span><span style="color: #008080;"><span style="color: #008080;"></span> -</span>Sb.append ("</property>"<span style="color: #000000;"><span style="color: #000000;">);</span></span><span style="color: #008080;"><span style="color: #008080;"></span> -</span>Sb.append ("<property>"<span style="color: #000000;"><span style="color: #000000;">);</span></span><span style="color: #008080;"><span style="color: #008080;"></span> the</span>Sb.append ("<name>nameNode</name>"<span style="color: #000000;"><span style="color: #000000;">);</span></span><span style="color: #008080;"><span style="color: #008080;"></span> -</span>Sb.append ("<value>" + Name_node + "</value>"<span style="color: #000000;"><span style="color: #000000;">);</span></span><span style="color: #008080;"><span style="color: #008080;"></span> -</span>Sb.append ("</property>"<span style="color: #000000;"><span style="color: #000000;">);</span></span><span style="color: #008080;"><span style="color: #008080;"></span> -</span>Sb.append ("<property>"<span style="color: #000000;"><span style="color: #000000;">);</span></span><span style="color: #008080;"><span style="color: #008080;"></span> +</span>Sb.append ("<name>oozie.coord.application.path</name>"<span style="color: #000000;"><span style="color: #000000;">);</span></span><span style="color: #008080;"><span style="color: #008080;"></span> -</span>Sb.append ("<value>" + coordinator_config_freq + "</value>"<span style="color: #000000;"><span style="color: #000000;">);</span></span><span style="color: #008080;"><span style="color: #008080;"></span> +</span>Sb.append ("</property>"<span style="color: #000000;"><span style="color: #000000;">);</span></span><span style="color: #008080;"><span style="color: #008080;"></span> a</span>Sb.append ("<property>"<span style="color: #000000;"><span style="color: #000000;">);</span></span><span style="color: #008080;"><span style="color: #008080;"></span> at</span>Sb.append ("<name>workflowAppUri</name>"<span style="color: #000000;"><span style="color: #000000;">);</span></span><span style="color: #008080;"><span style="color: #008080;"></span> -</span>Sb.append ("<value>" + AppPath + "</value>"<span style="color: #000000;"><span style="color: #000000;">);</span></span><span style="color: #008080;"><span style="color: #008080;"></span> -</span>Sb.append ("</property>"<span style="color: #000000;"><span style="color: #000000;">);</span></span><span style="color: #008080;"><span style="color: #008080;"></span> -</span>Sb.append ("<property>"<span style="color: #000000;"><span style="color: #000000;">);</span></span><span style="color: #008080;"><span style="color: #008080;"></span> -</span>Sb.append ("<name>start</name>"<span style="color: #000000;"><span style="color: #000000;">);</span></span><span style="color: #008080;"><span style="color: #008080;"></span> -</span>Sb.append ("<value>" + start + "</value>"<span style="color: #000000;"><span style="color: #000000;">);</span></span><span style="color: #008080;"><span style="color: #008080;"></span> in</span>Sb.append ("</property>"<span style="color: #000000;"><span style="color: #000000;">);</span></span><span style="color: #008080;"><span style="color: #008080;"></span> -</span> <span style="color: #008080;"><span style="color: #008080;"></span> to</span>Sb.append ("<property>"<span style="color: #000000;"><span style="color: #000000;">);</span></span><span style="color: #008080;"><span style="color: #008080;"></span> +</span>Sb.append ("<name>frequency</name>"<span style="color: #000000;"><span style="color: #000000;">);</span></span><span style="color: #008080;"><span style="color: #008080;"></span> -</span>Sb.append ("<value>" + frequency + "</value>"<span style="color: #000000;"><span style="color: #000000;">);</span></span><span style="color: #008080;"><span style="color: #008080;"></span> the</span>Sb.append ("</property>"<span style="color: #000000;"><span style="color: #000000;">);</span></span><span style="color: #008080;"><span style="color: #008080;"></span> *</span> <span style="color: #008080;"><span style="color: #008080;"></span> $</span>Sb.append ("<property>"<span style="color: #000000;"><span style="color: #000000;">);</span></span><span style="color: #008080;"><span style="color: #008080;">Panax Notoginseng</span></span>Sb.append ("<name>end</name>"<span style="color: #000000;"><span style="color: #000000;">);</span></span><span style="color: #008080;"><span style="color: #008080;"></span> -</span>Sb.append ("<value>" + END + "</value>"<span style="color: #000000;"><span style="color: #000000;">);</span></span><span style="color: #008080;"><span style="color: #008080;"></span> the</span>Sb.append ("</property>"<span style="color: #000000;"><span style="color: #000000;">);</span></span><span style="color: #008080;"><span style="color: #008080;"></span> +</span>Sb.append ("</configuration>"<span style="color: #000000;"><span style="color: #000000;">);</span></span><span style="color: #008080;"><span style="color: #008080;"></span> a</span> <span style="color: #008080;"><span style="color: #008080;"></span> the</span> <span style="color: #0000ff;"><span style="color: #0000ff;">return</span></span><span style="color: #000000;"><span style="color: #000000;">sb.tostring ();</span></span><span style="color: #008080;"><span style="color: #008080;"></span> +</span>}</pre><p><p>Because Oozie supports the restclient API, the job is Submitted. therefore, by constructing an HTTP request with the configuration parameters above, you can implement the scheduled job Submission.</p></p><p><p></p></p><p><p>Apache Oozie Coordinator Job Custom Configuration timer task</p></p></span>