Quartz timer in operator's use

Timing tasks in the long process of business should still be more, a non-real-time interface file interface, such as the Shell's crontab timed execution of scripts, but the shell processing complex logic is difficult to do, usually put in Java or C + + implementation, Enable a Java-tuned daemon thread with the nohup background Run command.

#!/bin/shaction="$"Procname="$"Exe_user= ' WhoAmI ' run_reck=com.linkage.qyw.main.schedulerstartmem_min="128M"mem_max="1024M"Perm_size="128m"Max_perm_size="256m"#check argsif[$# -ne 2]; ThenEcho "Please input args.";Echo "example:uipquartzjob Start (stop) Jzjktag (agent)";Exit 1;fiif["$ACTION"!="Start"] && ["$ACTION"!="Stop"] ; Then  Echo "args invalid! The first arg should be Start|stop ";Echo "Example:reck Start";Exit 1;fi#executerunclass=$run _reck; Java_language="-DFILE.ENCODING=GBK-DDEFAULT.CLIENT.ENCODING=GBK-DUSER.LANGUAGE=ZH-DUSER.REGION=CN"jpda_opts="-agentlib:jdwp=transport=dt_socket,address=8005,server=y,suspend=n"Ps-ef|grep mode="${procname}"|grep"$EXE _user"|grep-v Grep|awk' {print $} '>${procname}. pidpid_count= ' WC- L "${procname}. pid"|awk' {print '} '`if["$ACTION"="Start"] ; Then  if["$pid _count"  -GT 0] ; Then    Echo "${procname} had already running! $pid _count";Else.Setenv.sh; Nohup Java${java_language}-dapp=${procname}-dthr_num=5-dwork_load= --dmode=${procname}-dprov_code=051-ddb_num=4-duip_home=$HOME-dconfigpath=$HOME/file/uip_quartz/etc/-SERVER-XMS$mem _min-xmx$mem _max-xx:permsize=$perm _size-xx:maxpermsize=$max _perm_size $runclass  ${procname}>$HOME/log/${procname}. log2>&1&Echo "${procname} start to running ...";fielif["$ACTION"="Stop"] ; Then  if["$pid _count"  -GT 0] ; Then     forPidinch' Cat${procname}. PID ' DoKill-9 $pid; Done        Echo "Kill ${procname} $EXE _user done!";Else    Echo "${procname} $EXE _user do not exist!";fifiRm- F./${procname}. pid

If each timer is used crontab to configure, when the timer more and more time, crontab configuration is not good maintenance, and the key logic is also implemented in Java or C + +, then with the mature quartz framework is quite practical and important, This document does not speak quartz download configuration related knowledge, Google mainly explain its own project should be the scene!

1. Should be scenario asynchronous interface bulk Stop boot
   Background knowledge:
   Operators at the end of every month to do a letter-control batch shutdown, that is, all this is in another letter control system initiated, according to the user's arrears and the user's credit limit of the decision to stop, a stop is the amount of hundreds of watts; The shutdown is a long business process and also relies on external service open system, So the access interface is designed to be asynchronous. First real-time interface to start the list, and then write the table to save, and then launch the quartz timer to quickly re-initiate business logic processing; Of course, the reality of the situation is much more complicated (like a number of simultaneous launch of two stops and the boot order how to deal with ..., failure to resend)
   

After understanding the background, let's talk about the internal quartz settings

1. 1.2
   Programming overview
   

Corresponding to the above, in our system each table to be partitioned, a table will be divided into 4 libraries to save, so each timer job timed cycle build 4 thread threads, corresponding will also start 4 sets of thread pool, to execute runable in the logical processing method, Then the database thread pool is used to update the data processed by each runable logic;
Classes that correspond to the following relationships Orderreceivesonotifyjob.java, Orderreceivesonotifyth.java, and Orderreceivesosoap.java

1. 1.3 Detailed design of the program:
   Job class: Orderreceivesonotifyjob.java

ImportOrg.quartz.Job;ImportOrg.quartz.JobExecutionContext;ImportOrg.quartz.JobExecutionException;ImportCom.linkage.qyw.thread.OrderReceiveSoNotifyTh;ImportCom.linkage.util.PathConfig; Public  class orderreceivesonotifyjob implements Job {  Private Static intCount =0;StaticString Dbnum = Pathconfig.getconfig ("Orderreceiveso.dbnum");//Configurable execution of several library data (1,3,2,4)  //thread pool size  StaticExecutorservice Executorservice = Executors.newfixedthreadpool (Integer.parseint (Pathconfig.getconfig ("Orderreceiveso.threadpool")));protected Static FinalLogger Logger = Logger.getlogger (Orderreceivesonotifyjob.class); Public void Execute(Jobexecutioncontext context)throwsjobexecutionexception {String [] STRs =NULL;if(dbnum!=NULL&&!dbnum.equals ("") {STRs = Dbnum.split (","); for(inti =0; i < strs.length; i++) {NewOrderreceivesonotifyth (Strs[i], Executorservice). Start (); }      }Else{Logger.info ("DB number is null:"+ STRs); }    }}

Thread Class Orderreceivesonotifyth.java

 Public  class orderreceivesonotifyth extends Thread{  PrivateString hostId ="1";PrivateExecutorservice executorreceive =NULL; Public Orderreceivesonotifyth(String hostId, Executorservice executorservice) { This. hostId = HostId; This. executorreceive = Executorservice; } Public void Run()  {//Omit data connection query result code..........//finally each data in the map, check 600 data at a time (in SQL with rownum<600 limit) are installed in the list       for(Map map:list) {logger.info (map);//Logic processing class Orderreceivesosoap implements Runable interfaceOrderreceivesosoap so =NewOrderreceivesosoap (Map,hostid);      Executorreceive.submit (SO); }}

Runable Thread Orderreceivesosoap.java

Too much code is not affixed to this class like the bottom of the pyramid workers (understand into yards farmers are also right), do a bunch of menial work!!
Quartz_job.xml

Cron-expression expression configuration, which is performed every 10 seconds, and the configuration is consistent with Crontab in Linux
Note: thread pool size settings and SQL check how many data is a certain proportion, after repeated testing, 15 threads run 600 data is basically completed in 1 seconds!

To this a complete quartz timer is configured, but the above program is problematic!
problem A: running data repeatedly
Inherit job This timer is executed at the time of execution, regardless of whether the last time it was executed or not! This will result in the last job has not finished, the mark status in the database has not been changed, the next timer started, and took the last time the data is being executed, is to take a number of complex;
Issue B: Extended Timers
If the amount of data that a timer runs is limited, how to ensure that the data can be emitted quickly when the same data is added when multiple timers are run

Job also has a kind of stateful statefuljob interface, if we need to perform the next job execution after the last job execution, according to its execution results, we need to implement this interface; this solves problem A;
But again, if the last timer due to a series of problems (data connection/abnormal data) has not run, the next timer is always unable to execute, equivalent to not be able to timely or delay to the user to stop the boot, imagine a use to remember to find yourself being shut down, immediately the value of the boot, the results of a half-day did not boot,

referencing temporary tables table_tmp
There are 3 benefits of
1. Each check data is associated with table_tmp, to find that only the data is not in the temporary table processing, each time the data found in batches (hand-committed transactions) into the temporary table, time-consuming generally not more than 100 milliseconds

Note: Remember to set the transaction to false and then bulk commit

The advantage of this is that the first will not take the heavy data, followed by unlimited expansion and more than one timer, when the data more time, you can configure more than one job;
When a job has a problem or some unpredictable problem, the next job will be forced to execute the data that has not been processed, as long as each job is a fetch count SQL is associated with the temporary table each timer will no longer fetch duplicate data!

2. The problem of re-sending;
Before doing the re-send or is the operation of the manual update status of the re-issued, or else write a job to update the failure status, but now because of the temporary table, make good use of temporary table role;
When you associate a temporary table fetch, the initial state and the failure state are also associated with the temporary table, which normally does not take out the failed data because the temporary table has relevant data;
Write multiple jobs periodically delete temporary table data, how much time to delete, this and the business of how long it takes to repeat the data consistency; Deleting temporary table data can speed up the efficiency of SQL query and can re-send data, double benefit;

3. What is the concurrency of this pyramid, and can you quickly process bulk data
Solve the above two problems "duplicate data" and "re-issue" problem, with these two weapons can be unlimited increase in the number of jobs, or increase the number of SQL fetch and balance between the job time interval is able to send large quantities of data in a short time! Finally, this real-time, the amount of data into the job interval, the number of thread pool and the number of SQL fetch the balance between the art, this can only be found in practical practice;

finally say the difference between the crontab time configuration below Quartz_job.xml
0/1 * * *?
Run every minute is a full-minute execution, such as 12:01 minutes to execute, next time is 12:02 minutes to execute
215 0/1 * * *?
--Every minute execution is a full-minute delay of 15 seconds execution, such as 12:01:15 minutes to execute, next time is 12:02:15 minute execution

This may have been designed for the following scenario:
If multiple jobs do the same thing, you can use this delay to execute, each job is not the same delay;
If each job in the same second to the database fetch, that is equivalent to only one job can fetch data, first executed first robbed the work, the back of the work, so the wrong labor is still very necessary!

Quartz timer in operator's use