Timing Scheduling and management

There is no jobcenter when we have to face: e-commerce business chain is very long, business logic is also more complex, need hundreds of scheduled tasks. Most of litters's scheduled tasks actually call a local or remote Java/php/python Web Service. Without a unified dispatch and alarm, in the cluster environment, we will: Do not know which scheduled task execution failure or time-out, not necessarily be able to know the first time-until the end user complaints feedback; requires each scheduled task output a unified format of the log for monitoring system resolution. High requirements for each scheduled task maintainer. This is not our problem-solving idea. I do not know which timed task is not well-run, such as forgetting to start the boot, such as the previous line environment B and environment a coexistence caused the timing task to compete with each other; do not know how many scheduled tasks on the line now, what are the responsible person, some timing tasks are very important, can not be a single point, but can not simultaneously   Crontab, can only take Master/slave mode run-such as refund processing.    What is Jobcenter.     Litters's scheduled task management and scheduling platform, a utility, is a distributed multi-task coordination system consisting of task management, task scheduling, task monitoring and alerting, and the execution of the host task (note is no longer crontab). 2012, I saw the Storm Banjarmasin open Source a Cronhub (Time dispatch system) project. Now has no access to its site, you can look at the Baidu library ppt. The requirements described by Banjarmasin are similar to ours:

1, a lot of crontab management good annoying task is always not executed on time, a variety of reasons for failure, really let people crazy. 2, more than one server environment, management crontab more annoying, log on each machine to see crontab results are not torture always lazy programmer. 3, if you can have an automated management, available GUI interface management is good.

So storm video makes a "real universal", "real solution to daily needs" time scheduling system.

Because the previous said most of the scheduled tasks are actually called the Web interface, so our approach and cronhub slightly different, said to be a timed task, in fact, I just registered to call the remote interface, communication protocol, Crontab time format expression, execution machine group, timeout time, alarm receiver, and so on. The interface is notified to the peer on time by Jobcenter and receives the progress feedback and final execution results of the task performer, all in JSON format.   You can also add multiple execution machines for the same scheduled task, Jobcenter to ensure the notification is successful. Jobcenter was built at the beginning of the 2013, during which time there were several research and development projects launched in Nielandbin, such as notifyserver, tracing, Recsys and configserver.      After several months of online trials and improvements, we began to urge research and development organizations to migrate java/php scheduled tasks to this platform. Jobcenter is now also included in our idcenter system, so that you can share a set of account systems (LDAP), sharing a set of rights distribution system:

Figure 1 Jobcenter at the entrance of idcenter its main interface is as follows: Figure 2 Jobcenter The advantages of the main interface Jobcenter: Management is intuitive to specify Worker clusters for timed tasks, and to specify execution policies such as random selection of a machine to perform, such as the first You can specify a notification policy: Ensure that execution succeeds, only once, and you can set the timeout warning time, and you can further set the warning recipient (SMS and Mail), as shown in the following illustration: Task failure sends a message to the recipient of the warning, and the dispatch facilitates a centralized view of the execution of all scheduled tasks, as shown in the following figure: Timed task scheduling interface, pause scheduled tasks, or perform scheduled tasks immediately, and watch to see if it's last execution time, time-consuming, timeout, execution results, and notification results by scheduled tasks. As shown in the following illustration: View its execution trend graph by a timed task, which visually reflects the success, time, and timeout of each execution, as shown in the following illustration: You can drag the magnification timeline with the mouse on the chart, the yellow exclamation mark icon indicates the timeout, the Red Cross chart indicates that the task is not executed;

In short, this is a greatly improved implementation and management efficiency, easy-to-use self-development middleware.

How the Jobcenter works

The following figure is a schematic diagram of the year Nielandbin, although the following structure has been adjusted, but the figure below can explain the problem:

Figure 3 Jobcenter schematic diagram How it dispatches the host to perform timed tasks. As shown in the following illustration:

Figure 4 Swimlane diagram for jobcenter task execution

Jobcenter Notification guarantee Mechanism:

The notification guarantee mechanism has the following 3 kinds: Only one time guarantee success guarantee success (task is not in execution)

In particular, "ensure success (tasks are not in execution)" for the following instructions:

When a task has arrived at this round of notification time, Jobcenter will check the execution of the task, whether it is still executing (as executing, the client is not returning). If there is, then this execution directly fails, without notice.