Quartz.net-based unified task scheduling center

Preface:Quartz. NET is an open-source job scheduling framework and is a. Net port of the quartz API of opensymphony. It is written in C # and can be used in winform and Asp.net applications. It provides great flexibility without sacrificing simplicity. You can use it to create simple or complex scheduling for executing a job. It has many features, such as database support, clusters, plug-ins, and Cron-like expressions.

I am engaged in the communication industry and spent all my time in data heaps. Many of them are tedious and periodic low-tech data extraction and processing work. Most of the required data comes from the unified production database data managed by the province. Data Extraction consumes a lot of manpower. Therefore, we want to deploy crontab + local deployment task plan on the remote bastion host to implement automatic data extraction and push. Crontab implementation is very easy. The trouble is that there is no convenient and powerful tool like crontab locally. In the early stage, I generally used Windows plan + batch processing files, msqql stored procedures + jobs,ProgramScheduled task execution in three ways. However, the three methods have their own advantages and disadvantages. Once the task plan is too large, there are headaches in maintainability, migration and security. So I started to study the information about Task Plan Management in the last few months. net has a very good open-source job scheduling framework, and a senior Zhang Shanyou wrote a lot about quartz. net Course. So the author began to research and transform quartz. net, in order to build an enterprise-level Unified Task Scheduling center similar to the mssql2005 job management function.

After research, the Unified Task Scheduling center has begun to have a model. First, let's look at the audience. I will discuss the transformation process with you later.

Figure 1: Task Management

 

Figure 2: Task steps

Figure 3: Task Plan

Figure 4: Task notification

 

At present, the scheduler basically implements the Job scheduler function of mssql2005 (the function is slightly weaker, and the advantage is that it supports crontab). The task step only implements the execution of SQL statements (supporting multiple databases ), FTP upload/download. Quartz. Net runs in the database + winservice mode.

The following describes the problems I encountered during the study:

Question 1: How to store job information of quartz.net in a database.

Quartz.net does not provide much information, especially when running in databases. Later, I carefully studied the example of quartz.net source code to solve the problem.CodeAs follows (because I do not know how to configure the configuration file, set the corresponding configuration variables in the program in the example ):

Protected override void onstart (string [] ARGs)
{
Logger. Info ("Start service, time:" + datetime. Now. tostring ());
Startjobs ();
}

Private void startjobs ()
{
Namevaluecollection properties = new namevaluecollection ();
Properties ["quartz. jobstore. driverdelegatetype"] = "quartz. impl. adojobstore. sqlserverdelegate, quartz ";
Runadojobstoretest ("SqlServer-20", "sqlserver", properties );
}

Private void runadojobstoretest (string dbprovider, string connectionstringid,
Namevaluecollection extraproperties)
{
Namevaluecollection properties = new namevaluecollection ();

Properties ["quartz. schedame. instancename"] = "testscheduler ";
Properties ["quartz. scheduler. instanceid"] = "instance_one ";
Properties ["quartz. threadpool. Type"] = "quartz. simpl. simplethreadpool, quartz ";
Properties ["quartz. threadpool. threadcount"] = "10 ";
Properties ["quartz. threadpool. threadpriority"] = "normal ";
Properties ["quartz. jobstore. misfirethreshold"] = "60000 ";
Properties ["quartz. jobstore. Type"] = "quartz. impl. adojobstore. jobstoretx, quartz ";
Properties ["quartz. jobstore. driverdelegatetype"] = "quartz. impl. adojobstore. stdadodelegate, quartz ";
Properties ["quartz. jobstore. useproperties"] = "false ";
Properties ["quartz. jobstore. datasource"] = "default ";
Properties ["quartz. jobstore. tableprefix"] = "qrtz _";
Properties ["quartz. jobstore. Clustered"] = clustered. tostring ();
Properties ["quartz. datasource. Default. connectionstring"] = "Server =.; database = quartz; trusted_connection = true ;";

If (extraproperties! = NULL)
{< br> foreach (string key in extraproperties. keys)
{< br> properties [Key] = extraproperties [Key];
}< BR >}

If (connectionstringid = "sqlserver" | connectionstringid = "SQLite ")
{< br> // if running ms SQL Server WE NEED THIS
properties ["quartz. jobstore. lockhandler. type "] =
" quartz. impl. adojobstore. updatelockrowsemaphore, quartz ";
}

// properties ["quartz. datasource. default. connectionstring "] = (string) dbconnectionstrings [connectionstringid];
properties [" quartz. datasource. default. provider "] = dbprovider;

// first we must get a reference to a scheduler
ischedulerfactory Sf = new stdschedulerfactory (properties);
sched = SF. getscheduler ();
string [] triggergroups = sched. triggergroupnames;
sched. start ();
// thread. sleep (600*1000);

}

Protected override void onstop ()
{
Sched. Shutdown (true );
Logger. Info ("Stop Service, time:" + datetime. Now. tostring ());
}

 

Question 2:After the trigger is executed, it is deleted from the database table by the quartz.net service.

Generally, you do not want to delete a scheduler after you add a scheduler using an MSSQL Job. Therefore, I have transformed quartz.net. Use a stupid method to comment out the method for deleting a trigger. Comment out the removetrigger method of connectionandtransactionholder under quartz. impl. adojobstore so that it returns true forever.

Question 3: Implement the quartz.net plan in the form of MSSQL Job plan.

Solution: analyze the crontab expressions carefully. Analyze the crontab expressions by day, by week, or by month, and display them to users through the UI, stored in crontab or simple mode.

Question 4:How to start the task immediately.

It is actually relatively simple to start a task immediately. You only need to add a trigger to be executed immediately, but you need to not display the trigger in task plan management, therefore, you can hide trigger triggered immediately or clear trigger triggered immediately after execution.

 

In addition, it also improves task logs, adds task groups, manages task steps, and extends task attributes, such as creation time and status. Due to the limited knowledge and experience of the author, the solution may not be perfect. I hope that you will not be enlightened by the experts.