Z scheduler blocking principle and Solution

The quartz scheduler executes the scheduling task jobdetail in multi-thread mode. The default thread pool size is 10. That is to say, if 10 jobs in the scheduler are working (the thread is not finished ), even if jobdetail is triggered, the new jobdetail will not be executed. That is to say, the blocking condition is that the number of jobdetail running in the scheduler has reached the set value of 10.

 

Here is a specific example:

A. single job

Configuration:

The Joba trigger time is run once per second, and the execution time of each job is 30 seconds.

 

Run:

1. 10 Joba will be started continuously

2. After 10th Joba is started, all threads in the thread pool are exhausted, and the scheduler is blocked. That is, no new Joba is started, even if it is set to run once per second.

3. After 30 seconds, more than one Joba will be executed. In a short period of time, 10 new Joba will be started again and enter the blocking status of 2 again.

 

2. The status can be called the blocking status of the scheduler, and no new job can be executed. As a result, some operations such as regular data reading cannot continue. The new Joba can be executed only after Joba is executed. In actual operation, if the execution time of one Joba thread in the scheduler is greater than the two startup intervals, after several operations, all 10 thread resources will be exhausted, other scheduling tasks are blocked.

 

B. Multiple jobs (stateless jobs)

In this test, there can be a variety of different jobs (stateless jobs), but they all share these 10 threads, when the execution time of any job thread exceeds the two startup intervals, the scheduler may be blocked. For example:

Configuration:

The Joba trigger time is run once per second, and the execution time of each job is 30 seconds. The jobb trigger time is run once per second, and the execution time is less than 1 second.

 

Run:

1. Start Joba and jobb one after another

2. After a few seconds, the number of Joba instances reaches 10. When jobb is executed several times, the new Joba and jobb instances cannot be started, and the scheduler enters the blocking status.

3. 30 seconds later, Joba (0-9) was successively executed, and the new Joba and jobb both had the opportunity to be restarted. Within a short period of time, they again entered the blocking status of 2.

 

How to solve the scheduler blocking problem?

1. Extend the interval between jobs that may take a long time to execute. If the maximum execution time of a job is T1 and the interval between two jobs is D1, then D1> T1

2. Use statefuljob to replace stateless jobs. It is suitable for scenarios where the execution interval is as short as possible without blocking. A job with or without status can be scheduled at the same time, and a job with status scheduling job jobb. After jobb is blocked, it will not affect Joba, that is, the job that reads the report is blocked, it does not affect other tasks in the scheduler. After Joba is executed, you can continue to the next task again.

If Joba has a long execution time, Joba may always occupy one thread resource.

3. Note: If a scheduler contains many jobs (Joba, jobb, jobc ...), if one of them is easily congested, the job will also cause other jobs to be congested.

 

The thread pool size is configured in the quartz. properties file under org. Quartz.

Org. Quartz. threadpool. threadcount = 10

To modify the thread pool size, you can modify
The value of org. Quartz. threadpool. threadcount. You can also create an org. Quartz package and place the quartz. properties file under the package to overwrite the configuration in quartz. jar.

However, modifying the thread pool size does not solve the scheduler blocking problem, because resources are exhausted when the resource consumption speed is lower than the resource release speed.