Construction of Nodejs scheduled task management framework based on agenda

0. Background

In large projects, the application of timed tasks is becoming more and more widespread. In general, according to the idea of microservices, we will deploy a set of services separately, the core business interface independent to another service, thereby reducing the degree of coupling between each other. When you need to use a timed task, you only need to initiate an asynchronous interface call to the core business service by a timed task micro-service.

Based on Nodejs to build such a set of timed tasks, we can use the existing more mature third-party framework to quickly build, developers only need to focus on the implementation of the timing task logic.

1, agenda

There are many kinds of libraries, such as Node-schedule,agenda,node-cron, to search Nodejs schedule on GitHub. The choice of agenda is mainly caused by two reasons: 1, agenda interface is simple, you can also use human-readable cron expression, 2, Agenda provides a complete restful API interface, as well as a statistical view of the UI interface to the operation.

In this case, we just need to define the timed task processing logic in advance in the code, and then we can add or remove the scheduled tasks in the interface without having to go back online.

Let's start by building a agenda-based scheduled task from scratch.

2, Basic Environment preparation

• Mongodb:agenda needs to use MongoDB to store timed task data. We recommend that you use Docker for a built-in installation, easy to operate, and easy to remove. Just search for MONGO in Docker and select install the default recommended official version.
• Agenda: Timed task management framework, documentation reference: Https://github.com/agenda/agenda
• Agendash: Agenda-based UI management interface, document reference: Https://github.com/agenda/agendash
• Express: Used to start Web services with Agendash
• Nirvana-logger: Not required, Nodejs log output frame, document reference: Https://www.npmjs.com/package/nirvana-logger

3. Sample Code

1 //Framework Introduction2 varExpress = require (' Express ');3 varApp =Express ();4 5 varAgenda = require (' Agenda '));6 varAgendash = require (' Agendash '));7 varL = require (' Nirvana-logger ') (' Agenda '))8 9 //agenda initialized and connected to MongoDBTen varAgenda =NewAgenda ({db: {address: ' Mongodb://localhost:32768/agenda '}}); One  A //Define a test task -Agenda.define (' Testjob ',function(Job, done) { -   Try { theL (' Hello ', Job.attrs.data,NewDate ()) - Done () -}Catch(err) { -DoneNewError (err)) +   } - }) +  A //Agenda Frame launch atAgenda.on (' Ready ',function () { -L ("====>>>agenda Startup Success <<<<===") - Agenda.start (); - }) -  - //Agendash UI Interface inApp.listen (3000); -App.use ('/dash ', agendash (agenda));

13 Rows: Define a task, named Testjob, and we can schedule the task by that name later. The second parameter is a function that handles our business logic, and it is important to note that we need to invoke the done method proactively after the task is completed, so that agenda will mark the task as complete.

17 Rows: If an exception occurs during the execution of a task, such as a network request exception, an Error object needs to be passed to done, and agenda will mark the task as a fail state.

23 Line: After the agenda connection MongoDB succeeds, will trigger the ready hook, we can begin to arrange the scheduled task here, and lets agenda start to perform the scheduled task scan.

Line 29: Start Express on Port 3000 and launch the agenda Web management interface using the Agendash framework.

Execute the node command to start the JS file, and if you can see "Agenda boot Success" at the command line, the agenda connection database succeeds and starts successfully.

In fact, the careful classmate can, we just defined the task in the above code, but did not run the task on a timed basis.

So next we'll show you how to query, add, and delete scheduled tasks in the Web interface.

Now we can open: http://localhost:3000/dash/# to view the Web interface.

4. Timed Task Interface Management

The default interface for opening Agendash is this

As you can see from the interface, we do not actually start any scheduled tasks.

4.1 New Scheduled Tasks

Click on "Schedule job" to eject the Create job interface at the back:

JobName: The task names we need to use, that is, the tasks we set up in the program, such as the testjob we just defined

Schedule: Schedule A scheduled task where we enter a point in time that you want the scheduled task to execute, such as 5 minutes. Here time support specific please refer to: https://github.com/agenda/human-interval

Repeat every: cycle through timed tasks, where the cycle time interval is entered. Like 5 seconds. It is important to note that schedule scheduled tasks will only be executed once, and repeat tasks will always be executed in a loop. So the two generally just need to fill in one of the requirements.

Job Data: Additional data passed to the timed task, which we can obtain through Job.attr.data when the task is executed.

After clicking Save, our scheduled task starts to run and you can see the console start looping through the print log:

At the same time the Web interface data will be updated, we can also see the running status of scheduled tasks:

Schedule scheduling a one-time scheduled task is a similar operation, you can try it yourself.

It is worth noting that the tasks that we define are those that can be repeated to perform timed tasks.

4.2 Removing timed tasks

Click on a timed task to see the details of the task and the related actions.

Click "Delete selected" in the upper-right corner to delete the scheduled task.

5. Several points of attention

• Timing task Time In addition to using cron expressions, but also can only be simple English expression, specific reference https://github.com/agenda/human-interval
• The scheduled task will be set to a locked state during execution, and the task will be unlocked after the task execution is completed. So if the task is in the process of execution, the program exits, then it will cause the task is locked, agenda default 10 minutes after the automatic unlock task.
• In order to avoid the recurrence of scheduled tasks, in general, scheduled task microservices We only need to deploy one instance. The core business interface is load balanced, and several instances can be deployed on a per-traffic basis.

Construction of Nodejs scheduled task management framework based on agenda