Alternative crontab, mission planning unified centralized Management system Cronsun Introduction

I. BACKGROUND

Crontab is the most simple and easy-to-use timing task management tool in Linux, which is believed to be used by most development and operations. In our company, many of the business system's timing tasks are defined by crontab, long time will find that there are many problems:

• A large number of crontab tasks are scattered across servers, resulting in high maintenance costs
• The task was not executed on time, and even failed for a long time to discover that it needed to retry or troubleshoot
• Crontab scattered in many clusters, need a one to see the log analysis, the head is big
• Crontab there is a single point of issue, it is very nerve-racking for timed tasks that cannot be repeated
• I x,crontab was accidentally deleted, no backup? Nima!
• I Q, server to migrate, crontab on the history of the task is what ghost? I didn't even know when I asked a lap.
• ...

Therefore, we need a centralized management of the scheduled task system, I believe this is also suffering from crontab suffering from the operation or development of the voice.

Second, choose

An open source project: Cronsun, the main character introduced in this article, through the trial, found very fit with our current usage scenarios, introduced as follows:

Cronsun is a distributed task system with crontab approximation on a single node and on a Linux machine. is to solve the problem of crontab task management on multiple Linux machines, while providing high-availability support (when a node freezes, it can automatically dispatch to normal node execution). Support interface management tasks on the machine, support task failure email reminders, easy to install, easy to use, is a good choice to replace Crontab.

Github Address: Https://github.com/shunfei/cronsun

The deployment architecture for Cronsun is as follows:

                                                [Web]                                                  |                                     --------------------------           (add/del/update/exec jobs) |                        | (Query job exec result)                                   [ETCD]                 [MongoDB]                                     |                        ^                            --------------------              |                            |        |         |              |                         [Node.1]  [NODE.2]  [NODE.N]         |             (Job exec fail) |        |         |              |          [Send mail]<-----------------------------------------(Job exec result)

III. deployment

This article mainly introduces the function, here simply write down the key steps (Installation deployment use is very good):

1, the installation of MongoDB, strongly recommend the use of cluster mode

2, the installation of ETCD3, strongly recommend the use of cluster mode

3. Deploy Cronsun

①, download cronsun:https://github.com/shunfei/cronsun/releases (choose the latest version)

②, after decompression, modify the configuration files in the Conf directory: Db.json and Etcd.json, respectively, modify the actual address of MongoDB and ETCD.

③, start web:./cronweb-conf Conf/base.json (use Nohup if you want to run in the background)

④, start node:./cronnode-conf Conf/base.json (use Nohup if you want to run in the background)

⑤, Access front desk: http://x.x.x.x:7079/ui/

4, the deployment of authentication components Aproxy,cronsun in the authentication of the very rough, so here to use the Cronsun team developed Aproxy authentication components, the implementation of the principle is based on the Go language, reverse proxy back-end WEB, to achieve Domain name and page address access control, Introduction Address: https://www.cnblogs.com/QLeelulu/p/aproxy.html

We are going to use the production environment here, so we will focus on the availability and reliability of the deployment, here is a map of our deployment architecture, for reference:

Ps: The new version is currently supported for regular cleanup of history logs.

Here, ETCD and MongoDB Reuse 5 servers (which will continue to reuse other common components), where MongoDB takes a shard + replica set pattern.

Four, the function

Once the deployment is complete, you will be able to access the front desk to see the poor UI Cronsun manage the WEB:

Ps: After selecting the familiar language in the upper right, it is essential to follow the task of adding the tasks according to the page label.

1. Adding nodes

Cronsun based on the ETCD implementation of the automatic discovery and registration functions, so the addition of nodes is very simple, directly copy the Cronnode and conf to the client server startup, you can see the server on the front-end node page, of course node and ETCD and MongoDB Network must be unblocked.

2. Node grouping

After adding the required node servers, we can group the nodes to facilitate the addition of the Scheduled tasks:

3. Adding tasks

Once the nodes and the groupings are done, we can start adding timed tasks. The information you fill out on a timed task is slightly more complex, but you can easily get it done by following the prompts:

I have simply highlighted some of the areas that need to be specifically explained, and the other options are self-explanatory to the Chinese description. Of course, it is also important to note that the task script must have execute permissions, or the task will fail to execute.

4. Task List

Once you've added a task, you'll see all the added scheduled tasks and executions on the Tasks tab, where you can filter the tasks you care about by using group filtering or node filtering.

There are 3 small buttons on the right side of each task:

①, Success/failure: Shows whether the execution of the most recent task was successful or failed, and then you can view the task details, including the task output data:

②, Latest button: Click to view the recent implementation of the change task

③, Refresh Symbol button: Click to eject the immediate execution function, easy to debug tasks

V. Summary

Through a period of gray-scale trial, you can determine the Cronsun in the small and medium-sized scenes, is a better alternative to crontab, it can help operators out of the crontab difficult to manage, difficult to transport the suffering.

Of course, as an open-source product, Cronsun a lot of functional details there is a great increase in space, I am also with the Cronsun team for a long time to keep in touch with the production environment in the process of using some of the problems and suggestions one by one feedback, I believe this product can continue grinding optimization, more perfect, perfect.

Vi. Issues and updates "continuous"

Issue record:

1. Alarm Configuration

First of all, it is clear that the Cronsun alarm is issued by Cronweb, not Cronnode (but Cronnode Mail.json must also enable:true, otherwise it will not be able to issue an alarm).

Next, edit the configuration files for Cronweb and Cronnode: Mail.json, as follows

{    "Enable": True,    "to": ["Fill in the default default alarm mail receiving Address"],    "#HttpAPI": "If this field is sent by HTTP API,"    #Keepalive ":" If no mail is sent during this time period, close the SMTP connection, units/sec ",    " Keepalive ":"    #doc ":" https://godoc.org/github.com/go-gomail/gomail# Dialer ",    " Host ":" Fill in the SMTP server address, such as: stmp.qq.com ",    " Port ":"    Username ":" Here fill in the sender email address (for login smtp+from address) " ,    "Password": "Login Password",    "SSL": false,    "LocalName": ""}

Ps:localname suggest leave blank, Httpapi mode is not used, here omit, please test yourself.

Email alerts can be realized by launching cronweb at the end.

Of course, you also need to open the alarm on the one-way task interface on the web, as shown in:

The ps:cronweb must be filled in the Mail.json configuration to True to see the alarm switch button, otherwise it is not displayed.

2. Update record

See the GitHub release page: https://github.com/shunfei/cronsun/releases

Important Feature Updates:

①, script parameters are supported;

②, has supported the history log regular cleanup;

③, version 3.1 begins using the UUID as the node's unique identity.

Original address: 80621384

Alternative crontab, mission planning unified centralized Management system Cronsun Introduction