Gotorch-Multi-machine timing task management system

This is a creation in Article, where the information may have evolved or changed.

Objective

Recently in the Learning Go language, follow the "learn a language the best way is to use it" concept, thinking about what to achieve with go, just work there has been a comparison let me worry about the problem, so with go to solve, even if not in the production environment to use, but also as go language learning a way.

Let's introduce the following questions:

There are ten machines in the group, the above with cron to execute some scripts and shell commands, at the beginning of the task less time, we all remember which machine executes what, with the passage of time, the staff changes several times, the task is more and more, no one can remember clearly which tasks on which machines are executed, The problem of troubleshooting and resolving background scripts is getting more and more cumbersome.

There is no way to solve this problem:

• Maintain a wiki, update the wiki once the task has changed, but once you forget to update the wiki, the task will become orphaned, and when the problem is more difficult to check.
• Set up a machine, timed to pull the cron configuration file of each machine, compare statistics, and then summarize the results, but the command of a variety of styles, contrast command is also a matter of no mind.
• The use of open-source distributed task scheduling tasks, relatively heavy, and generally to lay out the database, backstage, more trouble.

In addition, the task of modification is very inconvenient, if you want to modify a task in the crontab, but also need to find operations operation. Although there are ways to solve this problem, use crontab cronfile.txt directly to let crontab load the file, but introduce a new problem: task file loading real-time bad control.

In order to solve the above problems, I combine cron and task management, spend a little time after work every day, realize a small function, finally completed the Gotorch of the available version. Look at GitHub's commit stats, there is a sense of accomplishment ~

Put the GitHub link address here: GitHub-zhenbianshu-gotorch Welcome star/fork/issue .

Introduce features:

• cron+, second-level timing, make task execution more flexible;
• Task List file path can be customized, it is recommended to use version control system;
• Built-in log and monitoring system, convenient for you to expand any students;
• Smooth reload the configuration file, once the configuration file changes, without affecting the task being performed, smooth loading;
• IP, maximum number of executions, task type configuration, support more flexible task configuration;

Here is the technical point of the feature implementation:

Article welcome reprint, but please bring this article source address: http://www.cnblogs.com/zhenbianshu/p/7905678.html, thank you.

cron+

In the implementation of similar cron function, I simply looked at the cron source code, the source code in https://busybox.net/downloads/can be downloaded, extracted after the file miscutils > crond.c .

The cron implementation is designed to be ingenious, presumably as follows:

Data:

1. Cron has a global structure that holds a list of tasks for each user;
2. Each task list is a structure cronfile, saving the user name and task chain list, etc.
3. Each task Cronline has the shell command, executes the PID, executes the time array cl_time and so on the property;
4. The maximum length of the execution time array is determined by the maximum value of the "time-of-day Week", and the value of the executable point is set true , for example, at 3 points daily cl_Hrs[3]=true ;

Execution mode:

1. Cron is a while(true) long loop of one type, each time sleep to the beginning of the next minute.
2. Cron iterates through the user cron configuration file at the beginning of every minute, parses the updated profile into a task into the global structure, and periodically checks to see if the configuration file has been modified.
3. Cron then resolves the current time to 第 n 分/时/日/月/周 , and determines that cal_time[n] is all true to perform the task.
4. The PID write is prevented from executing when the task is executed;
5. Subsequent Cron also carries out some exception detection and error handling operations.

Understand the cron execution mode, the sense that each unit of time to traverse the task to judge the performance of the loss, and I realized that the second level of execution, the performance of the Traverse judgment loss is greater, so consider optimization into:

Set a next_time timestamp for each task and update the timestamp after one execution, each time unit only needs to be judged task.next_time == current_time .

Later, because the "second time of day and month" date format carry irregular, the code is too complex, realize efficiency is not better than the original, finally gave up the idea. Adopt the same execution idea as Cron.

In addition, I added three ways to limit task execution:

• IP: Gets the Local intranet IP when the service is started and checks whether the pre-execution is in the IP list of the task;
• Task type: The task is daemon, when the task is not executing, the interrupt judgment is started directly;
• Maximum execution: Sets the slice of the PID component of a task in execution on each task, verifying the current number of executions before each execution.

and the task start mode, then use Goroutine directly with the exec package, each time the task is started a new goroutine, save the PID, and error handling. Since the service may scan the task more than once in a second, I have added a property for each task to the last execution timestamp of the process to be compared to the next execution to prevent the task from being scanned multiple times within a second.

Daemon process

This service is made of a similar nginx service, I will process of the PID saved in a temporary file, the process operation through the command line to send a signal to the process, only need to pay attention to the abnormal situation in time to clean up the PID file is good.

Here's how the Go daemon is created:

Since the Go program runtime may create multiple threads at startup (for memory management, garbage collection, goroutine management, etc.), and the fork and multithreaded environments do not coexist harmoniously, there is no fork method in the Unix system in go; so I started the daemon, I used exec. Immediately after execution, that fork and exec 's the way, while the GO exec package supports this approach.

At the beginning of the process gets and determines whether the process Ppid is 1 (the daemon's parent process exits, the process is "adoptive" to the Init process, its process number is 1), the process number of the parent process is not 1 o'clock, using all the parameters of the original process fork and exec a process that is the same as itself, close the new process and terminal contact, and exits the original process.

    filePath, _ := filepath.Abs(os.Args[0]) // 获取服务的命令路径    cmd := exec.Command(filePath, os.Args[1:]...) // 使用自身的命令路径、参数创建一个新的命令    cmd.Stdin = nil    cmd.Stdout = nil     cmd.Stderr = nil // 关闭进程标准输入、标准输出、错误输出    cmd.Start() // 新进程执行    return // 父进程退出    

Semaphore processing

After making the process as a daemon, the process communicates with the outside world to rely on the signal volume, Go signal package with Goroutine can easily listen to, processing the semaphore. At the same time we use the Kill method in the Syscall package to send semaphores to the process.

We listen to Kill the semaphore sent by default SIGTERM , to handle the cleanup before the service exits, and I also use a user-defined semaphore SIGUSR2 to be used as a terminal notification Service restart message.

The complete flow of a semaphore from the hearing capture to the processing is as follows:

1. First we used to create a type os.Sygnal of unbuffered channel to hold the semaphore.
2. Use the signal.Notify() function to register the semaphore to be monitored, pass in the channel you just created, and receive the semaphore when the semaphore is captured.
3. Creates a goroutine that blocks when there is no signal in the channel signal := <-channel .
4. Once the Go program captures the semaphore being monitored, it passes the semaphore over the channel, and the Goroutine does not continue to block.
5. The corresponding semaphore is processed by the following code.

The corresponding code is as follows:

    c := make(chan os.Signal)    signal.Notify(c, syscall.SIGTERM, syscall.SIGUSR2)     // 开启一个goroutine异步处理信号    go func() {        s := <-c        if s == syscall.SIGTERM {            task.End()            logger.Debug("bootstrap", "action: end", "pid "+strconv.Itoa(os.Getpid()), "signal "+fmt.Sprintf("%d", s))            os.Exit(0)        } else if s == syscall.SIGUSR2 {            task.End()            bootStrap(true)        }    }()

Summary

The development of Gotorch spent three months, half an hour or so each day, with a commits, experienced three major refactoring, especially in the code format is more frequent. But using go development is really comfortable, the code of Go is very concise, gofmt is very convenient. In addition, Go learning curve is quite smooth, familiar with the common standard package can be a simple development. Easy to learn, efficient and fast, no wonder go hot so fast.

About this article has any questions can be in the following message exchange, if you think this article is helpful to you, 推荐 You can click on the following support me, blog 关注 has been updated, welcome.

Reference:

On the fork () function and multithreaded programming in Linux

Signal detailed description of the Linux signal volume