Shell script implements file lock function _linux Shell

1. Background

When multiple processes may perform operations on the same data, these processes need to ensure that the other processes are not operating to prevent damage to the data. Typically, such a process uses a "lock file", which means that a file is created to tell other processes that they are running, and that a process that operates the same data is considered to be working if the file is detected. The problem is that the process accidentally dies and does not clean up the lock file, so it can only be cleaned manually by the user.

2. About Flock

Flock is a recommended lock for the entire file. That is, if a process is locked on a file (inode), then other processes can be known. (The proposed lock does not compel the process to comply.) Best of all, its first argument is the file descriptor, which is automatically released when the file descriptor closes. When the process terminates, all file descriptors are closed.

3. The command to implement the flock system call in the shell is flock, which uses the following two types (man flock)

Copy Code code as follows:

Flock [-sxon] [-w timeout] lockfile [-c] command ...
Flock [-sxun] [-w timeout] fd

Options and Parameters:

-S,--shared: Gets a shared lock, during the time that a shared lock was set on an FD that was directed to a file without releasing the lock, another process attempted to set an exclusive lock on an FD directed to this file and the other process attempted to set a request for a shared lock on the FD directed to this file to succeed.
-x,-e,--exclusive: Gets an exclusive lock, or write lock, as the default
-u,--unlock: Manually release the lock, generally do not have to, when the FD shutdown, the system will automatically unlock, this parameter used in the script command part of the need for asynchronous execution, part of the situation can be synchronously executed.
-N,--NB,--nonblock: non-blocking mode, which returns 1 instead of waiting when a lock failure is acquired
-W,--wait,--timeout seconds: Sets the blocking timeout, exits blocking mode when the set number of seconds is exceeded, returns 1, and continues execution of the following statement
-O,--close: means to turn off the FD of a set lock before executing the command so that the child process of the command does not remain locked.
-C,--command command: Executing subsequent statements in the shell

4. Implement exclusive lock in Shell to avoid repeated execution of scripts

Routine work scheduling in Linux crontab executes some scripts at timed intervals, but the execution times of the scripts are often beyond control, and when the script executes too long, the script for the last task is not finished, and the next task's script starts again. There may be some concurrency problems in this case, which can lead to a vicious cycle of dirty data/performance bottlenecks.
You can circumvent this problem by using flock to establish exclusive locks, and if a process has an exclusive lock on it, the other process cannot lock it, optionally waiting for a time-out or returning immediately. The test examples are as follows:

4.1 Creating execution Scripts

Copy Code code as follows:

#cat/scripts/shell/file_lock.sh
#!/bin/bash
# Description:test for file flock
Path=/bin:/sbin:/usr/bin:/usr/sbin:/usr/local/bin:/usr/local/sbin:~/bin
Export PATH

echo ""
echo "----------------------------------"

echo "start at ' date ' +%y-%m-%d%h:%m:%s ' ..."

Sleep 140s

echo "finished at ' date ' +%y-%m-%d%h:%m:%s ' ..."

4.2 Create timed task: Test exclusive lock

Copy Code code as follows:

#crontab-E
* * * * flock-xn/dev/shm/test.lock-c "sh/scripts/shell/file_lock.sh >/root/stdout.log" * * * * * * * *

Executes the script once every minute and writes the output information to the Stdout.log

View the output log as follows:

Copy Code code as follows:

----------------------------------
Start at 2014-04-10 10:23:01 ... #获取锁
Finish at 2014-04-10 10:25:21 ... #释放锁


----------------------------------
Start at 2014-04-10 10:26:01 #10:27:00 and 10:28:00 Scheduled tasks that were started because the lock could not be acquired, failed to exit execution, and the lock was not acquired until 10:26:00
Finish at 2014-04-10 10:28:21 ...

4.3 test exclusive lock, plus wait timeout

Copy Code code as follows:

* * * * * * flock-x-W 20/dev/shm/test.lock-c "sh/scripts/shell/file_lock.sh >/root/stdout.log"

To view log output information:

Copy Code code as follows:

----------------------------------
Start at 2014-04-10 10:29:01 ...
Finish at 2014-04-10 10:31:21 ...

----------------------------------
Start at 2014-04-10 10:31:21 #10:31:00 The scheduled task started after 20 seconds after the last task freed the lock, so this task can immediately get the lock and continue execution
Finish at 2014-04-10 10:33:41 ...