SQLite introduction and analysis (6)-talk about the SQLite lock

Preface: the implementation of the SQLite blocking mechanism requires the support of the underlying file system. Both Linux and Windows provide the file lock mechanism, which provides powerful support for SQLite. This section describes the file locks used by SQLite, which are mainly based on Linux and Windows platforms.

 

• Linux File lock

The file lock technologies supported by Linux mainly include advisory lock and mandatory lock. In addition, two forms of forced lock are introduced in Linux: share-mode mandatory lock and lease lock ). Here, we mainly discuss the advisory lock ).
The recommended locks are not enforced by the kernel. That is to say, if a process does not comply with the "game rules", it does not check whether the target file has been locked by another process and write data to it, then the kernel will not be blocked. Therefore, it is recommended that the lock should not prevent the process from accessing the file, but should only rely on each process to check whether the file has been locked by other processes before accessing the file for concurrent control. The process requires a prior agreement on the lock status, and determines whether other processes can perform the specified operation on the file based on the current status and relationship of the lock. The Force lock is a file Lock forced by the kernel. Because the kernel checks the corresponding lock for each read () and write () operation, the system performance is reduced.
For recommended locks, Linux provides two implementation methods: lock files and record locking ).

(1) lock files)
Lock files are the simplest method to lock files. Each data file to be locked has a lock file ). When the lock file exists, it is deemed that the data file has been locked, and other processes should not access it (but you have to access it, and Linux will not block it ). When the lock does not exist, the process can create a lock file and access the corresponding data file. As long as the lock creation process is atomic, only one process can have the lock at a time point. This method ensures that only one process accesses the file at a time point.
This idea is simple. When a process wants to access a file, it can lock the file as follows:

Fd = open ("somefile. lck", O_RDONLY, 0644 );
If (fd> = 0 ){
Close (fd );
Printf ("the file is already locked ");
Return 1;
} Else {
/* The lock file does not exist, we can lock it and access it */
Fd = open ("somefile. lck", O_CREAT | O_WRONLY, 0644 ");
If (fd <0 ){
Perror ("error creating lock file ");
Return 1;
}
/* We cocould write our pid to the file */
Close (fd );
}

After a process processes the file, it can call unlink ("somefile. lck") to release the lock-essentially deleting the somefile. lck file.
The above code is actually competing because the if statement block is not atomic and enters the if statement block. The kernel may schedule other processes to run. The better way is as follows:

Fd = open ("somefile. lck", O_WRONLY | O_CREAT | O_EXCL, 0644 );
If (fd <0 & errno = EEXIST ){
Printf ("the file is already locked ");
Return 1;
} Else if (fd <0 ){
Perror ("unexpected error checking lock ");
Return 1;
}

/* We cocould write our pid to the file */
Close (fd );

The O_EXCL flag ensures that the process of creating a lock file in open () is atomic.
Note the following:
1. Only one process can have a lock at any time.
2. The O_EXCL flag is only reliable for the local file system and does not support network file systems.
3. The lock is only built.
4. If a lock-holding process ends abnormally, the lock file will still exist. If the pid of the lock process is stored in the lock file, other processes can check whether the lock process exists and delete the lock when it ends. However, during the check, if the pid is used by other processes, there is no way to do it.

(2) Record Locking)
To overcome the disadvantages of lock files, System V and BSD4.3 introduce record locks, and the corresponding System calls are lockf () and flock (). POSIX provides another mechanism for record lock, and its system call is fcntl (). Linux provides three interfaces. Here we only discuss POSIX interfaces.
There are two important differences between record locks and lock files: first, record locks can lock any part of the file-this is of great help to applications such as DBMS, of course, SQLite does not have such advantages. Second, another advantage of record lock is that it is held by the kernel rather than by the file system. When the process ends, all locks are released.
Like locking files, POSIX locks are also built. Record locks include read locks and write locks ). The read lock is the shared lock, and the write lock is the exclusive lock ). For one record, only one process can hold the write lock, and the read lock cannot exist.
For a process itself, multiple locks will never conflict. If a process holds a read lock on-bytes of the file and then writes a write lock on-bytes of data, it will succeed. At this time,-2-25 is the write lock, while-2-50 bytes of data is the read lock. This rule mainly prevents deadlocks on the process itself (Although deadlocks may still occur between multiple processes ).

POSIX locks are implemented through the fcntl () system as follows:

 

# Include <fcntl. h>

Int fcntl (int fd, int command, long arg );

 

Arg is the pointer to the flock struct:

# Include <fcntl. h>

Struct flock {
Short l_type;
Short l_whence;
Off_t l_start;
Off_t l_len;
Pid_t l_pid;

};

In the flock Structure, Rochelle type is used to specify whether a shared lock or exclusive lock is created. The options include F_RDLCK and F_WRLCK) and F_UNLCK (delete the previously created lock); l_pid specifies the owner of the lock; l_whence, l_start, and l_end fields indicate the region of the file to be locked by the process, this area is a continuous collection of bytes. Therefore, a process can apply different locks to different parts of the same file. L_whence must be one of the SEEK_SET, SEEK_CUR, or SEEK_END values, which correspond to the file header, current position, and end. Rochelle whence defines the offset relative to Rochelle start. Rochelle start is calculated from the file.

You can perform the following operations:

* F_GETLK: The process can use it to obtain the lock information of the file opened through fd. When this operation is performed, the lock points to the structure that stores the lock (or the lock to be queried) You want to add to the file ). If such a lock does exist, it prevents the lock descriptor given by the flock structure to which the lock points, then the information of the existing lock is written into the flock Structure pointed by the lock, write the PID of the lock owner to the l_pid field and return the result. Otherwise, set the l_type in the flock structure to F_UNLCK and keep other information in the flock structure unchanged, instead of locking the file.
* F_SETLK: The process uses it to lock a certain area of the file (the value of l_type is F_RDLCK or F_WRLCK) or delete the lock (the value of l_type is F_UNLCK ), if other locks prevent the lock from being created, an error is returned in fcntl ().
* F_SETLKW: similar to F_SETLK, the only difference is that if other locks prevent the lock from being established, the calling process enters the sleep state and waits for the lock to be released. Once this call starts to wait, it will be returned only when the lock can be performed or when the signal is received.

It should be noted that F_GETLK is used to test whether locks can be applied. After F_GETLK is used to test whether locks can be applied, F_SETLK and F_SETLKW attempt to create a lock, but the two are not an atomic operation, that is to say, before F_SETLK or F_SETLKW is successfully locked, another process may have been inserted with a lock. Moreover, F_SETLKW may cause the program to sleep for a long time. In addition, the various locks owned by the Program on a file will be automatically cleared when the corresponding file descriptor is closed. After the program runs, the various locks it adds will be automatically cleared.

• File lock in Windows

In Windows, all locks are mandatory locks. In Windows, shared files are managed through the following mechanisms:
(1) through shared access control, the application can specify the entire file for shared read, write, or delete.
(2) You can use the byte range locks to read and write a part of a file.
(3) Windows file systems do not allow opening or deleting ongoing files.
The file sharing method is determined by the sharing mode parameter in CreateFile (), the open file function in WIN32 API:

HANDLE CreateFile (
Lptstr lpFileName,
DWORD dwDesiredAccess,
DWORD dw1_mode,
LPSECURITY_ATTRIBUTES lpSecurityAttributes,
DWORD dwCreationDisposition,
DWORD dwFlagsAndAttributes,
HANDLE hTemplateFile
);

The value of dwShareMode is generally:
File_pai_delete:
Enables subsequent open operations on an object to request delete access.
Otherwise, other processes cannot open the object if they request delete access.
If this flag is not specified, but the object has been opened for delete access, the function fails.
File_cmd_read:
Enables subsequent open operations on an object to request read access.
Otherwise, other processes cannot open the object if they request read access.
If this flag is not specified, but the object has been opened for read access, the function fails.
File_pai_write:
Enables subsequent open operations on an object to request write access.
Otherwise, other processes cannot open the object if they request write access.
If this flag is not specified, but the object has been opened for write access, the function fails.

Specific implementation functions:

BOOL LockFile (
HANDLE hFile,
DWORD dwFileOffsetLow,
DWORD dwFileOffsetHigh,
DWORD nNumberOfBytesToLockLow,
DWORD nNumberOfBytesToLockHigh
);

 

 

• Precautions for SQLite blocking Mechanism

The lock byte of SQLite is defined as follows:

Code
# Define PENDING_BYTE 0x40000000/* First byte past the 1 GB boundary */
# Define RESERVED_BYTE (PENDING_BYTE + 1)
# Define SHARED_FIRST (PENDING_BYTE + 2)
# Define SHARED_SIZE 510

(1) Why is PENDING_BYTE set to 0X4000 0000 (1 GB )?
In Windows files, the locked area does not require data, and it will prevent all processes from writing files to this area, including the first process holding the lock. to prevent errors caused by read/write operations on pages containing mandatory lock (which is not allowed in Windows), SQLite completely ignores pages containing pending bytes, therefore, pending byte generates a "file hole" in the database file ". If PENDING_BYTE is set so high, most database files will not experience space loss caused by "file holes" generated by PENDING_BYTE (unless the file size is particularly large and exceeds 1 GB ).

(2) For Windows, the LOCK areas in the file cannot overlap, in order to make the two read processes can access the file at the same time, for shared lock select a random number within the range of SHARED_FIRST--SHARED_FIRST + SHARED_SIZE, therefore, the two processes may obtain the same lock byte. Therefore, for Windows, the concurrency of SQLite is limited.

 

From: http://www.cnblogs.com/hustcat/