Synchronize kernel buffer sync, fsync, and fdatasync functions, fsyncfdatasync

Synchronize kernel buffer sync, fsync, and fdatasync functions, fsyncfdatasync
Synchronize the kernel buffer 1. introduction to the buffer zone one of the three illusion of life: when calling the function write (), we think that once the function is returned, the data has been written into the file. however, this concept is only macro-level. in fact, when the operating system implements some file I/O (such as disk files), in order to ensure the I/O efficiency, A dedicated area (memory or independent I/O address space) is usually used in the kernel as the I/O data buffer. it is used between the input and output devices and the CPU to cache data, so that low-speed devices and high-speed CPUs can work together to avoid low-speed input and output devices occupying CPU for a long time and reduce system calls, CPU efficiency is improved.
2. non-synchronous write () Traditional UNIX or LINUX systems use the kernel buffer during design, with a high-speed buffer or high-speed page buffer, most disk I/O are carried out through the buffer. when writing data to a file, the kernel usually copies the data to one of the buffer zones. If the buffer zone is not full, it is not discharged into the output queue, instead, wait until it is fully written or when the kernel needs to reuse the buffer to store data from other disk blocks, and then discharge the buffer into the output queue. Then, when it reaches the queue, to perform the actual I/O operation. this output mode is called delayed write.
When the write () function is called to write data, the function returns immediately once the data is written to the buffer zone (key: write only to the buffer zone. at this time, the written data can be read back using read () or read by other processes, but it does not mean that they have been written to the External Permanent storage media, even if close () is called () this may also happen after the file is closed. because the buffer data may still be waiting for output.
Therefore, from the perspective of data being actually written to a disk, the file data written using write () is not completely synchronized with the external storage device. the time interval for non-synchronization is very short, generally only a few or a dozen seconds, depending on the written data volume and the status of the I/O data buffer. although the time interval for non-synchronization is very short, if the power is down or the system crashes during this period, the written data cannot be written to the disk and will be lost.
Note: The kernel "writes" the data in the buffer zone to the standard input disk file. Here "write" is not to move the data in the buffer zone to the disk file, but to the disk file, in this case, a backup of the buffer content is retained in the disk file. 1. this design is also justified. If the disk is written to a disk file and the disk is broken or full, the data cannot be sent out. If no backup is made, the data is not lost. that is to say, the kernel will safely delete the backup data after the disk write operation is completed. this process will also be involved in the three functions discussed below.
 
Figure 1 data import process

To ensure the consistency between the actual file system on the disk and the buffer cache, UNIX provides three functions: sync, fsync, and fdatasync.
3. sync function header file: # include <unistd. h>
Define the function: void sync (void );
Returned value: if the request is successful, 0 is returned. If an error occurs,-1 is returned, and errno is set to indicate an error.
Function Description:
Sync is responsible for writing data in the system buffer to the disk to ensure data consistency and synchronization. note: The sync function only adds all modified block buffers to the write queue and returns the result. It does not wait until the actual I/O operation ends. therefore, do not think that if you call the sync function, you may find that the data has been securely sent to the disk file, but the sync function is unknown.
The system waits for the process to call the sync function every other time to ensure that the kernel block cache is refreshed regularly. in UNIX systems, the system waits for the Process update to periodically call the sync function (usually every 30 seconds. the command sync (1) also calls the sync function.
4. fsync function header file: # include <unistd. h>
Define the function: int fsync (int filedes );
Returned value: if the request is successful, 0 is returned. If an error occurs,-1 is returned, and errno is set to indicate an error.
Function Description:
Unlike the sync function, the fsync function only works for a single file specified by the file descriptor filedes, force all modified data (including data in the inner I/O buffer zone) of the file connected to the description fildes to the external permanent media, that is, to refresh all information of the file given by fildes, wait until the disk write operation ends and return. the process that calls fsync () will be blocked until the device reports that the transfer has been completed. this fsync is safe.
After a program writes data, if it needs to ensure that the written data has been written to the disk before subsequent processing, it should call fsync (). for example, database applications usually call fsync () while calling write () to save key transaction data (). this ensures data security and reliability.
5. fdatasync function header file: # include <unistd. h>
Define the function: int fdatasync (int filedes );
Returned value: if the request is successful, 0 is returned. If an error occurs,-1 is returned, and errno is set to indicate an error.
Function Description:
The fdatasync function is similar to the fsync function, but it only affects the file data and forces the transmission of user-written data to the physical storage device, excluding the characteristic data of the file. this reduces the amount of data transferred during file refresh. in addition to data, fdatasync will also synchronously Update file attributes.
6. Error Code: The file descriptor is invalid or the file is closed.
EIO: an error occurs during read/write.
EROFS, EINVAL: The file system where the file is located does not support synchronization.
7. Connection Between fflush () and fsync ()
The kernel I/O buffer is the space managed by the operating system, and the stream buffer is the user space managed by the standard I/O library. fflush () only refreshes the stream buffer in the user space. after fflush () is returned, it only ensures that the data is no longer in the stream buffer, and that they are not necessarily written to the disk. at this time, the data refreshed from the stream buffer may have been written to the disk, or it may still be in the kernel I/O buffer. to ensure that the data written by the stream I/O has been written to the disk, you should also call fsync () after calling fflush ().
8. Summary although delayed write reduces the number of disk reads and writes, it reduces the update speed of the file content so that the data to be written to the file is not written to the disk for a period of time. When the system fails, this delay may cause loss of file update content.

C ++ sync () Usage

I don't know what compiler you are using. My test is sync () (vc2013)
Sync is used:

This removes sync: