File IO buffers under Linux, and their associated operations

File IO operations under Linux

Talking about file IO buffering

The Read () and write () functions do not directly initiate disk access when manipulating disk files, but only replicate data between the user space buffer and the kernel buffer cache.

When the write () function is written to 3 bytes, the kernel will write the data in its buffer to disk because the system call is not synchronized with the disk operation at the end of the write () function at a later point. If, during this time, another process attempts to read these bytes of the file, the kernel will automatically supply the data from the buffer cache instead of the file.

In the same vein, for input, the kernel reads data from the disk and stores it in the kernel buffer. The read () call reads the data from the buffer until the data in the buffer is exhausted, and the kernel reads the next paragraph of the file into the buffer cache.

This design is designed to make read () and write () calls faster because they do not need to wait for disk operations. At the same time, this design is extremely efficient because it reduces the number of disk transfers that the kernel has to perform.

The Linux kernel has no fixed upper limit on the size of the buffer cache. The kernel allocates as many buffer cache pages as possible, and is limited to only two factors: the total amount of physical memory available, and the need for physical memory for other purposes. If there is not enough memory available, the kernel flushes some of the modified buffer cache pages to disk and frees them for system reuse.

Effect of buffer size on performance of IO system calls

Write 1000 bytes on the disk, regardless of whether you write one byte at a time or write 1000 bytes at a time, the kernel accesses the disk the same number of times. But the former consumes significantly more time than the latter, and the latter requires only one system call, and the former needs to be called 1000 times.

when using the Write function and the read function, we can define a buffer, a buffer of moderate size can greatly increase the efficiency of reading and writing, the size of the buffer can be determined according to the actual situation. In contrast, the IO operation in stdio is more flexible with the write function and the read function because it already has its own buffer in the Fwrite function and the Fread function.

1 Basic concepts of documents

In the Unix/linux system, almost everything can be considered a file, so the operation of the file for various input and so on, of course, directories can also be considered as files.

Such as:

File-related read and write functions

(1) Open function

#include <sys/types.h>

#include <sys/stat.h>

#include <fcntl.h>

int open (const char *pathname, int flags);

int open (const char *pathname, int flags, mode_t mode)

Take the second open function as an example and parse the following:

First parameter: file path and filename as a string

Second parameter: Operation flag

Must contain one of the following access modes:

o_rdonly-Read Only

o_wronly-Write only

O_RDWR-Readable and writable

You can also press a bit or the flag value

o_append-Append, writing to the end of the file

O_creat-file does not exist then created, exists then open

O_EXCL-Used in conjunction with O_creat, open fails if file exists

O_trunc-file exists and allows write, empty file

Third parameter: operation mode, permissions

When creating a new file, you need to specify the file permissions

Such as:

0644=> rw-r--r--

Return value: Successful return of new file descriptor (non-negative integer), failed return-1

function function: Mainly used to open/create a file/device

(2) Close function

#include <unistd.h>

int close (int fd);

Return value: Successfully returned 0, failure returned-1.

function function:

Used primarily to close the file descriptor specified by the parameter FD, which means that the descriptor FD is no longer associated with any one file for the next use.

(3) Read function

#include <unistd.h>

ssize_t Read (int fd, void *buf, size_t count);

First parameter: File descriptor (where to read from)

Second parameter: The first address of the buffer (where to save)

Third parameter: The size of the data read

Return value: The number of bytes read successfully returned, returning 0 to read to the end of the file

Failed return-1

function function: Represents the reading of data of a specified size from a specified file

(4) Write function

#include <unistd.h>

ssize_t Write (int fd, const void *buf, size_t count);

First parameter: File descriptor (where to write to)

Second parameter: The first address of the buffer (where the data comes from)

Third parameter: size of data to write

Return value: The number of bytes written successfully returned, 0 indicates no write

Failed return-1

function function: Indicates that the specified data is written to the specified file.

Attention:

Read and write functions are generally read and written in binary form by default.

(5) Lseek function

#include <sys/types.h>

#include <unistd.h>

off_t lseek (int fd, off_t offset, int whence);

First parameter: File descriptor (indicates which file to manipulate)

Second parameter: offset (positive number indicates backward offset, negative forward offset)

Third parameter: Start position (offset from where)

seek_set-file Start position

seek_cur-File Current Location

seek_end-file End Position

Return value: The offset from the file header was successfully returned, and the failure returned-1.

function function: Mainly used to adjust the file read and write location.

Attention:

The end position of the file refers to the next position of the last character in the file.

File IO buffers under Linux, and their associated operations