Linux advanced Programming--05. File read/write

Buffered I/O and non-buffered I/O

File read and write mainly involves the following five operations: Open, close, read, write, locate. In a Linux system, two sets of APIs are provided,

• Set is C standard Api:fopen, fclose, Fread, fwrite, Fseek,
• The other is the POSIX-defined system Api:open, close, read, write, Seek.

The POSIX-defined API is the system API, and the C standard API is based on the encapsulation of the system API and provides additional buffering capabilities. They can therefore also be called buffer I/O functions and non-buffered I/O functions.

In addition to the buffer IO functions described earlier, the C standard library also provides a series of encapsulated IO functions: such as puts, Putchar, printf, and so on.

Why do we have to increase the buffer function? Mainly because IO operation, the operating system from the user state to the kernel state, and this conversion process is relatively slow, it can be buffered to reduce the number of transitions to the kernel state.

So how does the buffer Io function work?

• When opening a file with fopen, in addition to assigning a file handle, an additional buffer is requested.
• When a file is read, it is read into the buffer first, then the part that the user needs is returned, the extra part is still in the buffer, and the next time you read it, you can return directly from the buffer.
• When a file is written, it is written to the buffer, and then the buffer is full before it is written to the file uniformly.

So, how do we choose which set of I/O functions?

• The non-buffered I/O functions must be in the kernel every time they read and write, and a system call is much slower than the function of a user space, so it is necessary to open I/O buffers in user space.
• Using the buffered I/O library function It is important to keep in mind that I/O buffers and actual files may be inconsistent and call fflush () when necessary.

I/O functions are also used to read and write devices such as terminals or network devices. This usually requires a faster response, and generally does not use buffered I/O functions.

PS: Strictly speaking, even if it is POSIX I/O function, still has the kernel I/Os buffering, so write is not necessarily written directly to the file, it may also write to the kernel I/O buffer, as to whether to write to the file or kernel buffer in the process is not much different, We don't have to pay much attention to this.

Blocking I/O and non-blocking I/O

File read and write usually have blocking and non-blocking two ways, in which the blocking mode is one of our more common way, when the function is blocked until the operation is complete. For example, for the following example, wait for the user to enter a string and output on the screen:

#include <unistd.h>#include <stdlib.h>int main(void){    char buf[10];    int n = read(STDIN_FILENO, buf, 10);    write(STDOUT_FILENO, buf, n);    return 0;}

When the function is executed, the read function blocks until the data is entered on the screen and returns (at which time stdin data is available).
One big problem with blocking IO is that concurrency cannot be achieved. When multiple IO operations are performed simultaneously, the previous file data is not available (often an IPC operation such as a socket), and subsequent IO operations cannot be performed.
Non-blocking IO can be a good solution to this problem, and to use non-blocking IO operations, the O_NONBLOCK flag needs to be developed at open. This way, if the device is temporarily without data readable, return-1, the caller should try to read again (again). This behavior is called polling (Poll), and the caller simply queries, not blocking here death, so that multiple devices can be monitored simultaneously:

#include <unistd.h>#include <fcntl.h>#include <stdlib.h>int main(void){    char buf[10];    int fd, n;    fd = open("/dev/tty", O_RDONLY|O_NONBLOCK);    while (1)    {        n = read(fd, buf, 10);        if (n >= 0)            break;        sleep(1);    }    write(STDOUT_FILENO, buf, n);    close(fd);    return 0;}

PS: For the sake of simple example functions, I do not consider the handling of exceptions (such as open failures), which are essential in the actual project.

Non-blocking I/O has a disadvantage, if all the devices have been no data arrives, the caller needs to query repeatedly, this will always occupy the CPU. Therefore, when using non-blocking I/O, it is usually not kept querying in a while loop (this is called Tight loop), but instead of waiting for a moment to query for each delay, so as not to do too much work, you can schedule other processes to execute when the delay waits.

However, this introduces a new problem that may lead to a lack of timely data reading, and in my previous example, I sleep for a second in each loop. If the data is available at the beginning of sleep, but it does not respond immediately, you need to go to the end of sleep to output the results.

To solve this problem satisfactorily, you need to use the Select function, it can block the simultaneous monitoring of multiple devices, you can also set the time-out of blocking wait, because the select is seen in the socket programming scenario, here is not a good example, follow-up if you will introduce the socket programming in detail and then introduce it, To understand how it works you can look at this article Select, Multi-sync I/O model.

From for notes (Wiz)

Linux Advanced Programming--05. File read/write