Linux/Unix System Programming Manual Reading Notes 8 (file I/O buffer)

Linux/Unix System Programming Manual Reading Notes

Chapter 2

This chapter mainly describes the file I/O buffer.

System I/O calls (kernel) and C standard library I/O functions (stdio functions) buffer disk operations. Through buffering, the user space can be separated from the actual physical device to a certain extent, and the number of times the kernel accesses the disk can be reduced.

Let's take a look at the kernel buffer high-speed buffer: read and write calls do not directly access the disk when operating disk files, as shown in.

For example: write (fd, "abc", 3) the write call transfers "abc" from the user space buffer to the kernel buffer and then returns it. At a later time point (the buffer zone is full or needs to be refreshed), the kernel will write the data in the buffer zone to the disk. Therefore, the write system calls and disk access operations are not performed in sync. Similarly, for read (fd, buffer, 3): the kernel reads data from the disk and stores the data in the kernel buffer, the read call then reads 3 bytes of data from the kernel buffer to the user buffer. When all data in the buffer zone is read, the kernel will read the next data segment from the disk file.

This design can enable the read and write System calls to speed up operations without waiting for disk operations, and reduce the number of times the kernel accesses the disk.

PS: the Linux kernel does not have a fixed upper limit on the size of the buffer (kernel buffer), but the total amount of available physical memory and the requirements of other processes for the physical memory will affect the size of the buffer.

The number of bytes that the kernel accesses to the disk is fixed, so try to make the number of bytes transmitted by each read (write) reach an appropriate number to reduce the time consumed by system calls.

The following table lists the time spent on copying a MB file in the book. BUF_SIZE indicates the number of bytes transmitted, Elapsed indicates the Total time, and Total CPU indicates the Total time spent on copying a file, when the User CPU is the User CPU, the System CPU is the System CPU. The file system to be tested is an ext2 with a block size of 4096 bytes.

When the BUF_SIZE is 4096 bytes, the optimal performance is achieved. Increasing the BUF_SIZE does not significantly affect the performance because the system calls (read and write) the time spent is insignificant compared with the time spent on data transmission and actual disk operations between the user space and the kernel space.

Let's take a look at the second table, which is about the time required to write a MB file.

In fact, after the write call, the disk I/O is not executed so quickly, because the actual computer RAM is very large (the test environment is 4 GB ), therefore, when combined with the previous table, we can know that most of the time consumed by copying files is used for disk reading.

Next, let's look at the buffer of the stdio library.

Some functions of the stdio Library (fprintf, fscanf, fgets, fputs, fgets, fputc, and fgetc) will automatically take large data buffers to reduce system calls.

Use setvbuf to set the buffer mode for stdio library functions

 #include <stdio.h>   setvbuf(FILE *stream,  *buf,  mode, size_t size);

If the call is successful, 0 is returned. If the call fails, a non-0 value is returned.

Stream is a file stream (PS: First open the file stream and then call setvbuf), buf is the buffer used, and size is the buffer size. When buf is not NULL, it points to a memory block of size as the buffer zone of stream. When buf is NULL, The stdio library automatically allocates a buffer zone for stream. Mode is the buffer type.

Mode value:

 

Besides setvbuf, setbuf and setbuffer

  _BSD_SOURCE    #include <stdio.h>   setbuf(FILE *stream,  *   setbuffer(FILE *stream,  *  

You can also use fflush to refresh the stdio buffer.

 #include <stdio.h>   fflush(FILE *stream);

If the call is successful, 0 is returned. If the call fails, EOF is returned.

If stream is NULL, fflush refreshes all the buffers.

If you use fflush in the input stream, you can discard all buffered input data.

In the implementation of the C function library, if stdin and stdout point to the same terminal, fflush (stdout) is implicitly called when reading input from stdin ).

---------------------- Temporarily omitting synchronous I/O .... This part of translation is difficult to understand ....

---------------------- And direct I/O ......................

Summary of buffer for I/O:

 

Exercise:

13-5. Run the tail [-n num] file command to print the last pavement line named file (10 lines by default ). Use the I/O system call (lseek (), read (), write (), and so on) to implement this command.

 

                    #include <sys/stat.h> #include <fcntl.h> #include <ctype.h> #include    BUF_SIZE 4096   main( argc,  *     off_t  seek, off, offset =       whence, fd, num, numRead, type = , i, off_cnt = , n_cnt =      Boolean flag =           *      buf[BUF_SIZE+      (strcmp(argv[], ) ==          usageErr(, argv[           file = argv[      (argc ==  && strcmp(argv[], ) ==          flag =         file = argv[            num = (flag == TRUE) ? getInt(argv[], GN_GT_0, ) :            fd =     (fd == -         errExit(      (fstat(fd, &statbuf) == -         errExit(           (statbuf.st_size <=         off =          whence =         type =                off = - *         whence =               ((seek = lseek(fd, off_cnt * off, whence)) != -         numRead =         (numRead == -             errExit(         (numRead >              (i = numRead-; i >=; --                 (buf[i] ==                      n_cnt++                 (n_cnt == num+                                                (n_cnt == num+                 offset += (numRead- -                                                offset +=                              ;               off_cnt++         memset(buf, , BUF_SIZE+      (seek == -         errExit(          (lseek(fd, ( - offset), SEEK_END) == -         errExit(      ((numRead = read(fd, buf, BUF_SIZE)) >          buf[numRead] =          printf(         memset(buf, , BUF_SIZE+      (numRead == -         errExit(   }

Test results:

I,

lancelot@debian:~/Code/tlpi$ ./ ((numRead = read(fd, buf, BUF_SIZE)) > = , BUF_SIZE+(numRead == -~/Code/tlpi$  ((numRead = read(fd, buf, BUF_SIZE)) > = , BUF_SIZE+(numRead == -

II,

lancelot@debian:~/Code/tlpi$  -n  (lseek(fd, ( - offset), SEEK_END) == -((numRead = read(fd, buf, BUF_SIZE)) > = , BUF_SIZE+(numRead == -~/Code/tlpi$ ./ -n  (lseek(fd, ( - offset), SEEK_END) == -((numRead = read(fd, buf, BUF_SIZE)) > = , BUF_SIZE+(numRead == -

--------------- Be proficient in using system calls and library functions at multiple points. As long as you are skilled in programming, you will not find it difficult to get started ......

--------------- In addition, the learning record about the introduction to dynamic programming cannot be written ..............

--------------- Continue to work hard !!!!!!