Linux System Programming Learning Note (ii) file I/O

1. Each Linux process has a maximum number of open files, by default, the maximum value is 1024 file descriptor can not only reference ordinary files, can also refer to sockets socket, directory, pipeline (everything is a file) by default, The child process obtains a full copy of its parent process file Table 2. Opening the file The open system call must include one of the O_RDONLY,O_WRONLY,O_RDWR three access modes Note O_nonblock mode

int 0644  int0644)

3. Read the file reads the system call will have the following results: (1) The return value is the same as the number of requests, all Len bytes are stored within BUF (2) The return value is less than the number of requests Len, but greater than 0. There are a number of reasons for this: A.read system call is interrupted B.read the system call occurs midway through error C. The number of readable bytes is greater than 0, but is less than lend. Encountering EOF (3) returns 0 before reading the Len Byte, indicating that the EOF (4) call is blocked because it is not currently readable, This situation does not occur in nonblocking mode (5) When the return -1,errno is set to EINTR, which means that the received signal before any byte is read (6) The return -1,errno is set to Eagain, indicating that the read operation is blocked because there is currently no readable byte, which only occurs in nonblocking mode (7) Returns -1,errno set to a value other than Eintr,eagain, indicating that other more serious errors have occurredRead all bytes:

size_t Readn (intFdvoid*buf, size_t len) {size_t tmp=Len; ssize_t ret=0;  while(Len! =0&& (ret = read (FD, buf, len))! =0) {        if(ret = =-1) {            if(errno = =eintr) {                Continue; } fprintf (stderr,"Read error\n");  Break; } Len-=ret; BUF+=ret; }    returnTMP-Len;}

Non-blocking reads: Sometimes we do not want the read system call to be blocked when there is no readable data, but rather that the call can be returned immediately, indicating that there is no data to read, which is non-blocking I/O 4. Write file writes system call no EOF, for normal file, write default action is write all, Unless an error is returned-1 for other files is likely to occur partial write, most typical of network programming socket read and write, should be as follows:

size_t Write_all (intFdvoid*buf, size_t len) {ssize_t ret=0; size_t tmp=Len;  while(Len! =0&& (ret = write (FD, buf, len))! =0) {        if(ret = =-1) {            if(errno = =eintr) {                Continue; } fprintf (stderr,"Write error\n");  Break; } Len-=ret; BUF+=ret; }    returnTMP-Len;}

Append mode ensures that the current position of the file is always at the end of the file, and that the file offset update operation can be considered atomic, so this mode is useful for multitasking append 5. File synchronization When write is called, the kernel copies the data from user buffer to the kernel buffer, But it doesn't have to be immediately written to the destination, the kernel usually performs some checks to copy the data from user buffer to a dirty buffer, and the kernel collects all these dirty buffer (contain data newer than what's on disk) , and then write back to the disk. This delay write does not change the POSIX semantics, but can improve read and write performance if a read is issued for Just-written data, lives in a dirty buffer and are not yet on disk, The request is satisfied from the buffer and not cause a read from the ' stale ' data on disk. So the read was satisfied from the in-memory cache without have to go to disk. Deferred writes can significantly improve performance, but sometimes you need to control the files that are written back to disk, which is required to ensure that file synchronization Fsync system calls ensure that the file data for the FD association is written back to disk

int ret = Fsync (FD);

Open call when the O_SYNC flag indicates that the file must be synchronized

int fd = open (file, O_wronly | O_sync);

O_sync cause I/O wait time to be very expensive, generally, need to ensure that the file write back to disk when we use the Fsync function

6. File location explicit file Locator function: A. Position the file offset to 1825off_t ret = Lseek (FD, (off_t) 1825, seek_set); b. Locate the file cheaply at the end of the file off_t ret = lseek (FD, 0, seek_end); c. position file offset to file open Starting at off_t ret = lseek (FD, 0, Seek_cur)

File positioning can be more than the end of the file, at this time the file write operation will fill 0, forming an empty, empty space does not occupy the physical disk. This implies, the total, size of all files on a filesystem can add up to more than the physical size of the disk7. truncate files

int ftruncate (int fd, off_t len);  

Truncates a given file to a given length, where the given length can be less than the file size or larger than the file size (which can result in voids)

8. Multiple I/O blocking I/o: If a file (such as a pipe input) does not have readable data when the read system is called, the process will block waiting until there is a readable data. Inefficient, multiple file read/write operations cannot be performed multiple I/O can allow programs to block concurrently on multiple files and receive notification immediately when any file becomes readable or writable

 for The application,designed similarly to the following activity:a. multiplexed I for i/Ob. Nothing, ready?   is ready?   for i/O, without bolockinge. Go back to step a

9.select

int Select(intNfds, fd_set* Readfds, fd_set* Writefds, fd_set* Exceptfds,structtimeval*timeout); FD_CLR (intFD, fd_set*Set);//removes a FD from a given setFd_isset (intFD, fd_set*Set);//test whether a FD is part of a given setFd_set (intFD, fd_set*Set);//adds a FD to a given setFd_zero (intFD, fd_set*Set);//removes all FDS from specified set. Shoule is called before every invocation of select ()

Because Fd_set is statically assigned, the system has a maximum open number of file descriptors fd_setsize, which is 1024 in Linux.

#include <stdio.h>#include<stdlib.h>#include<sys/time.h>#include<sys/types.h>#include<unistd.h>#defineTIMEOUT 5/* Select timeout in seconds */#defineBuflen 1024x768/* Read buffer in bytes */intMainintargcChar*argv[]) {    structTimeval TV; Tv.tv_sec=TIMEOUT; Tv.tv_usec=0; /*wait on stdin for input*/Fd_set Readfds; Fd_zero (&Readfds); Fd_set (Stdin_fileno,&Readfds); intRET =Select(Stdin_fileno +1, &readfds, NULL, NULL, &TV); if(ret = =-1) {fprintf (stderr,"Select error\n"); return 1; } Else if(!ret) {fprintf (stderr,"%d seconds elapsed.\n", TIMEOUT); return 0; }    if(Fd_isset (Stdin_fileno, &Readfds)) {        CharBuf[buflen +1]; intLen =Read (Stdin_fileno, buf, Buflen); if(len = =-1) {fprintf (stderr,"Read error\n"); return 1; }        if(Len! =0) {Buf[buflen]=' /'; fprintf (stdout,"read:%s\n", BUF); }        return 0; } Else{fprintf (stderr,"This should not happen\n"); return 1; }}

Ten. Poll

int poll (struct pollfd* fds, nfds_t  int timeout);

This was a program that uses poll () to check whether a read from stdin and a write to stdout would block

#include <unistd.h>#include<poll.h>#defineTIMEOUT 5intMainintargcChar*argv[]) {    structPOLLFD fds[2]; /*watch stdin for input*/fds[0].FD =Stdin_fileno; fds[0].events =Pollin; /*Watch stdout for alibity to write*/fds[1].FD =Stdout_fileno; fds[1].events =pollout; intRET = Poll (FDS,2, TIMEOUT * +); if(ret = =-1) {fprintf (stderr,"Poll error\n"); return 1; }    if(!ret) {fprintf (stdout,"%d seconds elapsed.\n", TIMEOUT); return 0; }    if(fds[0].revents &Pollin) {fprintf (stdout,"stdin is readable\n"); }    if(fds[1].revents &pollout) {fprintf (stdout,"stdout is writable\n"); }    return 0;}

Poll vs Select a. Poll does not require the user to calculate and pass the file descriptor parameters (the value must be set to the maximum number of descriptors plus 1) b. The fd_set of the Select is statically assigned and has a maximum file limit of fd_setsize, Poll there is no such restriction, just create an array of the appropriate size of struct C. Select has better portability and supports select Unix more d. Select support for finer Timeout,poll only supports milliseconds

11. Kernel implementation of the Linux kernel mainly by virtual filesystem, page cache, page Write-back to support the effective and powerful I/O mechanism (1) virtual filesystemthe virtual Filesys TEM (also called a virtual file switch) is a mechanism of abstraction that allows the Linux kernel to call filesystem func tions and manipulate filesystem data without knowing the specific type of filesystem being used. So, a single system call can read any filesystem on any medium, all filesystems support the same concepts, the same interf Aces, and the same calls  (2) page cachethe page cache is a in-memory store of recently accessed data from an On-disk FileSystem. Storing requested data in memory allows the kernel to fulfill subsequent requests for the same data  from memory, AVO IDing repeated disk Accessthe page cache exploits the concept of temporal locality, which says that a resource accessed at One point have a high probability of being accessed again in the near future  time locality: the page cache is the first place That's kernel looks for filesystem data.The first time any item of SATA are read, it is transferred from the disk into the page cache, and are returned to the Appli cation from the cache.   spatial locality: The data is often referenced sequentially. The kernel implements page cache  readahead (pre-read). Readahead is the act of reading extra data off the disk and into the page caches following each read request. In effect, reading a little bit ahead.   (3) page Write-backwhen a process issues a write request, the data is Co Pied into a buffer, and the buffer are marked dirty, denoting that the in-memory copy is newer than the on-disk copy. Eventually, the dirty buffers need to being committed to disk, sync the on-disk files with the data in memory.