Summary of file I/O programming in Linux

1.1 file descriptor
The file descriptor (fd) is equivalent to the file handle in windows programming. It enables a non-negative integer to reference an open file.
In Unix, the file descriptor 0 (STDIN_FILENO) is the standard output, 1 (STDOUT_FILENO) is the standard output, and 2 (STDERR_FILENO) is the standard error output.

1.2 open and close a file
1.2.1 related functions
Int open (const char * pathname, int flags );
Int open (const char * pathname, int flags, mode_t mode );
Int creat (const char * pathname, mode_t mode );
Int close (int fd );
1.2.2 flag options
O_RDONLY, O_WRONLY, and O_RDWR. Only one of the three labels can be specified. You can also select the following constants: O_APPEND, O_CREAT, O_EXEL (specified at the same time as O_CREAT, and an error occurs if the file already exists; otherwise, it is created), O_TRUNC, and O_NONBLOCK (non-blocking), O_SYNC (synchronous write, every write wait until the completion of physical I/O) and other common options.
1.2.3 create a new file
When flags specifies O_CREAT, the third parameter mode must be specified to indicate the access permission for the new file. You can also use the creat function, which is equivalent:
Open (pathname, O_WRONLY | O_CREAT | O_TRUNC, mode)

1.3 change the file offset
1.3.1 related functions
Off_t lseek (int fildes, off_t offset, int whence );
1.3.2 set file displacement
The offset parameter specifies the offset. The whence parameter can be specified as SEEK_SET (relative to the file header), SEEK_CUR (relative to the current position), and SEEK_END (relative to the end of the slave file ).
Because the displacement value may be negative, be cautious when comparing the return value of lseek. Do not test whether it is less than 0, but test whether it is equal to-1.
1.3.3 file Holes
The file displacement can be greater than the current length of the file. In this case, the next write operation on the file will extend the file. To form an empty hole. The bytes in the empty hole are all 0.

1.4 file read/write
1.4.1 related functions
Ssize_t read (int fd, void * buf, size_t count );
Ssize_t write (int fd, const void * buf, size_t count );
1.4.2 read files
The returned value of the read function is the number of actually read bytes, which may be less than the number of bytes to be read. count: read the end of the file; you can also read terminal devices, network devices, or some recorded-oriented devices.
1.4.3 write files
The write operation starts from the current displacement. If the O_APPEND option is set when the file is opened, the current displacement is set to the end of the file before each write.
1.4.4 Efficiency
Use different BUFFSIZE to read data from the standard input and write it to the standard output. In the following test, the standard input is redirected to a 7175K file, and the standard output is redirected to/dev/null.
BUFFSIZE is 1:
Real 0m15. 945 s
User 0m5. 420 s
Sys 0m10. 500 s

BUFFSIZE is 4:
Real 0m4. 025 s
User 0m1. 570 s
Sys 0m2. 460 s

BUFFSIZE is 16:
Real 0m1. 057 s
User 0m0. 200 s
Sys 0m0. 860 s

BUFFSIZE is 256:
Real 0m0. 153 s
User 0m0. 030 s
Sys 0m0. 120 s

Buffsize is 1024:
Real 0m0. 093 s
User 0m0. 000 s
Sys 0m0. 100 s

Buffsize is 4096:
Real 0m0. 081 s
User 0m0. 010 s
Sys 0m0. 080 s

Buffsize is 16384:
Real 0m0. 078 s
User 0m0. 000 s
Sys 0m0. 080 s

BUFFSIZE is 65536:
Real 0m0. 080 s
User 0m0. 000 s
Sys 0m0. 080 s
Increasing BUFFSIZE does not affect the system time.

1.5 copy the file descriptor
1.5.1 related functions
Int dup (int oldfd );
Int dup2 (int oldfd, int newfd );
1.5.2 description
These functions return a new fd that shares the same file table item with oldfd. The difference is that dup2 can specify the value of the new fd. If newfd is enabled, it is disabled first. They can be implemented using fcntl (oldfd, F_DUPFD, 0/newfd). The difference is errno, and dup2 is an atomic operation.

1.6 fcntl Functions
1.6.1 related functions
Int fcntl (int fd, int cmd );
Int fcntl (int fd, int cmd, long Arg );
Int fcntl (int fd, int cmd, struct flock * Lock );
1.6.2 Functions

• Copy an existing Descriptor (cmd = F_DUPFD ).

• Obtain/set the file descriptor flag (cmd = F_GETFD or F_SETFD ).

• Obtain/set the File status flag (cmd d = F_GETFL or F_SETFL ).

• Obtain/set the asynchronous I/O permission (cmd = F_GETOWN or F_SETOWN ).

• Obtain/set the record lock (cmd = F_GETLK, F_SETLK or F_SETLKW ).

1.6.3 f_getfl or f_setfl
For these two operations, you can obtain or set the File status flag. When f_getfl is used, o_rdonly, o_wronly, o_rdwr, o_append, o_nonblock, o_sync, and o_async can be obtained. For the first three flags, you must use the blocked word o_accmode, and then compare the results with the three flags one by one. You can only set the last four flag types when f_setfl.
Accmode = Val & o_accmode;
If (accmode = o_rdonly )...
Else if (accmode = o_wronly )...
Else if (accmode = o_rdwr )...
Else err_dump ("unknown access mode ");

If (Val & o_append )...
If (Val & o_nonblock )...
......

1.7 record lock
1.7.1 related functions:
Int fcntl (int fd, int cmd );
Int fcntl (int fd, int cmd, struct flock * Lock );
1.7.2 structure:
Struct flock {
Short l_type;/* f_rdlck, f_wrlck, f_unlck */
Short l_whence;/* seek_set, seek_cur, seek_end */
Off_t l_start;/* Starting offset for Lock */
Off_t l_len;/* Number of bytes to lock */
Pid_t l_pid;/* PID of process blocking our lock */
};

Rochelle start and Rochelle whence determine the starting position of the lock or unlock area. Rochelle Len determines its length. This area can be crossed by the end of the file, but cannot be crossed by the start position of the file. When l_len is 0, it indicates that the lock area ranges from the starting position to the end of the file, regardless of how the file grows. The general method to lock the entire file is: l_start is 0, l_whence is SEEK_SET, and l_len is 0.
1.7.3 shared and exclusive locks
A shared lock is also called a read lock (RDLCK). An exclusive lock is also called a write lock (WRLCK ).

Table 1 (read and write locks)

All current requirements	Read lock	Write lock
Read lock (one or more)	Yes	Reject
Write lock	Reject	Reject

1.7.4 lock and unlock
Perform the following operations based on the CMD parameter of the fcntl function:

• F_GETLK: Check whether the lock described by the lock parameter can be created.

• F_SETLK: sets the lock description. An error is returned immediately.

• F_SETLKW: the blocking version of F_SETLK. If the lock is not allowed, wait.

1.7.5 inheritance and release of locks
When a process terminates, all the locks it creates are released.
When a file descriptor is disabled, all the locks set by the process on the file associated with the descriptor are released, even if the process still has opened descriptors pointing to the file.
The child process generated by fork does not inherit the lock set by the parent process. After exec is called, the new program can inherit the lock of the original execution program.
1.7.6 create locks and mandatory locks
The Advisory lock does not guarantee that other processes with write permission on the file read and write files. The program that uses the lock must process the record lock in a consistent way. When Mandatory lock is used, the kernel checks whether the calling process violates a lock for each open, read, and write operation.
To use a mandatory lock, you must first enable this function for the file system where the file is located (add the-o mand parameter when mounting), and then enable its set-guid, close the group execution bit (g-x and g + s in the chmod command. The use of the mandatory lock is basically the same as that of the locks.]
The mandatory lock is not POSIX standard.

1.8 I/O multi-channel Transfer
1.8.1 related functions:
Int select (int n, fd_set * readfds, fd_set * writefds, fd_set * contains TFDs, struct timeval * timeout );
Int pselect (int n, fd_set * readfds, fd_set * writefds, fd_set * contains TFDs, const struct timespec * timeout, const sigset_t * sigmask)
Int poll (struct pollfd * ufds, unsigned int nfds, int timeout );

FD_CLR (int fd, fd_set * set );
FD_ISSET (int fd, fd_set * set );
FD_SET (int fd, fd_set * set );
FD_ZERO (fd_set * set );
1.8.2 Related Structure
Fd_set is a collection of fd, and there are four macros to operate on it: clear an fd (FD_ZERO), check whether the specified fd is in the Set (FD_ISSET), add (FD_SET) and remove (FD_CLR) An fd.
Select uses the structure timeval:
Struct timeval {
Long TV _sec;/* seconds */
Long TV _usec;/* microseconds */
};
Pselect uses the structure timespec:
Struct timespec {
Long TV _sec;/* seconds */
Long TV _nsec;/* nanoseconds */
};
1.8.3 description
Select waits for the specified fd file to be ready. For the files in readfds, wait for readable; for the files in writefds, wait for the files to be writable; for the files in exceptfds, wait for exceptions. N is the maximum fd plus 1 in the parameter. It can be set to FD_SETSIZE, but it will affect the efficiency. Timeout is the waiting time. If it is NULL, It is infinite. The returned value is the number of ready fd.
In the select function, if the end of a file is encountered on an fd, select considers the fd to be readable rather than an exception.
Pselect is similar to select, but the timespec structure is used to specify the time. In addition, similar to sigsuspend, the signal shielding will be changed to sigmask while waiting.
Poll is another version of the select function. The difference is that it constructs an array of pollfd structures and specifies a fd for each element and the conditions of interest to it.
1.8.4 usage
Select can be used as a more accurate sleep function.
When you must read and write multiple fd data, you can use the select function to avoid long-time blocking on one fd while the data of the other fd cannot be processed in a timely manner.

1.9 read/write multiple caches
1.9.1 related functions
Ssize_t readv (int fd, const struct iovec * vector, int count );
Ssize_t writev (int fd, const struct iovec * vector, int count );
1.9.2 Related Structure
Struct iovec {
Void * iov_base;/* Starting address */
Size_t iov_len;/* Number of bytes */
};
1.9.3 description
These two functions are used to read and write multiple caches. The read/write order is in the order of arrays.
1.9.4 Performance
Two caches, one 512 and the other 1024, are written to the file 10000 times in three ways respectively. The first is to call writev, and the second is to call write twice in each loop, the third type is a 1536 cache. Copy the content of the first two caches into the cache and then call the write command. The result is as follows. On the right side is the test result on apue. The cache size is 100, 200, and the machine size is 80386.
One writev:
Real 0m0. 318 s 8.2 s
User 0m0. 000 s 0.3 s
Sys 0m0. 320 s 7.8 s

Two write:
Real 0m0. 990 s 13.7 s
User 0m0. 000 s 0.5 s
Sys 0m0. 330 s 13.1 s

Memcpy and one write
Real 0m0. 255 s 8.1 s
User 0m0. 010 s 0.7 s
Sys 0m0. 250 S 7.3 s

It is strange that the system time used to call two writes should be twice that of one write or writev call, but my result is that the system time used to call one writev is similar to that used for two writes, memcpy calls a write operation later than a writev operation.

1.10 I/O storage ing
1.10.1 concepts
Storage ing maps a disk file or device to a cache area of the bucket. Therefore, accessing data from the cache is equivalent to reading and writing files.
1.10.2 related functions
Void * mmap (void * start, size_t length, int prot, int flags, int fd, off_t offset );
Int munmap (void * start, size_t length );
Int msync (void * start, size_t length, int flags );
1.10.3 description
Generally, the start parameter is set to 0, that is, the system selects the starting address of the ing cache, which is obtained through the return value. Fd, length, and offset specify the file area. The prot specifies the protection mode for the ing area, and flags indicates some identification spaces. MAP_SHARED indicates that the ing is shared with other processes, and the access cache retrieval is directly reflected in the file. MAP_PRIVATE indicates that this process is exclusive. writing to the ing cache does not affect the original file. Only one of these two identifiers can be specified.
Munmap removes the storage ing. It should be noted that disabling the file descriptor does not cancel the ing of the file. Calling munmap does not write the content in the ing area to a file. You can call the msync function to synchronize the file and the ing area.
1.10.4 usage
Using mmap/memcpy for file operations is faster than reading/write, because the former directly performs operations on the ing cache, and the latter, the kernel needs to copy data between the user cache and its own cache.
However, the memory ing cannot be used between some devices (such as networks and terminal devices) for replication, and you must note whether the file length is changed during operations.