Linux cat command source code analysis, linuxcat source code analysis

Linux cat command source code analysis, linuxcat source code analysis

I have been reading APUE recently, but I still have to do it. in linux, many commands are open-source. You can directly view the source code. GNU coreutils is a good choice. the source code package contains the source code of the most commonly used commands such as ls and cat. Each Command is short and concise and suitable for reading. below are some notes for reading the cat command.

Download the source code here. in the source code root directory. /configure; make can be directly compiled. After modification, make can be compiled. the command source code is in the src/directory, and some auxiliary functions and constant definitions are available in the lib/directory.

1. Command Line Parsing

Basically, all Linux commands use the getopt function to parse command line parameters. cat is no exception. cat uses the getopt_long function to parse long parameters, use some bool variables to store option values. nothing to say.

2. Check whether the input and output files are the same

For example, in cat test.txt> test.txt, the input and output files are the same, which is invalid.

The input stream of cat is given by the command line. The default value is standard input (stdin), and the output stream is standard output (stdout ). therefore, the string comparison method cannot determine whether the input and output are the same. in addition, for some special files, such as tty, we allow them to have the same input and output, such as cat/dev/tty>/dev/tty, which is legal. cat checks whether the device ID and I-node are the same for the regular file. ignore non-regular file detection. the code for this part is as follows:

Obtain file attributes.

if (fstat (STDOUT_FILENO, &stat_buf) < 0)    error (EXIT_FAILURE, errno, _("standard output"));

Extract the file device number and I-node. For non-regular files, ignore detection.

if (S_ISREG (stat_buf.st_mode))    {      out_dev = stat_buf.st_dev;      out_ino = stat_buf.st_ino;    }  else    {      check_redirection = false;    }

Check. check_redirection is false.

If (fstat (input_desc, & stat_buf) <0) <span style = "white-space: pre"> </span> // input_desc is the input file descriptor {error (0, errno, "% s", infile); OK = false; goto contin ;}

If (check_redirection & stat_buf.st_dev = out_dev & stat_buf.st_ino = out_ino & (<strong> input_desc! = STDIN_FILENO </strong>) <span style = "white-space: pre"> </span>/* If the input is a standard input or its redirection, It is not detected, therefore, the command cat <test> test is also valid */{error (0, 0, _ ("% s: input file is output file"), infile); OK = false; goto contin ;}

Tips: '-' indicates the standard input. For example, the cat-Command actually reads bytes from the standard input. therefore, cat can be used in combination with the pipeline command: echo abcd | cat file1-file2. input only the cat command to read bytes from the standard input by default.

3. Number of bytes read/write at a time

Cat is implemented based on the read and write functions. The number of bytes read/write at a time also affects the program performance.

The insize and outsize variables indicate the number of bytes for one read and write respectively.

insize = io_blksize (stat_buf);

Enum {IO_BUFSIZE = 128*1024}; static inline size_tio_blksize (struct stat sb) {return MAX (IO_BUFSIZE, ST_BLKSIZE (sb); <span style = "white-space: pre "> </span>/* ST_BLKSIZE () macro value depends on the system, defined in lib/stat-size.h */}

The setting of outsize value is similar to that of insize.

4. simple_cat

For example, cat commands do not use any format parameters, such as-v or-t. simple_cat is called to complete the operation. The advantage of simple_cat is that it is fast because it may read and write files in binary mode on some systems. see man 3 freopen.

If (! (Number | show_ends | squeeze_blank) {file_open_mode | = O_BINARY; <span style = "white-space: pre "> </span>/* in linux, O_BINARY is 0, which has no effect, but some systems indicate opening the file in binary format */if (O_BINARY &&! Isatty (STDOUT_FILENO) <span style = "white-space: pre"> </span>/* Call freopen, including error handling, change the output stream mode to "wb" */xfreopen (NULL, "wb", stdout );}

If no format parameter exists, simple_cat is called.

If (! (Number | show_ends | show_nonprinting | show_tabs | squeeze_blank) {insize = MAX (insize, outsize); <span style = "white-space: pre "> </span>/* xzz allocates memory. If the memory fails, call xmalloc-die () to terminate the program and report an error */inbuf = xmalloc (insize + page_size-1 ); OK & = simple_cat (<strong> ptr_align </strong> (inbuf, page_size), insize );}

Ptr_align is an auxiliary function. Because I/O operations read one page at a time, ptr_align makes the starting address of the buffer array an integer multiple of the size to increase IO efficiency.

static inline void *ptr_align (void const *ptr, size_t alignment){  char const *p0 = ptr;  char const *p1 = p0 + alignment - 1;  return (void *) (p1 - (size_t) p1 % alignment);}

The simple_cat function is simple.

Static boolsimple_cat (/* Pointer to the buffer, used by reads and writes. */char * buf,/* Number of characters preferably read or written by each read and write call. */size_t bufsize) {/* Actual number of characters read, and therefore written. */size_t n_read;/* Loop until the end of the file. */while (true) {/* Read a block of input. * // * normal read may be interrupted */n_read = safe_read (input_desc, Buf, bufsize); if (n_read = SAFE_READ_ERROR) {error (0, errno, "% s", infile); return false;}/* End of this file? */If (n_read = 0) return true;/* Write this block out. */{/* The following is OK, since we know that 0 <n_read. */size_t n = n_read;/* both full_write and safe_read call safe_sw. Use a macro to implement it. * Check safe_write.c to find the key to its implementation. */if (full_write (STDOUT_FILENO, buf, n )! = N) error (EXIT_FAILURE, errno, _ ("write error "));}}}

5. safe_rw, full_rw Function

The read and write functions may be interrupted by signal before reading and writing the first character. safe_rw can restore the interrupted read and write processes. this function is very tricky. Its names safe_rw and rw are both macro definitions. Conditional compilation can compile this function into safe_read and safe_write functions.

<Strong> size_t </strong>/* The original return value of the read () function is ssize_t */safe_rw (int fd, void const * buf, size_t count) {/* Work around a bug in Tru64 5.1. attempting to read more than INT_MAX bytes fails with errno = EINVAL. see 

The read/write process may be interrupted by signal. full_rw can restore the read/write process until the specified number of bytes are read or reached the end of the file (EOF), or the read/write error occurs. returns the number of read/write bytes. the full_rw () function name is also macro-defined. In fact, full_read () full_write () is implemented ().

/* Write(read) COUNT bytes at BUF to(from) descriptor FD, retrying if   interrupted or if a partial write(read) occurs.  Return the number   of bytes transferred.   When writing, set errno if fewer than COUNT bytes are written.   When reading, if fewer than COUNT bytes are read, you must examine   errno to distinguish failure from EOF (errno == 0).  */size_tfull_rw (int fd, const void *buf, size_t count){  size_t total = 0;  const char *ptr = (const char *) buf;  while (count > 0)    {      size_t n_rw = safe_rw (fd, ptr, count);      if (n_rw == (size_t) -1)<span style="white-space:pre"></span>/* error */        break;      if (n_rw == 0)<span style="white-space:pre"></span>/* reach EOF */        {          errno = ZERO_BYTE_TRANSFER_ERRNO;          break;        }      total += n_rw;      ptr += n_rw;      count -= n_rw;    }  return total;}

Tips: Check the files safe_read.c and safe_write.c In the lib directory to see how this function is expanded into two different functions.



6. cat function, processing formatted output
Simple_cat only blocks the input and does not process it. All the content related to the formatted output is stored in the cat function.
The implementation of cat functions involves many techniques. for example, a sentin' \ n' is used to mark the end of the input buffer. in addition, a character array is used to count the number of rows, so that the 64-bit integer type cannot be used.
The following is the code for this row counter.

/* Position in 'line_buf' where printing starts.  This will not change   unless the number of lines is larger than 999999.  */static char *line_num_print = line_buf + LINE_COUNTER_BUF_LEN - 8;/* Position of the first digit in 'line_buf'.  */static char *line_num_start = line_buf + LINE_COUNTER_BUF_LEN - 3;/* Position of the last digit in 'line_buf'.  */static char *line_num_end = line_buf + LINE_COUNTER_BUF_LEN - 3;


static voidnext_line_num (void){  char *endp = line_num_end;  do    {      if ((*endp)++ < '9')        return;      *endp-- = '0';    }  while (endp >= line_num_start);  if (line_num_start > line_buf)    *--line_num_start = '1';  else    *line_buf = '>';  if (line_num_start < line_num_print)    line_num_print--;}

The key to understanding this function is to understand the function of the newlines variable. cat formatting outputs the main operations to determine the line breaks and consecutive empty rows. The newlines variable marks the number of empty rows, if the value is 0, the inbuf's reading position is at the beginning of a row. 1 indicates that there is an empty row.-1 indicates that a row has just been parsed and is ready to enter the next row, we can see that the last while (true) two break statements of the cat function set newlines to-1.
Cat formats the output by scanning the input buffer array one by one, and stores the converted characters in the output buffer array.



[Help] A problem with linux source code analysis: how to compile terminal commands?
These commands are actually some programs, and ls and tc are the same process as compiling common programs;

The source files of these commands are not in the kernel and are customized during file system creation;

For example, if you want to add a cat command, you only need to find the cat source code and put it under the/usr/bin directory of your file system after compilation, then cat can be used after startup;

Who can give me a detailed analysis of the du command in Linux? (Including source code analysis)
The Linux du command is used to view the space used. However, unlike the df command, the Linux du command is used to view the space used by files and directory disks. It is also different from the df command, here we will introduce the Linux du command.
Du: query the disk space used by an archive or directory
A: displays the disk space occupied by all directories and each file in the second directory.
B: The size is represented by bytes (the default value is k bytes)
C: add the total value (default)
S: only display the total size of each file
X: only calculate the files of the same file system.
L: Calculate the size of all files
Common commands: du-
Operation Details
Reference
The command du can display the disk space occupied by all files in each directory in the unit of subdirectories in the specified directory. For example:
# Du-h/etc
104 K/etc/defaults
6.0 K/etc/X11
8.0 K/etc/bluetooth
4.0 K/etc/gnats
52 K/etc/isdn
388 K/etc/mail
68 K/etc/mtree
2.0 K/etc/ntp
38 K/etc/pam. d
44 K/etc/periodic/daily
6.0 K/etc/periodic/monthly
42 K/etc/periodic/security
16 K/etc/periodic/weekly
110 K/etc/periodic
6.0 K/etc/ppp
318 K/etc/rc. d
2.0 K/etc/skel
130 K/etc/ssh
10 K/etc/ssl
1.7 M/etc
The-h parameter is used to display the human-readable format. In the application, we can use the "du" command to check which directory occupies the most space. However, the output result of du is usually very long. We can add the-s parameter to omit the subdirectories in the specified directory, and only display the total combination of the directories:
# Du-sh/etc
1.7 M/etc
When viewing the usage of the Directory, we can export the output results to the sort command for sorting to know which file has used the most space:
# Du/etc | sort-nr | more
1746/etc
388/etc/mail
318/etc/rc. d
130/etc/ssh
110/etc/periodic
104/etc/defaults
68/etc/mtree
52/etc/isdn
44/etc/periodic/daily
42/etc/periodic/security
38/etc/pam. d
16/etc/periodic/weekly
10/etc/ssl
8/etc... the remaining full text>