Linux Command Text Processing (1): linux Command Text Processing

Linux Command Text Processing (1): linux Command Text Processing

Wc command

Used to count the number of characters, number of lines, and number of words in a file. frequently-used commands (I didn't even think about this command during the interview. How frustrated I was ...)
The format is wc options file-lists. If file-lists is empty or "-", the data is read from the standard input. If options is empty, by default, the output data is the number of rows, the number of words, and the number of bytes, as shown below:

    m@meng:~$ wc examples.desktop    240  569 8980 examples.desktop

The following describes the options:

• -C: only the number of bytes is displayed. Note that the number of bytes is different for a character in different languages. An English letter generally only occupies one byte.

        m@meng:~$ cat new        a        m@meng:~$ wc -c new         2 new

There is only one letter a in the new file, but the statistical result is two bytes. This is because wc also counts the line break at the end of the file. This line break is also very interesting, I will write an article to analyze it in another day.

• -M: only the number of characters is displayed. Experiment in Chinese:

M @ meng :~ $ Cat new hello m @ meng :~ $ Wc-c new 7 new m @ meng :~ $ Wc-m new 3 new

The linefeed is indeed only three characters, but the number of bytes is 7, indicating that a Chinese character occupies three bytes.

• -L: displays only the number of rows.

• -L: display the length of the longest row, as shown below:

    m@meng:~$ cat new     baa    m@meng:~$ wc -L new     3 new

It can be seen that the line length does not include the line break at the end of the line.

• -W: displays the number of words. The exact meaning is: A word is a non-zero-length sequence of characters delimited by white space.

Sort command

It seems that sorting is a rigid demand of computers, and it seems that sorting is required in any field. Sort is used to sort the lines of files and output the sorted results without changing the source files.
The unit of sorting by sort is the row, which is compared from the first letter of each line, and the rows are arranged in ascending order according to the ASCII value of the first letter; if the first letter of a row is the same, the second letter is compared, and so on. However, this sorting will be affected by the locale environment variables, and the expected results may not appear, as shown below:

    m@meng:~$ cat new     apple 3    Apple 7    pear 6    pear  4    banana 1    orange 8    m@meng:~$ sort new     apple 3    Apple 7    banana 1    orange 8    pear  4    pear 6

Apple should be placed in the first line according to ASCII, but it appears in the second line because the current locale is zh_CN. The modification is as follows:

    m@meng:~$ export LC_ALL=C    m@meng:~$ sort new     Apple 7    apple 3    banana 1    orange 8    pear  4    pear 6

Note that changing to en_US does not work. Changing to C is because manual has The following sentence: "the locale specified by The environment affects sort order. set LC_ALL = C to get the traditional sort order that uses native byte values ", which is now sorted in the desired way.

• -T and-k options
  The more powerful or commonly used function of the sort command is to sort formatted rows. Formatting means that each line is divided into several regular segments by a separator, in this way, you can specify the segments to be sorted, rather than comparing them one by one from the beginning of each line as in normal sorting. If the data in each row is irregular, sorting by field is invalid.
  -T is used to specify the delimiter, and-k is used to specify which field, and the field starts counting from 1. For example:

      m@meng:~$ sort -t " " -k 2 new       banana 1      apple 3      pear  4      pear 6      Apple 7      orange 8

The default delimiters are those blank characters, such as spaces and tabs.-t can be omitted for these delimiters. Therefore, the-t "above can be used multiple times. The separators can only be single characters, therefore, no quotation marks are required.
You can specify multiple-k options, such as-k 2-k 3, to sort by the second field first. When the second field is the same, it is sorted by the third field. -K also has some more complex usage, see man.

• -N Option
  Sort by number. By default, numbers in text are treated as common strings rather than real numbers. Now we change the number of oranges in the text to 11. If there is no-n, it is as follows:

      m@meng:~$ sort new -k 2      banana     1      orange     11      apple  3      pear   4      pear   6      Apple  7

Orange is placed in the second row, which is a typical string sorting method. After adding-n, it is as follows:

      m@meng:~$ sort new -k 2 -n      banana     1      apple  3      pear   4      pear   6      Apple  7      orange     11

• -R option: reverse sorting. In combination with the-k option, you can directly write it after the number of fields.
• -O option: This option is equivalent to redirection. It specifies the output file. The sorting result is not output to the standard output but to the specified file.
• -C option: Check whether the file is sorted.
• -U option: Remove duplicate rows. Some people think that it can be used with-k to remove rows with the same field value. I tried it and it does not seem to work, as shown below:

        m@meng:~$ sort new -k 1 -u        Apple    7        apple    3        banana   1        orange   11        pear     4        pear     6

It seems that two rows must be identical before removal.

• -D option: Only letters and white spaces are considered. Other characters are automatically ignored, for example, # $ %.
• -F option: case insensitive.
• -I option: ignore non-printable characters.
  These are the main options. I will try other good options later.