Text Processing of Linux commands (i)

• WC command
  Used to count the number of characters, number of lines, number of words, and so on, the most commonly used commands (when the interview unexpectedly did not think of this command, I am how frustrated ...) ）
  The format is: WC options file-lists, if file-lists is empty or "-", then the data is read from the standard input; When options are empty, the default output is the number of rows, words, and bytes, as follows:

    m@meng:~$ wc examples.desktop    240  5698980 examples.desktop

The various options are described below:

• -C: Only the number of bytes, note is a byte, not a character, a character in different languages occupies a different number of bytes, an English letter is generally only one byte.

        m@meng:~$ cat new        a        m@meng:~$ wc -c new         2 new

There is only one letter A in the new file, but the result is 2 bytes, because the WC also counts the line breaks at the end of the file, and this line break is also interesting, and I'll write an article to analyze it sometime.

• -M: Show only the number of characters, experiment with Chinese:

        m@meng:~$ cat new         你好        m@meng:~$ wc -c new         7 new        m@meng:~$ wc -m new         3 new

The line break is counted as three characters, but the number of bytes occupied is 7, indicating that a Chinese character occupies 3 bytes.

• -L: Show only the number of rows

• -L: Displays the length of the longest line, as follows:

    m@meng:~$ cat new     baa    m@meng:~$ wc -L new     3 new

Thus, the length of the line does not include line-ending wrapping.
--W: Displays the number of words, exactly meaning: A word is a non-zero-length sequence of characters delimited by white space.

• Sort command
  It seems that sorting is a rigid requirement for computers, regardless of which field seems to need to be sorted. Sort is used to order the rows of a file, outputting a sequential result without changing the source file.
  Sort is a row, which starts from the first letter of each line, arranges rows from small to large with the ASCII value of the first letter, compares the second letter of the first letter of the two lines, and so on. However, this sort is affected by the locale environment variable, not necessarily the desired result, as follows:

  m@meng:~$ 3Apple76pear  418m@meng:~$ 3Apple718pear  46

  According to ASCII, Apple should be on the first line, but it appears on the second line because the current locale is ZH_CN, modified as follows:

  m@meng:~$ LC_ALL=Cm@meng:~$ Apple7318pear  46

  Note that modifying into en_US is not working, and changing to C is because manual has a phrase "the locale specified by the environment affects sort order. Set Lc_all=c to get the traditional sort order that uses native byte values ", now sorted in the way you want.

  • -T and-K options
    The more powerful or more common function of the sort command is to sort the formatted rows, which means that each line is divided into a regular number of segments by a delimiter, so that you can specify which segment to sort by, instead of comparing the characters from one character to the beginning of each line like a normal sort. If the data for each row is irregular, sorting by field is not valid.
    -T is used to specify the delimiter,-K to specify which field, and the field is counted from 1. For example:

    m@meng:~$ " "213pear  46Apple78

    The default delimiter is those whitespace characters, such as spaces, tab, etc., on these separators can omit-T, so the above-T "" is a few times, the delimiter can only be a single character, it is generally not quoted.
    You can specify multiple-K options, such as-K 2-k 3, which means that the second field is sorted first, and the second field is sorted by a third field. There are some more complex uses of-K, see Man.

  • -N option
    Sort by number size. By default, the number in the text is treated as a normal string, not a real number. We now change the number of orange in the text to 11, without the-n as follows:

    m@meng:~$ 2banana     1orange     11apple  3pear   4pear   6Apple  7

    Orange is ranked in the second row, which is a typical string ordering method. After adding-N, the following:

    m@meng:~$ 2 -nbanana     1apple  3pear   4pear   6Apple  7orange     11

  • -r option: Reverse sort. When mated with the-K option, you can write directly after the number of fields.
  • -o option: Equivalent to redirect, specify output file, sort result no longer output to standard output, but to specified file.
  • -C option: Do not really sort, just check whether the file is already sorted.

  • -U option: Remove duplicate rows. Some people think that you can also cooperate with-K to remove a field value of the same row, I tried, it seems not possible, as follows:

    m@meng:~$ 1 -uApple    7apple    3banana   1orange   11pear     4pear     6

    It looks like two lines are exactly the same to be removed.

  • -D option: only consider letters and whitespace characters, other characters are automatically ignored, such as # $%, etc.
  • -F option: Ignore case.
  • -I option: ignores nonprinting characters.
    The main thing is these options, and later encounter other good options to add.

Text Processing of Linux commands (i)