Linux Shell Common Tips (vi)

12. Sort commands for Rowssort:

1. Sort command-line options:

Options	Describe
-T	Separators between fields
-F	Ignore case when sorting based on character
-K	Define sorted field fields, or sort by partial data based on field fields
-M	Merges the sorted input file into a sorted output data stream
-N	Comparing fields with integer types
-O outfile	Writes output to the specified file
-R	The order of the inverted sort is from large to small, and the normal sort is from small to large
-U	Only a unique record, discarding all records with the same key value
-B	Ignore the preceding spaces

2. Sort usage Examples:
Tip: The first sort field is highlighted in red in the output below, followed by purple, green.
/> sed-n ' 1,5p '/etc/passwd > Users
/> Cat Users
Root:x:0:0:root:/root:/bin/bash
Bin:x:1:1:bin:/bin:/sbin/nologin
Daemon:x:2:2:daemon:/sbin:/sbin/nologin
Adm:x:3:4:adm:/var/adm:/sbin/nologin
Lp:x:4:7:lp:/var/spool/lpd:/sbin/nologin

#-t defines the delimiter between a colon and a domain field, and-K 2 specifies a forward sort based on the second field (the field order starts from 1).
/> sort-t ': '-K 1 users
Adm:x:3:4:adm:/var/adm:/sbin/nologin
Bin:x:1:1:bin:/bin:/sbin/nologin
Daemon:x:2:2:daemon:/sbin:/sbin/nologin
Lp:x:4:7:lp:/var/spool/lpd:/sbin/nologin
Root:x:0:0:root:/root:/bin/bash

#还是以冒号为分隔符, this time it is based on the third domain field for the inverted sort.
/> sort-t ': '-K 3r users
Lp:x:4:7:lp:/var/spool/lpd:/sbin/nologin
Adm:x:3:4:adm:/var/adm:/sbin/nologin
Daemon:x:2:2:daemon:/sbin:/sbin/nologin
Bin:x:1:1:bin:/bin:/sbin/nologin
Root:x:0:0:root:/root:/bin/bash

#先以第六个域的第2个字符到第4个字符进行正向排序, reverse-order based on the first field.
/> sort-t ': '-K 6.2,6.4-k 1r users
Bin:x:1:1:bin:/bin:/sbin/nologin
Root:x:0:0:root:/root:/bin/bash
Daemon:x:2:2:daemon:/sbin:/sbin/nologin
Lp:x:4:7:lp:/var/spool/lpd:/sbin/nologin
Adm:x:3:4:adm:/var/adm:/sbin/nologin

#先以第六个域的第2个字符到第4个字符进行正向排序, make a forward sort based on the first field. Compared to the previous example, the 4th and 5th lines exchanged positions.
/> sort-t ': '-K 6.2,6.4-k 1 Users
Bin:x:1:1:bin:/bin:/sbin/nologin
Root:x:0:0:root:/root:/bin/bash
Daemon:x:2:2:daemon:/sbin:/sbin/nologin
Adm:x:3:4:adm:/var/adm:/sbin/nologin
Lp:x:4:7:lp:/var/spool/lpd:/sbin/nologin

#基于第一个域的第2个字符排序
/> sort-t ': '-K 1.2,1.2 users
Daemon:x:2:2:daemon:/sbin:/sbin/nologin
Adm:x:3:4:adm:/var/adm:/sbin/nologin
Bin:x:1:1:bin:/bin:/sbin/nologin
Root:x:0:0:root:/root:/bin/bash
Lp:x:4:7:lp:/var/spool/lpd:/sbin/nologin

#基于第六个域的第2个字符到第4个字符进行正向排序, the-u command requires that duplicate rows of key values be removed when sorting.
/> sort-t ': '-K 6.2,6.4-u users
Bin:x:1:1:bin:/bin:/sbin/nologin
Root:x:0:0:root:/root:/bin/bash
Daemon:x:2:2:daemon:/sbin:/sbin/nologin
Adm:x:3:4:adm:/var/adm:/sbin/nologin

/> cat/etc/passwd | wc-l#计算该文件中文本的行数.
39
/> sed-n ', $p '/etc/passwd > Users2#取最后5行并输出到users2中.
/> Cat Users2
Sshd:x:74:74:privilege-separated Ssh:/var/empty/sshd:/sbin/nologin
Mysql:x:27:27:mysql Server:/var/lib/mysql:/bin/bash
Pulse:x:496:494:pulseaudio System Daemon:/var/run/pulse:/sbin/nologin
Gdm:x:42:42::/var/lib/gdm:/sbin/nologin
Stephen:x:500:500:stephen:/home/stephen:/bin/bash

#基于第3个域字段以textSort of form
/> sort-t ': '-K 3 users2
Mysql:x:27:27:mysql Server:/var/lib/mysql:/bin/bash
Gdm:x:42:42::/var/lib/gdm:/sbin/nologin
Pulse:x:496:494:pulseaudio System Daemon:/var/run/pulse:/sbin/nologin
Stephen:x:500:500:stephen:/home/stephen:/bin/bash
Sshd:x:74:74:privilege-separated Ssh:/var/empty/sshd:/sbin/nologin

#基于第3个域字段以DigitalSort of form
/> sort-t ': '-K 3n users2
Mysql:x:27:27:mysql Server:/var/lib/mysql:/bin/bash
Gdm:x:42:42::/var/lib/gdm:/sbin/nologin
Sshd:x:74:74:privilege-separated Ssh:/var/empty/sshd:/sbin/nologin
Pulse:x:496:494:pulseaudio System Daemon:/var/run/pulse:/sbin/nologin
Stephen:x:500:500:stephen:/home/stephen:/bin/bash

#基于当前系统执行进程的owner名排序, and writes the results of the order to the result file
/> Ps-ef | sort-k 1-o result

13. Delete duplicate row commands Uniq:

Uniq has 3 of the most commonly used options, see the following list:

Options	Command description
-C	The number of times the row repeats can be added before each output line
-D	Only duplicate rows are displayed
-U	Lines that appear as duplicates

/> Cat Testfile
Hello
World
Friend
Hello
World
Hello

#直接删除未经排序的文件, you will find that no rows were deleted
/> Uniq testfile
Hello
World
Friend
Hello
World
Hello

#排序之后删除了重复行, the number of times the row repeats at the beginning of the line
/> Sort testfile | uniq-c
1 friend
3 Hello
2 World

#仅显示存在重复的行, and displays the number of repetitions of the row at the beginning of the line
/> Sort testfile | uniq-dc
3 Hello
2 World

#仅显示没有重复的行
/> Sort testfile | uniq-u
Friend

14. File Compression decompression Command tar :

1. Tar command-line options

Options	Command description
-C	Create a compressed archive
-X	Extract
--delete	Delete an existing file from the archive, and if the file appears multiple times in the package, it will all be deleted.
-T	View a list of files in a compressed package
-R	Append files to the end of a compressed archive file
-U	Update the files in the original compressed package
-Z	Compress to gzip format, or unzip in gzip format
-j	Compress to BZIP2 format, or extract in bzip2 format
-V	Displays the process of compression or decompression, which is generally not suitable for background operations
-F	Using the file name, this parameter is the last parameter and can only be followed by the file name.

2. Examples of tar use:
#将当前目录下所有文件压缩打包, it should be explained that many people are accustomed to name the extension of files compressed by the Tar tool. Tar
/> TAR-CVF Test.tar *
-rw-r--r--. 1 root root 183 Nov 08:02 users
-rw-r--r--. 1 root root 279 Nov 08:45 users2

/> cp.  /*.log. #从上一层目录新copy一个 the. log file to the current directory.
/> TAR-RVF test.tar *.log #将扩展名为. log files are appended to the Test.tar package.
/> TAR-TVF Test.tar
-rw-r--r--root/root 183 2011-11-11 08:02 Users
-rw-r--r--root/root 279 2011-11-11 08:45 users2
-rw-r--r--root/root 48217 2011-11-11 22:16 Install.log

/> Touch Install.log#使原有的文件更新一下最新修改时间
/> TAR-UVF test.tar *.log In #重新将更新后的log文件更新到test. Tar
/> TAR-TVF Test.tar#从输出结果可以看出tar包中多出一个更新后install. log file.
-rw-r--r--root/root 183 2011-11-11 08:02 Users
-rw-r--r--root/root 279 2011-11-11 08:45 users2
-rw-r--r--root/root 48217 2011-11-11 22:16 Install.log
-rw-r--r--root/root 48217 2011-11-11 22:20 Install.log

/> Tar--delete install.log-f test.tar #基于上面的结果, remove Install.log from the compressed package
-rw-r--r--root/root 183 2011-11-11 08:02 Users
-rw-r--r--root/root 279 2011-11-11 08:45 users2

/> rm-f users users2 #从当前目录将tar中的两个文件删除
/> tar-xvf Test.tar#解压
/> ls-l users*#仅列出users和users2的详细列表信息
-rw-r--r--. 1 root root 183 Nov 08:02 users
-rw-r--r--. 1 root root 279 Nov 08:45 users2

#以gzip的格式压缩并打包, decompression should also be extracted in the same format, it should be explained that the format of the package is compressed in the extension after adding. GZ
/> Tar-cvzf test.tar.gz *
    /> TAR-TZVF test.tar.gz #查看压缩包中文件列表时也要加ZOptionsgzipFormat
-rw-r--r--root/root 48217 2011-11-11 22:50 Install.log
-rw-r--r--root/root 183 2011-11-11 08:02 Users
-rw-r--r--root/root 279 2011-11-11 08:45 users2

/> rm-f users users2 install.log
/> tar-xzvf test.tar.gz#以gzipThe format decompression
/> ls-l *.log users*
-rw-r--r--root/root 48217 2011-11-11 22:50 Install.log
-rw-r--r--root/root 183 2011-11-11 08:02 Users
-rw-r--r--root/root 279 2011-11-11 08:45 users2

/> rm-f test.*#删除当前目录下原有的压缩包文件
#以bzip2The format of the compression and packaging, decompression should also be extracted in the same format, need to explain that the format of the package is compressed in the extension after the add. bz2
/> TAR-CVJF test.tar.bz2 *
/> tar-tjvf test.tar.bz2#查看压缩包中文件列表时也要加JOptionsbzip2Format
-rw-r--r--root/root 48217 2011-11-11 22:50 Install.log
-rw-r--r--root/root 183 2011-11-11 08:02 Users
-rw-r--r--root/root 279 2011-11-11 08:45 users2

/> rm-f *.log user*
/> tar-xjvf test.tar.bz2#以bzip2The format decompression
/> ls-l
-rw-r--r--. 1 root root 48217 Nov 22:50 Install.log
-rw-r--r--. 1 root root 10530 Nov 23:08 test.tar.bz2
-rw-r--r--. 1 root root 183 Nov 08:02 users
-rw-r--r--. 1 root root 279 Nov 08:45 users2

15. Large file Split command split:

The following list shows several command-line options that are most commonly used by this command:

Options	Describe
-L	Specifies the number of rows, each separated into a single file, with a default value of 1000 rows.
-B	Specifies the number of bytes, supported by: K and M
-C	Similar to the-B parameter but maintains the integrity of each line as much as possible when cutting
-D	The suffix of the generated file is a number, and if you do not specify this option, the default is the letter

/> Ls-l
-rw-r--r--. 1 root root 10530 Nov 23:08 test.tar.bz2

/> Split-b 5k test.tar.bz2 #以每文件5k的大小切割test. tar.bz2
/> ls-l#查看切割后的结果, the file name is split by default in the following form.
-rw-r--r--. 1 root root 10530 Nov 23:08 test.tar.bz2
-rw-r--r--. 1 root root 5120 Nov 23:34 Xaa
-rw-r--r--. 1 root root 5120 Nov 23:34 xab
-rw-r--r--. 1 root root 290 Nov 23:34 xac

/> rm-f x*#删除拆分后的小文件
/> split-d-B 5k test.tar.bz2 #-d option to name the split small file after the suffix as a number
/> ls-l
-rw-r--r--. 1 root root 10530 Nov 23:08 test.tar.bz2
-rw-r--r--. 1 root root 5120 Nov 23:36 x00
-rw-r--r--. 1 root root 5120 Nov 23:36 x01
-rw-r--r--. 1 root root 290 Nov 23:36 x02

/> WC install.log-l#计算该文件的行数
/> split-l install.log#每300行拆分成一个小文件
/> ls-l x*
-rw-r--r--. 1 root root 11184 Nov 23:42 Xaa
-rw-r--r--. 1 root root 10805 Nov 23:42 xab
-rw-r--r--. 1 root root 12340 Nov 23:42 xac
-rw-r--r--. 1 root root 11783 Nov 23:42 xad
-rw-r--r--. 1 root root 2105 Nov 23:42 xae