Common CentOS Shell skills: sort uniq tar split

Line sorting command sort:

1. sort command line options:

Option description-delimiter between t fields-f when character sorting is ignored, or sort some data based on the domain field-m sorts the sorted input files, merged into a sorted output data stream-n the order in which the output data is written to the specified file-r inverted sorting by the integer comparison field-o outfile is large to small, normally sorted as small as large-u only has unique records, discard all records with the same key value-B ignore the leading space

2. sort instance:
Tip: In the following output results, red is marked as the first sorting field, followed by Purple and green.
/> Sed-n '1, 5'/etc/passwd> users
/> Cat users
Root: x: 0: 0: root:/bin/bash
Bin: x: 1: 1: bin:/sbin/nologin
Daemon: x: 2: 2: daemon:/sbin/nologin
Adm: x: 3: 4: adm:/var/adm:/sbin/nologin
Lp: x: 4: 7: lp:/var/spool/lpd:/sbin/nologin

#-T defines the colon as the delimiter between domain fields.-k 1 specifies the forward sorting based on the first field (the field sequence starts from 1 ).
/> Sort-t': '-k 1 users
Adm: x: 3: 4: adm:/var/adm:/sbin/nologin
Bin: x: 1: 1: bin:/sbin/nologin
Daemon: x: 2: 2: daemon:/sbin/nologin
Lp: x: 4: 7: lp:/var/spool/lpd:/sbin/nologin
Root: x: 0: 0: root:/bin/bash

# Use the colon as the separator. This time, the sorting is performed based on the third field.
/> Sort-t': '-k 3r users
Lp: x: 4: 7: lp:/var/spool/lpd:/sbin/nologin
Adm: x: 3: 4: adm:/var/adm:/sbin/nologin
Daemon: x: 2: 2: daemon:/sbin/nologin
Bin: x: 1: 1: bin:/sbin/nologin
Root: x: 0: 0: root:/bin/bash

# Perform forward sorting based on the sixth domain's 2nd to 4th characters, and then reverse sorting based on the first domain.
/> Sort-t': '-k 6.2, 6.4-k 1r users
Bin: x: 1: 1: bin:/sbin/nologin
Root: x: 0: 0: root:/bin/bash
Daemon: x: 2: 2: daemon:/sbin/nologin
Lp: x: 4: 7: lp:/var/spool/lpd:/sbin/nologin
Adm: x: 3: 4: adm:/var/adm:/sbin/nologin

# First, forward sorting is performed based on the sixth domain's 2nd to 4th characters, and then forward sorting is performed based on the first domain. Compared with the previous example, rows 4th and 5th exchange positions.
/> Sort-t': '-k 6.2, 6.4-k 1 users
Bin: x: 1: 1: bin:/sbin/nologin
Root: x: 0: 0: root:/bin/bash
Daemon: x: 2: 2: daemon:/sbin/nologin
Adm: x: 3: 4: adm:/var/adm:/sbin/nologin
Lp: x: 4: 7: lp:/var/spool/lpd:/sbin/nologin

# Sort by 2nd characters in the first domain
/> Sort-t': '-k 1.2, 1.2 users
Daemon: x: 2: 2: daemon:/sbin/nologin
Adm: x: 3: 4: adm:/var/adm:/sbin/nologin
Bin: x: 1: 1: bin:/sbin/nologin
Root: x: 0: 0: root:/bin/bash
Lp: x: 4: 7: lp:/var/spool/lpd:/sbin/nologin

# Forward sorting is performed based on the 2nd to 4th characters in the sixth domain. The-u Command requires that duplicate key-value Rows be deleted during sorting.
/> Sort-t': '-k 6.2, 6.4-u users
Bin: x: 1: 1: bin:/sbin/nologin
Root: x: 0: 0: root:/bin/bash
Daemon: x: 2: 2: daemon:/sbin/nologin
Adm: x: 3: 4: adm:/var/adm:/sbin/nologin

/> Cat/etc/passwd | wc-l# Calculate the number of lines of text in the file.
39
/> Sed-n'35, $ P'/etc/passwd> users2# Extract the last five rows and output them to users2.
/> Cat users2
Sshd: x: 74: 74: Privilege-separated SSH:/var/empty/sshd:/sbin/nologin
Mysql: x: 27: 27: MySQL Server:/var/lib/mysql:/bin/bash
Pulse: x: 496: 494: PulseAudio System Daemon:/var/run/pulse:/sbin/nologin
TPD: x: 42: 42:/var/lib/TPD:/sbin/nologin
Stephen: x: 500: 500: stephen:/home/stephen:/bin/bash

# Text sorting based on 3rd domain fields
/> Sort-t': '-k 3 users2
Mysql: x: 27: 27: MySQL Server:/var/lib/mysql:/bin/bash
TPD: x: 42: 42:/var/lib/TPD:/sbin/nologin
Pulse: x: 496: 494: PulseAudio System Daemon:/var/run/pulse:/sbin/nologin
Stephen: x: 500: 500: stephen:/home/stephen:/bin/bash
Sshd: x: 74: 74: Privilege-separated SSH:/var/empty/sshd:/sbin/nologin

# Sort by number based on 3rd domain fields
/> Sort-t': '-k 3n users2
Mysql: x: 27: 27: MySQL Server:/var/lib/mysql:/bin/bash
TPD: x: 42: 42:/var/lib/TPD:/sbin/nologin
Sshd: x: 74: 74: Privilege-separated SSH:/var/empty/sshd:/sbin/nologin
Pulse: x: 496: 494: PulseAudio System Daemon:/var/run/pulse:/sbin/nologin
Stephen: x: 500: 500: stephen:/home/stephen:/bin/bash

# Sort the result by the owner name of the execution process of the current system and write the result to the result file.
/> Ps-ef | sort-k 1-o result

13. Command uniq for deleting duplicate rows:

Uniq has three most common options, as shown in the following list:

Option command description-c you can add the number of times that the row repeats before each output line-d. Only duplicate rows are displayed-u. No duplicate rows are displayed.

/> Cat testfile
Hello
World
Friend
Hello
World
Hello

# Directly Delete unordered files. No rows are deleted.
/> Uniq testfile
Hello
World
Friend
Hello
World
Hello

# Duplicate rows are deleted after sorting, and the number of repeated rows is output at the beginning of the row.
/> Sort testfile | uniq-c
1 friend
3 hello
2 world

# Only duplicate rows are displayed, and the number of repeated rows is displayed at the beginning of the row.
/> Sort testfile | uniq-dc
3 hello
2 world

# Only show rows with no duplicates
/> Sort testfile | uniq-u
Friend

14. File compression and decompression command tar:

1. tar command line options

Option command description-c create a compressed file-x decompress -- delete an existing file from the compressed package. If the file appears multiple times in the package, delete it all. -T view the file list in the compressed package-r append the file to the end of the compressed archive file-u update the file in the original compressed package-z compression to gzip format, or decompress-j in gzip format to bzip2 format, or decompress-v in bzip2 format to display the compression or decompression process. This option is generally not suitable for background operations-f uses the file name, this parameter is the last parameter and can only be followed by the file name.

2. tar instance:
Compress all the files in the current directory. The extension name of the files compressed by the tartool is named .tar.
/> Tar-cvf test.tar *
-Rw-r --. 1 root 183 Nov 11 08:02 users
-Rw-r --. 1 root 279 Nov 11 08:45 users2

/> Cp ../*. log.# Copy a new. log file from the previous directory to the current directory.
/> Tar-rvf test.tar *. logAdd the file named .logto the test.tar package.
/> Tar-tvf test.tar
-Rw-r -- root/root 183 2011-11-11 users
-Rw-r -- root/root 279 2011-11-11 users2
-Rw-r -- root/root 48217 install. log

/> Touch install. log# Update the last modification time of the original file
/> Tar-uvf test.tar *. logRefresh the updated logfile to test.tar.
/> Tar-tvf test.tar# From the output, we can see that there is an additional update in the tar package and the install. log file is displayed.
-Rw-r -- root/root 183 2011-11-11 users
-Rw-r -- root/root 279 2011-11-11 users2
-Rw-r -- root/root 48217 install. log
-Rw-r -- root/root 48217 install. log

/> Tar -- delete install. log-f test.tar# Based on the above results, delete install. log from the compressed package
-Rw-r -- root/root 183 2011-11-11 users
-Rw-r -- root/root 279 2011-11-11 users2

/> Rm-f users users2# Delete two files in tar from the current directory
/> Tar-xvf test.tar# Decompress
/> Ls-l users *# List details of users and users2 only
-Rw-r --. 1 root 183 Nov 11 08:02 users
-Rw-r --. 1 root 279 Nov 11 08:45 users2

Bytes
/> Tar-cvzf test.tar.gz *
/> Tar-tzvf test.tar.gz # Add the z option when viewing the file list in the compressed package (gzip format)
-Rw-r -- root/root 48217 install. log
-Rw-r -- root/root 183 2011-11-11 users
-Rw-r -- root/root 279 2011-11-11 users2

/> Rm-f users users2 install. log
/> Tar-xzvf test.tar.gz# Decompress the package in gzip format
/> Ls-l *. log users *
-Rw-r -- root/root 48217 install. log
-Rw-r -- root/root 183 2011-11-11 users
-Rw-r -- root/root 279 2011-11-11 users2

/> Rm-f test .*# Delete the original compressed package files in the current directory
Bytes
/> Tar-cvjf test.tar.bz2 *
/> Tar-tjvf test.tar.bz2# Add the j option when viewing the file list in the compressed package (bzip2 format)
-Rw-r -- root/root 48217 install. log
-Rw-r -- root/root 183 2011-11-11 users
-Rw-r -- root/root 279 2011-11-11 users2

/> Rm-f *. log user *
/> Tar-xjvf test.tar.bz2# Decompress the package in bzip2 format
/> Ls-l
-Rw-r --. 1 root 48217 Nov 11 install. log
-Rw-r --. 1 root 10530 Nov 11 23:08 test.tar.bz2
-Rw-r --. 1 root 183 Nov 11 08:02 users
-Rw-r --. 1 root 279 Nov 11 08:45 users2

Fifteen. Large file splitting command split:

The following lists the command line options that are most commonly used for this command:

Option description-l specifies the number of rows, which are separated into one file. The default value is 1000. -B indicates the number of bytes. The supported units are k and m-C, which are similar to the-B parameter. However, when cutting, maintain the integrity of each line. The suffix of the file generated by-d is a number, if this option is not specified, the default value is a letter.

/> Ls-l
-Rw-r --. 1 root 10530 Nov 11 23:08 test.tar.bz2

/> Split-B 5 k test.tar.bz2Cutting test.tar.bz2 with 5kof each file
/> Ls-l# View the cut result. By default, the split file name is in the following format.
-Rw-r --. 1 root 10530 Nov 11 23:08 test.tar.bz2
-Rw-r --. 1 root 5120 Nov 11 23:34 xaa
-Rw-r --. 1 root 5120 Nov 11 23:34 xab
-Rw-r --. 1 root 290 Nov 11 23:34 xac

/> Rm-f x *# Delete small files after splitting
/> Split-d-B 5 k test.tar.bz2# The-d option is suffixed with a number to name the small file after the split
/> Ls-l
-Rw-r --. 1 root 10530 Nov 11 23:08 test.tar.bz2
-Rw-r --. 1 root 5120 Nov 11 23:36 x00
-Rw-r --. 1 root 5120 Nov 11 23:36 x01
-Rw-r --. 1 root 290 Nov 11 23:36 x02

/> Wc install. log-l# Calculating the number of rows of the file
/> Split-l 300 install. log# Split each 300 rows into a small file
/> Ls-l x *
-Rw-r --. 1 root 11184 Nov 11 23:42 xaa
-Rw-r --. 1 root 10805 Nov 11 23: 42 xab
-Rw-r --. 1 root 12340 Nov 11 23:42 xac
-Rw-r --. 1 root 11783 Nov 11 23: 42 xad
-Rw-r --. 1 root 2105 Nov 11 23:42 xae