Shell Advanced 2

Eleven. AWK Programming:
1. Variables:
The awk variable is defined when it is assigned.
The type of the variable can be a number, a string.
The value of the uninitialized variable is 0 or the blank string "",
List of assignment symbols for variables

=+=-=
*=/=%=^=

awk ' $ ~/tom/{wage = $ $ * $ $; print wage} ' filename
awk ' {$ = $ * $/$; print} ' filename

Twelve. Sort commands for rows sort:
1. Sort command-line options:
The delimiter between-t fields
-F ignores case when sorting based on character
-K defines the sorted field fields or is sorted based on partial data of field fields
-M merges the sorted input file into a sorted output data stream
-N Comparing fields with integer types
-o outfile writes output to the specified file
-R The order of the inverted sort is from large to small, normal sort is from small to large
-U has only a unique record, discarding all records with the same key value
-B ignores the preceding spaces

2. Sort usage Examples:

Sort-t ': '-K 1 users
-T defines the delimiter between a colon and a domain field, and the-K 2 specifies a forward sort based on the second field (field order starting from 1).

Sort-t ': '-K 3r Users
Or a colon as a delimiter, this time based on the third field to sort upside down.

Sort-t ': '-K 6.2,6.4-k 1r users
A forward sort is preceded by the 2nd character of the sixth field to the 4th character, and is reversed based on the first field.

Sort-t ': '-K 6.2,6.4-k 1 users
A forward sort is made from the 2nd character of the sixth field to the 4th character, and the first field is based on a forward sort.
Compared to the previous example, the 4th and 5th lines exchanged positions.

Sort-t ': '-K 1.2,1.2 users
Sort 2nd characters based on the first field

Sort-t ': '-K 6.2,6.4-u users
A forward sort based on the 2nd character to the 4th character of the sixth field, and the-u command requires that duplicate rows of key values be removed when sorting

Sort-t ': '-K 3 users2
Sort based on 3rd field fields in text form

Sort-t ': '-K 3n users2
Sort based on the 3rd field fields in numeric form

Ps-ef | Sort-k 1-o Result
Sorts based on the owner name of the current system execution process and writes the results of the order to the result file

13. To delete a duplicate line command uniq:
Uniq common options are the following

-C To add the number of times the row repeats before each output line
-D Show only duplicate rows
-U appears as a repeating row

Sort Testfile | Uniq-c
|uniq-dc
Uniq-uc

14. File Compression decompression Command tar:

-C Create Compressed archives
-X Decompression
--delete deletes an existing file from the compressed package, and if the file appears multiple times in the package, it will all be deleted.
-T view a list of files in a compressed package
-R append files to the end of the archive file
-U update files in original compressed package
-Z Compression to gzip format, or unzip in gzip format
-J compressed to bzip2 format, or extracted in bzip2 format
-V shows the process of compression or decompression, which is generally not suitable for background operations
-F Use the file name, this parameter is the last parameter, can only be followed by the file name.

TAR-CVF Test.tar *
TAR-RVF Test.tar *.log
TAR-TVF Test.tar
TAR-UVF Test.tar *.log
Tar--delete install.log-f test.tar# remove Install.log from compressed package
TAR-CVZF test.tar.gz *
TAR-TZVF test.tar.gz
TAR-JCVF test.tar.bz2 *

xv. Large file Split command split:
-l Specifies the number of rows, each separated into a single file, and the default value is 1000 rows.
-b Specifies the number of bytes, supported in the following units: K and M
-C is similar to the-B parameter, but maintains the integrity of each line as much as possible when cutting
-d The suffix of the generated file is a number, and if you do not specify this option, the default is the letter

Split-d-B 5k test.tar.bz2 #-d option to name the split small file after the suffix as a number
Split-l Install.log #每300行拆分成一个小文件

16. File Find command find:
1. Search by file name:
-name: File name is case sensitive when looking up.
-iname: File name case is not sensitive when looking up.

Find. -name "*.log"

2. Search by file time attribute:
-atime-n[+n]: Find out the file access time within n days [outside] of the file.
-ctime-n[+n]: Find files where the file change time is within n days [outside].
-mtime-n[+n]: Find the file that modifies the data time within n days [outside].

-amin-n[+n]: Find file access time within n minutes [outside].
-cmin-n[+n]: Find files where the file change time is within n minutes [].
-mmin-n[+n]: Find the file that modifies the data time within n minutes [].

Find-ctime-2 #找出距此时2天之内创建的文件
Find-ctime +2 #找出距此时2天之前创建的文件
Find. -cmin-3 #找出修改状态时间在3分钟之内的文件.

3. Perform the specified action based on the found file:
-exec: Executes the shell command given by the parameter to the matching file. The corresponding command is in the form of ' command ' {} \;
Note the space between {} and \; and there are no spaces between two {}
-ok: The main function and syntax format is exactly the same as the-exec, which is prompted each time the shell command is executed.
The subsequent shell command will be executed only if the answer is Y.
It is necessary to note that this option does not apply to automation scripts because the offer may suspend the entire automation process.

Find. -ctime-2-exec ls-l {} \;
Find the files created within 2 days of this time, and based on the results of Find, apply the command after-exec, i.e. Ls-l,
This allows you to display a clear list of find files directly.

Find. -name "*.log"-mtime-1-exec rm-f {} \;
Find. -name "*.dat"-mtime-1-ok rm-f {} \;

4. Search by Owner and group to which the file belongs
Find. -user Root #搜索owner是root的文件
Find. ! -user Root #搜索owner不是root的文件, note that there are spaces between and-user.
Find. -group Root #搜索所属组是root的文件
Find. ! -group Root #搜索所属组不是root的文件, note that there are spaces between and-user.

5. Search by the specified directory depth:
-maxdepth: The following parameter represents the depth specified from the current directory, where 1 represents the current directory, 2 represents a primary subdirectory, and so on.
Find. -maxdepth 1-name "*"
Find. -maxdepth 2-name "*"

6. Exclude the specified subdirectory lookup:
-path Pathname-prune: Avoid specifying subdirectories pathname lookups.
-path Expression-prune: Avoids a set of pathname lookups specified in the expression.
It is necessary to note that if you use the-depth option at the same time, then-prune will be ignored by the Find command.

Find. -path "./dontsearchpath"-prune-o-name "datafile*"-print
Under current directory, avoid the Dontsearchpath subdirectory and search for all files named datafile*.

Find. \ (-path "./dontsearchpath"-o-path "./dosearchpath" \)-prune-o-name "datafile*"-print
The current directory, while avoiding the Dontsearchpath and Dosearchpath two subdirectories, search all files named datafile* file

7. Search by file permission property:
-perm mode: File permissions exactly match mode (mode is the octal representation of file permissions).
-perm +mode: The file Permissions section conforms to mode. If the command parameter is 644 (-rw-r--r--), then as long as there are any permissions in the file permission attribute and 644 overlap, such files can be selected.
-perm-mode: File permissions are fully compliant with mode. If the command parameter is 644 (-rw-r--r--), when the permission specified in 644 has been fully owned by the current file, and the file has additional permission attributes, such a file can be selected.

Find. -perm 644 #查找所有文件权限正好为644 (-rw-r--r--) file.
Find. -perm-444 #644所包含的权限完全覆盖444所表示的权限

8. Search by File type:
-type: Specifies the type of file later.
B-block device files.
D-Directory.
C-character device file.
P-Pipeline file.
L-Symbolic link file.
F-Normal file.

9. Search by File Size:
-size [+/-]100[c/k/m/g]: Represents a file with a length equal to [greater than/less than] 100 blocks [byte/k/m/g].
-empty: Find empty files.

Find. -size +4k-exec ls-l {} \; #查找文件大小大于4k的文件, print out the details of the found file at the same time
Find. -size-4k-exec ls-l {} \; #查找文件大小小于4k的文件
Find. -size 183c-exec ls-l {} \; #查找文件大小等于183字节的文件.
Find. -empty-type f-exec ls-l {} \;

10. By changing the time than the specified file new or older than the way the file is found
-newer file1! File2: Find file changed date is newer than file1, but older than file2 file.
Ls-lrt
Find. -newer Users
Find. ! -newer users2 #查找文件更改日期不比users1新的文件
Find. -newer users2! -newer test.tar.bz2

17. Xargs command:
The Find command passes all matching files to exec, and some systems have a limit on the length of the command that can be passed to exec
The Xargs command takes only a subset of the files at a time, not all, unlike the-exec option. This allows it to first process a portion of the file that was first fetched, then the next batch, and so on.
In some systems, the use of the-EXEC option initiates a corresponding process for processing each matching file, not all of the matching files are executed once as parameters, so that in some cases there will be too many processes and degraded system performance, so the efficiency is not high;
With the Xargs command, there is only one process. In addition, when using the Xargs command, whether to get all the parameters at once or to get the parameters in batches, and the number of parameters to get each time will be determined according to the command's options and the corresponding tunable parameters in the system kernel.

/> Find. -type F-print | Xargs file
#查找当前目录下的每一个普通文件, and then use the Xargs command to test which type of file they belong to.

/> Find. -type F-print | Xargs chmod a-x
#回收当前目录下所有普通文件的执行权限.

/> Find. -type F-print | Xargs grep "hostname"
#在当面目录下查找所有普通文件 and use the grep command to find the word hostname in the searched file.

/> Find/-name "core"-print | Xargs echo "" >/tmp/core.log
#在整个系统中查找内存信息转储文件 (core dump), and then save the results to the/tmp/core.log file.

/> Pgrep MySQL | Xargs kill-9 #直接杀掉mysql的进程

18. Shell commands related to system health:

1. Linux Real-time monitoring Command (watch):
watch-d-N 1 ' who ' #每隔一秒执行一次who命令 to monitor the status of current user logins on the server
-D indicates where the highlight has changed
-N indicates how many seconds the interval executes "command"

Watch can run multiple commands at the same time, separated by semicolons

watch-d-N 1 ' df-h; Ls-l '
Monitor the usage of the disk, and the status of the files in the current directory, including the new, deleted, and updated file modification dates.

2. View current system memory usage (FREE):
Common options
-B Displays the data in bytes.
-K Displays the data in kilobytes (KB) (the default value).
-m displays data in megabytes (MB).
-S delay this option will make the free continuous refresh, the interval between each refresh is the number of seconds specified by the delay,
If it contains a decimal point, it will be accurate to milliseconds, such as 0.5 for 500 milliseconds and 1 for one second.

The free command output contains the following columns:
Total totals the size of the physical memory.
Used the amount of memory that has been used.
The amount of memory available for free.
Shared memory total for multiple processes.
Buffers/cached the size of the disk cache.

Free-k-S 1.5 #以千字节 (KB) displays data at the same time, with every 1.5 refresh output, until you press CTRL + C to exit

3. CPU real-time monitoring tool (Mpstat):

Mpstat 2 5
The command will output the current health information of the CPU every 2 seconds, output 5 times,
If there is no second numeric argument, Mpstat will execute every two seconds until you press CTRL + C to exit.

Shell Advanced 2