Notes for common Linux commands

Introduction to find and xargs commands

Find basic command syntax:Find path-options [-print-exec-OK]

Pathname: directory path searched by the find command. For example, "." is used to represent the current directory, "..." is used to represent the upper-level directory, and "/" is used to represent the system root directory.
-Print: The find command outputs matching files to the standard output.
-Exec: The find command executes the shell command given by this parameter on the matching file. The corresponding command is in the form of 'command' {}\;. Note the space between {} And. However, in some systems, only LS and other operations are allowed, and the execution speed of-exec is slow. We recommend that you use xargs to perform the same functions.
-OK: The Role of-exec is the same, but the shell command given by this parameter is executed in a safer mode. A prompt is displayed before each command is executed, let the user determine whether to execute.

-Options:

-Name: searches for files by file name.
-Prem: searches for files based on their permissions.
-User: search by file owner.
-Group: search by the group to which the file belongs.
-Amin: searches for files that have been accessed in the last n minutes.
-Atime: searches for files that have been accessed for the last n * 24 hours.
-Mmin: searches for files modified in the last n minutes.
-Mtime-n (+ n): searches for Objects Based on their modification time. -N indicates that the file change time is less than n days ago, and + n indicates that the file change time is earlier than n days ago.
-Cmin: searches for files whose status has changed in the last n minutes.
-Ctime: searches for files in the changed status in the last n * 24 hours.
-Nouser: Find the file without a valid owner, that is, the file owner does not exist in/etc/password.
-Nogroup: Find files without a valid group, that is, the file group does not exist in/etc/group.
-Newer file1! File2 searches for files whose change time is newer than file1 but older than file2.
-Type: searches for files of a certain type, including:

B	Block Device Files
D	Directory
C	Character Device File
P	Named Pipe file (FIFO)
L	Symbolic Link file
S	Socket File
F	Common File

-Size n [c]: searches for files with a length of n blocks. If a file contains c, the file length is measured in bytes.
-Maxdepth (-mindepth) levels: Find the directories at the maximum (minimum) layers.
-Fstype: searches for files in a certain type of file system. These file system types can usually be found in the configuration file/e t c/f s t a B, this configuration file contains information about the file system in the system.

 

Xargs is used to accept the input of stdin and compress it into a command line for execution.

Application Instance

Search for all common files in the current directory and display them in a long list.

find ./ -type f -exec ls -l {} \;find ./ -type f | xargs ls -l

Search for all common files in the current directory. The owner has the read and write permissions and the users in the file group and other users have the read permissions. The files are displayed in a long list:

find ./ -type f -perm 644 -exec ls -l {} \;find ./ -type f -perm 644 | xargs ls -l

Ignore a directory. If you want to ignore a directory when searching for a file, you can use the-prune option to specify the directory to be ignored. Be careful when using this option, because if you use the-depth option at the same time, the-prune option will be ignored by the find command.

Find/home-path "/home/Tom"-prune-o-print # search for files in/home, but you do not want to find/home/SAM \ (-path/home/SAM/dir1-o-path/home/SAM/dir2 \) in the/home/Tom directory \) -prune-o-print # search for files under/home/Sam, but avoid/home/SAM/dir1 and/home/SAM/dir2

Other instances

Find/-user Sam # Find/-Amin-10 for all files with Sam as the system owner # Find/-mmin-the files that have been accessed for the last 10 minutes in the system- 5 # Find/-atime-2 files modified in the last 5 minutes of the system # Find/-mtime-2 files accessed in the last 48 hours of the system # search in find/-empty # Find a file or folder empty in the system '*. core' | xargs echo-N 1 "">/tmp/core. log # search for the memory dump file (core dump) in the entire system, and save the result to/tmp/core. log. Find/-type F | xargs grep "hostname" # Find/-size + 100c files containing the word hostname in all files # Find/-size for all files larger than bytes + 500kc-and-size-1000kc # search for files larger than 500kb and smaller than 1000kb

Grep command Introduction

Grep basic syntax:Grep-options regex files

-Options:

-A n: the output matches the row with the last n rows.
-B N: the output matches the row with the first n rows.
-C only outputs the Count of matched rows.
-I is case insensitive (only applicable to single characters ).
-H: When querying multiple files, the file name is not displayed.
-L only names containing matching characters are output when multiple files are queried.
-N: the matching row and row number are displayed.
-S does not display the error message that does not exist or does not match the text.
-V: displays all rows that do not contain matched text.
-R performs recursive search on directories. Note that this option is not POSIX.
-O only outputs RE-compliant strings. (the new version of gnu is unique and not necessarily supported by all versions .)
-W integral word comparison, similar \ .
-Q silent mode. No result is output (except stderr. Usually used to get return value (or stored in the environment variable $? ), True; otherwise, false .)
-E switch to egrep.

For regex, see "regular expressions"

Application Instance

Grep "tom"/etc/passwd # Find the user information in the local machine where the user name is tom ps waux | grep httpd # Find information about all httpd processes in the system grep-E '(begin | start) '* # search for all the files containing the begin or start character grep-nr "sys_open \ B" fs/# Recursively search for the string "sys_open" grep-nr "foo \ B in the fs directory ". /| grep-v "bar \ B" # search for files containing "foo" but not "bar" in the current directory

Introduction to sort, uniq, and cutsort commands

Sort command format:Sort-cmu-o output_file [other options] + pos1 + pos2 input_files

Parameter introduction:
-C: whether the test file has been classified.
-M combines two classification files.
-U: delete all duplicate rows.
-O stores the output file name of the sort result.

-When B uses a domain for classification, the first space is ignored.
-N indicates that the category is sorted by number on the domain (by default, it is sorted by letter dictionary ).
-T domain separator; use non-spaces or tab keys to separate fields.
-R calculates the inverse of the classification order or comparison.
+ N is the domain number. Use this domain number to start classification.
-K specifies the column number, which is generally used with-t.
Post1 is passed to m, n. M is the domain number, and n is the number of characters that start to be classified. For example, 4 or 6 means that the data is classified by 5th domains and starts from 7th characters.

About uniq commands

Uniq is used to remove or disable duplicate rows from a text file. In general, uniq assumes that the files are sorted and the results are correct (So uniq is often used with sort ).
Uniq command format:Uniq-options INPUT [OUTPUT]

-Options:

-C	Displays the number of rows, that is, the number of duplicate rows.
-D	Display duplicate data rows
-U	Show only non-repeated rows
-F	N is a number and the first n fields are ignored.

Introduction to cut commands

Cut is used to cut columns or fields from standard input or text files. Cut text can be pasted into a text file.
Cut command format:Cut-options file1 file2

-Options:-specifies the number of bytes selected by B bytes;-c list specifies the number of cut characters;
-F field specifies the number of cut fields;
-D specifies the domain separator different from the space and tab key;
-C is used to specify the cut range, as shown below:
-C 1st-7 Cut 5th characters, followed by 7th to characters;
-Cut the first 50 characters in the c1-50;
-F is in the same format as-c;
-F 1st: Cut 5th domains, domains;
-F 1st-12 cut the 10th domain, 12th domain to domain;
-S does not print the output for rows that do not contain delimiters.

Application Instance

Sort-u file1-O file2 # Remove the duplicate row in the file file1 and store it in file2sort file1 | uniq> file2 # Remove the duplicate row in the file file1, save to file2sort file1 | uniq-c | sort-NK1 # calculate the number of occurrences of the same row in file1, and sort-K according to the number of occurrences: -T 2 file # Sort the files by 2nd columns separated by ":". Sort-t. -K 1, 1N-K 2, 2n-K 3, 3n-K 4, 4n ipaddresses.txt # Sort the IP addresses in cut-D. -f1-4 file # put the file according to ". "extract the first to fourth columns of the split cut-s-d:-F2, 5-7 file # split the file into the second column by": ", and retrieve the fifth to seventh columns, does not output rows that do not contain ":"

Awk

Awk contains too much content.

Application Instance

• Retrieve domain data (awk uses space and tab by default as the domain delimiter. If both are present, the data is separated by space; you can also use the-F parameter to manually specify the delimiter)

Awk '{print $0}' file # output the entire article awk '{print $1}' file # output the first column awk-F "\ t" '{print $2 }' # retrieve the 2nd Column awk-F separated by tab ": "'{print $ NF}' file # retrieve according to": "The last column awk-F" "'{print NF}' file # output the number of columns in each row divided by space. awk-F" \ t "'{print NF }' file # output the number of columns in each row split by tab. awk '{print $0 "\ n"}' file # insert an empty line between each row of the file awk-F ": "'{If (NF> 6) Print $0 ;}' # extract rows with more than 6 columns separated ":"

Value Range

Awk '{If ($2 ~ /^ Begin/) & ($3 <10) Print $ NF;} 'filename # Starts Column 2nd with "begin, and awk-F "\ t" '{if ($3> 5) Print $0 ;} 'filename# extract the rows with 3rd columns and more than 5 columns separated by Tab

Others

Awk '{$ NF = ""; print $0} 'filename# output awk '{print NR "\ t" $0} 'file1 | sort-nk1-r | awk for all columns except the last -F "\ t" '{print $2}' # output a file in reverse order of the row (the first row is the last row, the last row is the first line) awk-F "\ t" '{SUM + = $2} END {print SUM}' filename # adds the 2nd columns separated by tab, output and ps a | awk '{print $5}' | sort | uniq-c | sort-nk1 # view all processes in the system, and sorted by process name running times from small to large

Sort the following files in ascending order based on their total attributes:

$cat sampleJohn 5Jam 10Tom 20Jam 34Tom 10John 6Jerry 2$awk '{array[$1]+=$2}END{for(i in array) print i" "array[i]}' sample |sort -n -k 2

Other Application tips: log keyword coloring monitoring

When monitoring the rolling logs of the application server, you often need to color and display the keywords of the logs to facilitate monitoring of server errors. For example, the showlist script that monitors maillog of the mail server:

Send the mail server Postfix running report every day

Use pflogsumm to monitor the running status of Postfix yesterday (pflogsumm needs to be installed), and then add Automatic Execution in/etc/crontab:

0 2 * * * root zcat /var/log/maillog.0.bz2 | pflogsumm -d yesterday | mail -s "Mail Report from `hostname` on `data`" user@domain.com