First, awk
Command format:
awk ' Begin{commands} pattern {commands} end{commands} ' file
Working mode:
1. Execute Begin{commands} statement block statement, optional statement block
2. Read a line from the file or stdin, and then execute {commands}, repeating the process until the file is all read out
3. When reading to the end of the input stream, execute the end{commands} statement block
Special variables:
Filename:awk Browse by file name
NR: Number of records, corresponding to the current line number during execution
NF: Number of fields, number of fields corresponding to the current line during execution
FS: Set delimiter, command line-F
$: text content of the current line during execution
$: The text content of the first field
$: The text content of the second field, and so on
print&printf functions for printing output
When the parameters of print are delimited by commas, the parameters are printed with spaces as delimiters, and in Awk's print statement, the double quotes are used as concatenation operators.
The printf function, whose usage is basically similar to printf in the C language, can format strings, and when the output is complex, printf is more useful and the code more understandable.
Process Control Statements
if (condition) else{}
while () {}
Do{}while ()
for (;;) such as
Built-in string control functions:
1.length (String): Returns the length of a string
2.index (string,search_string): Returns the position where the search_string appears in the string
3.substr (String,start_pos,end-pos): Creates a substring from start-pos to End-pos position in a string
4.split (String,array,delimiter): Generates a list of strings with delimiter and stores the list in an array, delimiter using the current FS value by default.
5.sub (regex,replace_str,string): Replace the first content of the regular expression with REPLACE_STR
6.gsub (regex,replace_str,string): replace regular expression to match all content
7.match (regex,string): detects if regular expressions can match strings
Application:
1. Read a line from a file or stdin and loop through {commands} until you finish reading
echo "Axbx cxdxe a;b;cx d;e" | Awk-f "[x|;|] + "' Begin{print" BEGIN: "} {for (i=1;i<=nf;++i) {++s[$i];p rintf (" $%d=%s\n ", I, $i)}} end {print" End: "; For (i in S) print I, S[i]} '
2.split function
Ii. SED
SED syntax format:
Sed[options] ' command ' file (s)
Sed common options:
-N: Only the line (or action) that has been specially processed by SED is listed.
-E: Combining multiple SED commands
-F: Perform sed action within filename
-r:sed's actions support the syntax of extended formal notation.
-I: Directly modifies the contents of the read file, not the screen output
Sed common commands:
A: Append sed ' a xxx ' file after each line append xxxx sed ' $a xxx ' file last line append
I: Insert the sed ' i xxx ' file before inserting xxx into each line
C: replace sed ' 2c xxx ' file second line with XXX instead
D: Delete sed '/^$/d ' file delete blank line sed ' 1,5d ' file delete 第1-5 line
S: replace sed ' s/pattern/replace_str/g ' file/g means that SED will replace each match, otherwise it is the first match
&: A string that marks a matching style and can use matched content when replacing a string
\1 or \ 2:1th or 2nd matching string
Application:
1.&: A string that marks a matching style and can use matched content when replacing a string
2.\1 or \ 2:1th or 2nd matched string, \ (pattern\) used to match substrings, pattern is included in () escaped with slash
3. Combine multiple SED commands with pipelines
Sed ' expression ' | Sed ' expression '
Sed ' expression;expressiong '
Sed-e ' expression '-e ' expression '
Three, TR---translate conversion
TR Syntax Format:
TR [Options] Set1 Set2
Principle: Mapping Set1 into Set2 through the concept of set mappings
-Common parameters
-D: Specifies the set of characters that need to be deleted
-C: Using the collection's complement
Tr–c Set1 Set2 uses the complement of Set1, that is, all characters that do not contain Set1
-S: Compresses repeated characters in the input, matches consecutive repeated words to a single character
Application:
1. Encrypt and decrypt
2. Adding digital files
Convert \ n in file to +, use $[operation] to perform arithmetic operations
3.tr Collection List
TR [: Class:] [: Class:]
Alnum: Letters and Numbers Alpha: letters
Digit: Digital Cntrl: Controlling (nonprinting) characters
Lower: Lowercase letter upper: Uppercase
Graph: Graphic character punct: punctuation
Space: Blank character xdigit: hexadecimal character
Iv. Find
Find: Traverse down the file hierarchy to match the eligible files and perform the appropriate actions
Common parameters
-print: Prints the file name (path) of the critical file, \ n is the delimiter for the output file
-print0:\0 (NULL) as the delimiter for the output file, xargs-0
-name: File name must match string, \ (-name "xx"-o-name "xx" \) matches multiple strings
-iname: Ignore letter Case
-path: Matching file path
-type: File type filtering
-size: File Size Search
-delete: Delete the found file
-perm: Matching a file with a specific permission value
-newer: Specify a reference file to compare timestamps, matching all files updated with reference files
-user: Developing a user file
-exec: With-exec and other commands, the command format is: commands {} \; Note the spaces between {} and \
-ok: Same as-exec, just operate in safe mode, press Y Yes, press n No
-atime: Access time, the last time the user visited the file, units: Days-amin units: minutes
-mtime: Modified time, the last time the file content was modified, units: Days –mmin units: minutes
-ctime: File (for example, permissions or ownership) the last time the change was in units: Days –cmin units: minutes
-Represents less than (most recent) + means greater than (more than) no-or + = digit Day
!: Represents a file that does not match the rule
Application:
1.-exec: With-exec and other commands, the command format is: commands {} \; Note the spaces between {} and \
Wu, Xargs
Xargs: Constructing command line arguments and running commands
The difference between the same pipe
The pipeline is implemented with the standard output of the previous standard input as the back
Xargs is a parameter that implements standard input as a command
The default command for Xargs is Echo, and the default delimiter is a space (white space character)
This means that if the input passed to Xargs contains line breaks and white space characters, line breaks and whitespace will be replaced by a space by Xargs, so Xargs is one of the important components of building a single-line command
Common parameters
-N Number: Multi-line output, multiple parameters
-D X: Custom delimiter
-I. | -I replace_str: REPLACE_STR is replaced with each line parameter from standard input when Xargs is extended
-L Number: standard input reads non-empty number rows as parameters,-l 1 is equivalent to-I
-P: asking whether to run
-T: Enable trace mode
-S size: Sets the maximum number of bytes for the command line
-X: Stop running the xargs command if any command line is greater than the number of bytes specified by the-s Size flag
Application:
1.-n Number: Sets the parameters of the command
2.-i replace_str: Command parameter substitution
3.xargs and find are a couple of buddies.
-print0 and Xargs of Find-0
PS: As long as the output of find is used as input to Xargs, the-print0 must be combined with find, separated by the character null (+), and xargs-0 null as the delimiter
Simple operation of Linux commands