1. Find File Lookup
Find txt and PDF files:
Find. (-name "*.txt"-o-name "*.pdf")-print
Regular way to find. txt and PDF:
Find. -regex ". * (. txt|. PDF) $ "
-iregex: Ignoring case-sensitive regular
Negative arguments to find all non-txt text:
Find. ! -name "*.txt"-print
Specify the search depth to print out the current directory file (depth is 1):
Find. -maxdepth 1-type F
Custom Search
Find. -type d-print//List all directories only
-type f File/L symbolic link/d catalogue
Find supports file retrieval types that can differentiate between common files and symbolic links, directories, and so on, but binary and text files cannot be distinguished directly by the type of find;
The file command can check the specific type of files (binary or text):
$file redis-cli # binary file redis-cli:elf 64-bit LSB executable, x86-64, version 1 (SYSV), dynamically linked (uses shared Lib s), for Gnu/linux 2.6.9, not stripped$file redis.pid # text file, Redis.pid:ASCII text
Therefore, you can use the following command combination to find all binaries in the local directory:
LS-LRT | awk ' {print $9} ' |xargs file|grep elf| awk ' {print '} ' |tr-d ': '
-atime access Time (in days, minutes units is-amin, similar to the following)
-mtime modification Time (content modified)
-ctime Change Time (metadata or permission changes)
Search by Time
All files accessed on the 7th day of the latest:
Find. -atime 7-type F-print
All files that have been visited in the last 7 days:
Find. -atime-7-type F-print
Search all files that were visited 7 days ago:
Find. -atime +7 Type F-print
W-word k M G looking for files larger than 2k:
Find. -type f-size +2k
Search by permissions:
Find. -type f-perm 644-print//Find all files with executable permissions
Search by User:
Find. -type f-user weber-print//Find the files owned by the user Weber
Follow-up action found after
Delete all SWP files in the current directory:
Find. -type f-name "*.SWP"-delete
Another syntax:
Find. Type F-name "*.SWP" | Xargs RM
Change ownership in the current directory to Weber:
Find. -type f-user root-exec chown Weber {};
Note: {} is a special string, for each matching file, {} will be replaced with the corresponding file name;
Copy all the found files to another directory:
Find. -type f-mtime +10-name "*.txt"-exec cp {} old;
If you need to execute multiple commands later, you can write multiple commands into a single script. The script can then be executed when the-exec is invoked:
-exec./commands.sh {};
Delimiter of the-print
Default use of ' ' as the delimiter for the file;
-print0 uses ' ' as a delimiter for the file, so you can search for files that contain spaces;
2. grep Text Search
grep match_patten File//default access matching line
Common parameters
-O only outputs matching lines of text VS -v outputs no matching lines of text
-N Prints matching line numbers
-I ignore case when searching
-L print File name only
Recursive search of text in a multilevel directory (the programmer searches for code favorites):
grep "Class". -r-n
Match multiple modes:
Grep-e "Class"-E "vitural" file
grep outputs a file name (-Z) with 0 as the trailing character:
grep "Test" file*-lz| xargs-0 RM
Integrated application: Finds all SQL lookups with a where condition in the log:
Cat log.* | TR A-Z | grep "from" | grep "WHERE" > B
Find Chinese Example: Project catalog in utf-8 format and gb2312 format two kinds of files, to find the word is Chinese;
Find its UTF-8 encoding and gb2312 encoding are e4b8ade69687 and D6D0CEC4 respectively.
Inquire:
GREP:GREP-RNP "\xe4\xb8\xad\xe6\x96\x87|\xd6\xd0\xce\xc4" * can
Chinese character coding query: http://bm.kdd.cc/
3. Xargs command line parameter conversion
Xargs can convert input data into command-line arguments for a particular command, so that it can be combined with a number of commands. Like grep, for example, find;-Convert multi-line output to single-line output
Cat file.txt| Xargs
n is the delimiter between multiple lines of text
Cat Single.txt | Xargs-n 3
-N: Specify the number of fields to display per row
Xargs parameter Description
-D defines delimiters (the delimiter is n for multiple lines by default)
-n Specifies that the output is multiple lines
-I {} Specifies the replacement string, which is replaced when the xargs extension is used, when multiple arguments are required for the command to be executed
-0: Specify 0 for Input delimiter
Example:
Cat File.txt | Xargs-i {}./command.sh-p {} -1# number of counting lines find source_dir/-type f-name "*.cpp"-print0 |xargs-0 Wc-l#redis store data via string, pass Over set storage index, all values need to be queried by index:./REDIS-CLI Smembers $ | awk ' {print '} ' |xargs-i {}./redis-cli get {}
4. Sort sorting
Field description
Example:
SORT-NRK 1 DATA.TXTSORT-BD Data//ignore leading whitespace characters such as spaces
5. Uniq Eliminate Duplicate rows
Sort Unsort.txt | Uniq
Sort Unsort.txt | Uniq-c
Sort Unsort.txt | Uniq-d
You can specify the duplicates that need to be compared in each row:-S start position-W comparison character number
6. Convert with TR
echo 12345 | Tr ' 0-9 ' 9876543210 '//Add decryption conversion, replace the corresponding character cat text| TR ' '//tab to space
Cat File | Tr-d ' 0-9 '//Delete all numbers
-C Seeking complement set
Cat File | Tr-c ' 0-9 '//Get all the numbers in the file cat file | Tr-d-C ' 0-9 '//Delete non-digital data
Tr-s the repeating characters that appear in compressed text; most commonly used to compress extra spaces:
Cat File | Tr-s "
How to use: TR [: Class:] [: Class:]
TR ' [: Lower:] ' [: Upper:] '
7. Cut split text by column
cut-f2,4 filename
CUT-F3--complement filename
Cat-f2-d ";" FileName
N-nth field to end
-M 1th Field M
N-m N to M Fields
Range of cut and take
Example:
Cut-c1-5 File//print first to 5th character cut-c-2 file//print first 2 characters
5th to 7th columns of truncated text
$echo String | Cut-c5-7
8. Paste concatenation of text by column
Stitch two text together by column;
Cat File112cat file2colinbookpaste file1 file21 colin2 Book
The default delimiter is a tab character, which can be specified with the-D delimiter:
Paste File1 file2-d "," 1,colin2,book
9. Tools for WC statistics lines and characters
$WC-L File//statistic number of rows $wc-w file//Statistics number of words $wc-c file//statistic characters
Sed text replacement weapon
Sed ' s/text/replace_text/' file//replace the first matching text of each line
Sed ' s/text/replace_text/g ' file
After the default substitution, output the replaced content, if you need to replace the original file directly, use-I:
Sed-i ' s/text/repalce_text/g ' file
Sed '/^$/d ' file
The matched string is referenced by the tag &.
echo this is en example | Sed ' s/w+/[&]/g ' $>[this] [is] [en] [Example]
The first matching parenthesis content is referenced using the tag one
Sed ' S/hello ([0-9])//'
Sed is usually quoted as a single quotation mark, or double quotation marks are used, and double quotation marks are used to evaluate an expression:
Sed ' s/$var/hlloe/'
When using double quotes, we can specify variables in the SED style and in the replacement string;
Eg:p=pattenr=replacedecho "line con a patten" | Sed "s/$p/$r/g" $>line con a replaced
String insertion character: Converts each line of content in the text (ABCDEF) to Abc/def:
Sed ' s/^. {3}/&//g ' file
One. awk Data Flow processing tool
awk ' begin{statements} statements2 end{statements} '
1. Execute the statement block in begin;
2. Read a line from the file or stdin, and then execute the STATEMENTS2, repeating the process until the file is fully read;
3. Execute the end statement block;
Print printing when moving forward
Echo-e "Line1line2" | awk ' Begin{print ' "start"} {print} end{print "END"} '
echo | awk ' {var1 = ' v1 '; var2 = "V2"; var3= "V3"; print var1, var2, var3;} ' $>V1 V2 v3
echo | awk ' {var1 = ' v1 '; var2 = "V2"; var3= "V3"; print var1 "-" var2 "-" VAR3;} ' $>v1-v2-v3
Special variable: NR NF $ $ $
NR: Indicates the number of records, in the course of the implementation of the forward number;
NF: Indicates the number of fields, the total number of fields that should go forward during the execution;
$: This variable contains the text content of the current line during execution;
$: The text content of the first field;
$: The text content of the second field;
Echo-e "Line1 F2 F3 line2 line 3" | awk ' {print NR ': ' $ '-' $ '-' $ '
awk ' {print $, $ $} ' file
awk ' END {print NR} ' file
Echo-e "1 2 3 4" | awk ' begin{num = 0;p rint "BEGIN";} {sum + = $;} END {print "= ="; Print sum} '
Passing external variables
Var=1000echo | awk ' {print Vara} ' vara= $var # input from Stdinawk ' {print Vara} ' vara= $var file # input from files
To filter the rows that awk handles with a style
awk ' NR < 5 ' #行号小于5awk ' nr==1,nr==4 {print} ' file #行号等于1和4的打印出来awk '/linux/' #包含linux文本的行 (can be specified with regular expressions, super powerful) awk '!/linu x/' #不包含linux文本的行
Set delimiter
Use-F to set the delimiter (the default is a space):
Awk-f: ' {print $NF} '/etc/passwd
Read command output
Using Getline, the output of the external shell command is read into the variable cmdout:
echo | awk ' {"grep root/etc/passwd" | getline cmdout; print Cmdout} '
Using loops in awk
for (i=0;i<10;i++) {print $i;} For (i in array) {print array[i];}
Eg: The following string, which prints out the time string:
2015_04_02 20:20:08:mysqli Connect failed, please check connect info$echo ' 2015_04_02 20:20:08:mysqli connect failed, pl Ease Check Connect info ' |awk-f ': ' {for (i=1;i<=;i++) printf ("%s:", $i)} ' >2015_04_02 20:20:08: # This way the last colon is printed out $ Echo ' 2015_04_02 20:20:08:mysqli connect failed, please check connect info ' |awk-f ': ' {print $ ': ' $ $ ': ' $ $;} ' >2015_04_02 20:20:08 # This way to meet the demand
And if you need to print out the rest of the section (the time section and the following text are printed separately):
$echo ' 2015_04_02 20:20:08:mysqli connect failed, please check connect info ' |awk-f ': ' {print $ ': ' $ $ ': $ $; Print $4;} ' >2015_04_02 20:20:08>mysqli Connect failed, please check connect info
Print lines in reverse order: (Implementation of the TAC command):
Seq 9| awk ' {LIFO[NR] = $ LNO=NR} end{for (; lno>-1;lno--) {print Lifo[lno];}} '
Awk combines grep to find the specified service and then kill it
ps-fe| grep MSV8 | Grep-v Mforward | awk ' {print $} ' | Xargs kill-9;
AWK implements head, tail commands
awk ' Nr<=10{print} ' filename
awk ' {buffer[nr%10] = $;} End{for (i=0;i<11;i++) {print buffer[i%10]}} ' filename
Print the specified column
LS-LRT | awk ' {print $6} '
LS-LRT | Cut-f6
Print the specified text area
Seq 100| awk ' Nr==4,nr==6{print} '
Print text between Start_pattern and End_pattern:
awk '/start_pattern/,/end_pattern/' filename
Example:
SEQ 100 | awk '/13/,/15/' cat/etc/passwd| awk '/mai.*mail/,/news.*news/'
awk common built-in functions
Index (string,search_string): Returns the position search_string appears in the string
Sub (regex,replacement_str,string): Replace the first content of the regular match with the REPLACEMENT_STR;
Match (regex,string): Checks if the regular expression matches the string;
Length (String): Returns the string length
echo | awk ' {"grep root/etc/passwd" | getline cmdout; print length (cmdout)} '
printf, similar to the C language, formats the output:
Seq 10 | awk ' {printf '->%4s ', ' $ '
12. Iterate over the lines, words, and characters in the file
1. Iterate through each line in the file
While read Line;doecho $line;d One < file.txt to sub shell:cat file.txt | (While read line;do echo $line;d one)
Cat file.txt| awk ' {print} '
2. Iterate through each word in a row
For word in $line;d oecho $word;d One
3. Iterate through each of the characters
${string:start_pos:num_of_chars}: Extracts a character from a string; (bash text slices)
${#word}: Returns the length of a variable word
For ((i=0;i<${#word};i++)) Doecho ${word:i:1);d One
Display files in ASCII characters:
$od-C filename
Common techniques for Linux text processing