Sed && awk tools and some frequently used shell scripts

(i) SED

Sed is a streamlined, non-interactive, streaming editor that enters edit commands and specifies file names on the command line. Then look at the output on the screen.

The progressive reading of the file contents is stored in a temporary buffer called pattern space. Then, the contents of the buffer are processed with the SED command, and the contents of the buffer are sent to the screen after processing. Proceed to the next line, which repeats itself until the end of the file. The original file love your content has not changed.

Sed ' 4, $d ' test.in # delete 4~ last line sed ' 3q ' test.in # read to the specified line and exit sed ' s/public/public/' test.in # Replace public for Publicsed-n ' s/public/ public/p ' test.in #打印匹配行

sed [options] sed_cmd files

-e Connect multiple edit commands
-f Specifies the SED script file name
-N prevents input lines from being actively output

Often grep can also achieve results. For example, count the number of rows in the specified file.
count=$ (cat $FILE | grep-a "keyword" | wc-l)

Three ways to specify multiple commands:
1. Separating commands with semicolons

Sed ' s/public/public/;s/north/north/' test.in

2. Place e before each command

Sed-e ' s/public/public '-e ' S/north/north ' test.in

3. Using the branch command function, a multiline prompt (>) will appear when you press ENTER after entering a single quote

Sed ' > s/public/public/> s/northnorth ' test.in

frequently used positional commands:

Watermark/2/text/ahr0cdovl2jsb2cuy3nkbi5uzxqv/font/5a6l5l2t/fontsize/400/fill/i0jbqkfcma==/dissolve/70/gravity /center ">

Common editing commands:

P: Print matching line sed-n ' 3,5p ' test,in=: Displays the line number of the matching row sed-n '/north/= ' test.ind: Delete the matching row sed-n '/north/d ' test.ina\: Append one or more lines of text after the specified line, and display New content added to the display.
Sed '/NORTH/A\AAA>BBB>CCC ' test.ini\: Inserts one or more lines of text before the specified line. and display the new content, ibid.
C\: Replaces the specified line with the new text, using the format above L: Displays all the characters in the specified line, including the control character (nonprinting character) sed-n '/north/l ' test.ins: Replace command. Use format for [address] s/old/new/[gpw]address: suppose omitted. Represents editing all rows g: Global substitution P: Print the modified line W fname: Writes the replaced content to the specified file Sed-n ' s/north/north/gp ' test.insed-n ' s/north/north/w data ' test.in The sed ' s/[0-9][0-9]$/&.5/' datafile& symbol, when used in a replacement string, represents the substituted string R: Read the file, append the contents of another file to the specified line after sed ' $r data ' test.inw: Write the file. Writes the specified line to another file Sed-n '/public/w date2 ' Test.inn: reads the following line of the specified line into the edit buffer sed-n '/public/{n;s/north/north/p} ' test.in when using multiple sed editing commands at the same time for a specified line, enclose the braces, separating the commands with semicolons; Q: Exit, after reading to the specified line, exit sedsed ' 3q ' test.in

(ii) awk

Awk is able to work with columns
Simple way to use:
awk [Options] sed_script files
-f Specifies the delimiter for the input record field, the value of the default environment variable IFS
-F reads from the specified file Awk_script
-V Sets the variable for awk

Awk-f: ' {print $} '/etc/passwdawk-f: '/root{print $ ' | ' $ '/etc/passwd

Awk runs the process:
1. Assume that the Begin,awk first runs the actions he has specified
2. Awk reads a line from the input, called a message record
3. Awk cuts the recorded records into several fields and puts the first field in the variable $ $. The second one is put in the variable, and so on, and the whole record is represented by a. The field delimiter can be specified by the option-F, otherwise the default delimiter is used.
4. Compare the current input records sequentially with the pattern in each awk_cmd:
The assumption matches. Then run the appropriate actions
If the hypothesis does not match, skip the corresponding actions until all the Awk_cmd are complete.
5. When an input record is complete, awk reads the next line of input, repeating the process above. All processing is completed until all inputs are processed.
6. When awk processes the input of the complete section, there is an end. Run the corresponding actions
7. Assume that the input is a list of files. AWK will process each file in the list sequentially

Examples of awk processing:

Ifconfig | awk '/inet addr/{print $} ' | Awk-f: ' {print-$} ' ifconfig | awk '/inet addr/{print $ ' |awk-f: ' Begin {print ' Begin ... '} ' {print $ {} end {print ' End ... '} '

in addition. Awk can also handle compound expressions:

Watermark/2/text/ahr0cdovl2jsb2cuy3nkbi5uzxqv/font/5a6l5l2t/fontsize/400/fill/i0jbqkfcma==/dissolve/70/gravity /center ">

Finally, awk is not just a command. It's more of a programming language
Variable:
1. Internal variable awk ' {print nr,$0} ' #给文件加上行号
2. Define your own variables
Function:
1. Built-in functions
2. Define your own functions
awk ' {print sum ($1,$2)} function sum (x, y) {S=x+y;return s} ' grade.txt
Array
awk ' BEGIN {print split ("123#456", arr, "#"); for (i in arr) {print arr[i]}} '

Because these are more complex, this article is no longer introduced.

finally , a few simple shell scripts are attached. A lot of other examples are on my Github .

(1) Comparison size

#!/bin/bashecho "Please input the numbers:" Read aread bif (($a = = $b)), then    echo "a = B" elif (($a > $b)) and then    E Cho "a > B" Else    echo "A < B" fi

(2) Find Files

#!/bin/bashecho "Enter a file name:" Read AIF [-e/home/tach/$a];then    echo "The file is exist!" else    echo "The file is not exist!" Fi

(3) Delete files of size 0

#!/bin/bashfor filename in ' ls ' does         a=$ (ls-l $filename | awk ' {print $} ')        if (a==0), then            rm $filename        F Idone

(4) View native IP address

#!/bin/bashifconfig | grep "inet Address:" | awk ' {print $} ' | Sed ' s/address: '//g

(5) IP address legality inference

#!/bin/bashcheckipaddr () {   #IP地址需由三个. Delimited and all digital   echo $ | grep ^[0-9]\{1,3\}\.\ ([0-9]\{1,3\}\.\) \{2\}[0-9]\{ 1,3\}$ ">/dev/null   if [$?-ne 0];then       return 1   fi   ipaddr=$1   a= ' echo $ipaddr | awk-f. ' {print $ } '   b= ' echo $ipaddr | awk-f ' {print $} '   c= ' echo $ipaddr | awk-f. ' {print $} '   d= ' echo $ipaddr | awk-f. ' {print $4} ' for   num in $a $b $c $d do       if [$num-gt 255-o $num-lt 0];then           return 1       fi   D One   return 0}if [$#-ne 1];then    echo "Usage: $ ipaddr."    Exitelse    checkipaddr $    ans=$?    If [$ans-eq 0];then        echo "Legal IP address."    else        echo "Unlegal IP address."    Fifi

(6) Other

#!/bin/bash# Display current date and time echo ' date +%y-%m-%d-%h:%m:%s ' #查看哪个IP地址连接的最多netstat-an | grep established | awk ' {print $} ' |awk-f: ' {print '} ' | Sort | Uniq-c#awk do not sort delete repeatedly line awk '!x[$0]++ ' filename #x仅仅是一个数据參数的名字, is a map that does the logical inference specified. If the logical inference is true, run the specified command; No, skip this line. # See the 10 UNIX commands most commonly used awk ' {print $ ' ~/.bash_history | Sort | uniq-c | Sort-rn | The-R in Head-n 10#sort is descending, _-n is sorted by value (default comparison character, 10<2) #逆序查看文件cat 1.txt | awk ' {a[i++]=$0} END {for (j=i-1; j>=0;) print a[j--]} ' #查看第3到6行awk ' Nr >=3 && nr <=6 ' filename #crontab文 Some demo samples of the pieces * * * */usr/local/etc/rc.d/lighttpd restart #每晚9.30 Restart Apache45 4 1,10,22 */usr/local/etc/rc.d/lighttpd re   Start #每月的1, 10, 22nd of 4:45 Restart APACHE10 1 * * 6,0/usr/local/etc/rc.d/lighttpd restart #每周六, 1:10 restart of the day apache0,30 18-23 * * * /USR/LOCAL/ETC/RC.D/LIGHTTPD Restart #表示在每天18.00 to 23.00 restarts every 30 minutes apache* 23-7/1 * */USR/LOCAL/ETC/RC.D/LIGHTTPD RE Start #晚上11点到早上7点之间. Restart Apache every hour

Sed && awk tools and some frequently used shell scripts