Collection of 48 shell scripting tips _linux Shell

This article collects a bunch of shell scripting tricks, I said, I write a blog mainly to make some learning notes to facilitate their own review, so I will make such an article, there is nothing incomprehensible. About the origin of these techniques, eh, I also forget, may come from Theunixschool, Commandlinefu, cool and igigo.net, of course, there are some of my own experience, tube him, into my brain is my.

0. Shell debugging

Copy Code code as follows:

Sh-x somefile.sh

Add Set+x set-x to the somefile.sh file.
1. Use && | | Simplifying if Else

Copy Code code as follows:

Gzip-t a.tar.gz
if [[0 = $?]]; Then
echo "Good zip"
Else
echo "Bad zip"
Fi

Can be simplified to:

Copy Code code as follows:

Gzip-t a.tar.gz && echo "Good zip" | | echo "Bad zip"

2. Determine the file is not empty

Copy Code code as follows:

if [[s $file]]; Then
echo "Not Empty"
Fi

3. Get File Size

Copy Code code as follows:

Stat-c%s $file
Stat--printf= '%s\n ' $file
Wc-c $file

4. String replacement

Copy Code code as follows:

${string//pattern/replacement}
A= ' A,b,c '
echo ${a//,//}
5. Contains substring?
String= "My string"
if [[$string = = *my*]]; Then
echo "It ' s there!"
Fi

6. Rsync Backup

Copy Code code as follows:

Rsync-r-t-v/source_folder/destination_folder
Rsync-r-t-v/source_folder [User@host:/destination_folder

7. Batch renaming files

Add the. bak suffix to all txt files:

Copy Code code as follows:

Rename '. txt '. Txt.bak ' *.txt

Remove all the bak suffixes:

Copy Code code as follows:

Rename ' *.bak ' *.bak

Change all the blanks to underline:

Copy Code code as follows:

Find Path-type f-exec rename ' s//_/g ' {} \;

Change the file name to uppercase:

Copy Code code as follows:

Find Path-type f-exec rename ' y/a-z/a-z/' {} \;

8. For/while Cycle

Copy Code code as follows:

For ((I=0 i < i++)); do echo $i; Done
For line in $ (cat a.txt); do echo $line; Done
for f in *.txt; do Echo $f; Done
while read line; do echo $line; Done < A.txt
Cat A.txt | while read line; do echo $line; Done

9. Delete blank line

Copy Code code as follows:

Cat A.txt | Sed-e '/^$/d '
(echo "ABC"; echo ""; echo "DDD";) | awk ' {if (0!= NF) print $} '

10. Compare File modification time

Copy Code code as follows:

[[File1.txt-nt File2.txt]] && echo true | | echo False
[[File1.txt-ot File2.txt]] && echo true | | echo False

11. Realize Dictionary Structure

Copy Code code as follows:

Hput () {
Eval "hkey_$1" = "$"
}
Hget () {
Eval echo ' ${' "hkey_$1" '} '
}
$ hput K1 AAA
$ hget K1
Aaa

12. Remove the second column

Copy Code code as follows:

$echo ' a b c d E F ' | Cut-d '-f1,3-
$a C d E F

13. Save the stderr output to the variable

Copy Code code as follows:

$ a=$ ((echo ' out '; Echo ' error ' 1>&2) 2>&1 1>/dev/null)
$ echo $a
Error

14. Delete the first 3 lines

Copy Code code as follows:

$cat A.txt | Sed 1,3d

15. Read multiple fields to variables

Copy Code code as follows:

Read a b c <<< "xxx yyy zzz"

16. Traverse Array

Copy Code code as follows:

Array= (one Two three)
For i in ${array[@]}
Todo
Echo $i
Done

17. View Directory Size

Copy Code code as follows:

$ du–sh ~/apps

18. View CPU Information

Copy Code code as follows:

$ cat/proc/cpuinfo

Date

Copy Code code as follows:

$ date +%y-%m-%d
2012-12-24
$ date +%y-%m-%d–date '-1 day '
2012-12-23
$ date +%y-m-%d–date ' Dec 25 '
2011-12-25
$ date +%y-m-%d–date ' Dec 25–10 days '
2011-12-15

20. Get the path name and filename

Copy Code code as follows:

$ DirName '/home/lalor/a.txt '
/home/lalor
$ BaseName '/home/lalor/a.txt '
A.txt

21. Parallel sets and intersections

Comm can be used to set, intersect, and set the difference, assuming there are now two files A and B, and their contents are as follows:

Copy Code code as follows:

$cat A
1
3
5

$cat b
3
4
5
6
7

$comm a B
1
3
4
5
6
7

$comm-1-2 a B #交集
3
5

$comm a B | Sed ' s/\t//g ' #并集
1
2
3
4
5
6
7

$comm-1-3 A B | Sed ' s/\t//g ' #b-A
4
6
7

awk Complex Separator

Multiple characters as delimiters

Copy Code code as follows:

$ echo "a| | b| | c| | D "| Awk-f ' [|] [|]' ' {print $} '
C

Multiple Separator 1

Copy Code code as follows:

$echo "a| | B, #c d "| Awk-f ' [|, #]+ ' {print $} '
D

Multiple Separator 2

Copy Code code as follows:

$echo "a| | b# #c | #d | Awk-f ' ([|] [|])| ([#][#]) ' {Print $NF} '
c| #d

23. Generate a random number

Copy Code code as follows:

Echo $RANDOM

24. Split file according to pattern

Copy Code code as follows:

Csplit server.log/pattern/-n 2-s {*}-F Server_result-b "%02d.log"-Z

/pattern/is used to match a row, the split process begins
{*} Repeat split according to match
-S silent mode
Ñ the number of digits in the filename suffix after the split
-F split filename prefix
-B Specify suffix format

25. Get file name or extension

Copy Code code as follows:

Var=hack.fun.book.txt
Echo ${var%.*}
Hack.fun.book
Echo ${var%%.*}
Hack
Echo ${var#.*}
Fun.book.txt
Echo ${var##.*}
Txt

26. Execute the previous command with the root account.

Copy Code code as follows:

$sudo!!

Among them: *!! Refers to the previous command *!$ last parameter of previous command *!* all parameters of previous command *!:3 the 3rd parameter of the previous command
For example:

Copy Code code as follows:

$ls/tmp/somedir
Ls:cannot access/tmp/somedir:no such file or directory
$mkdir!$
Madir/tmp/somedir

27. Use Python to build a simple Web server that can be accessed through the http://$HOSTNAME: 8000.

Copy Code code as follows:

Python-m Simplehttpserver

28. You do not need to have permission to save edited files in Vim.

Copy Code code as follows:

: w!sudo Tee%

29. Replace Foo in the previous command with bar and execute.

Copy Code code as follows:

^foo^bar

30. Quickly back up or copy files.

Copy Code code as follows:

CP Filename{,.bak}

31. Copy the SSH keys to User@host to enable password-free SSH logons.

Copy Code code as follows:

$ssh-copy-id User@host

32. Record the Linux desktop as a video.

Copy Code code as follows:

Ffmpeg-f x11grab-s wxga-r 25-i: 0.0-sameq/tmp/out.mpg

Magical Man

Copy Code code as follows:

Mans ASCII
Mans test

34. Edit the previous command in VIM

Copy Code code as follows:

Fc

35. Delete 0-byte files or junk files

Copy Code code as follows:

Find. -type f-size 0-delete
Find. -type f-exec rm-rf {} \;
Find. -type f-name "A.out"-exec rm-rf {} \;
Find. Type F-name "A.out"-delete
Find. Type F-name "*.txt"-print0 | xargs-0 rm-f

36. Display multiple lines of information when writing a shell

Copy Code code as follows:

Cat << EOF
+--------------------------------------------------------------+
| = = = Welcome to tunoff services = = |
+--------------------------------------------------------------+
Eof

Note that when you specify a terminator, it must be the only content of the row, and the line must begin with this character.
37. How to build a soft link to MySQL

Copy Code code as follows:

Cd/usr/local/mysql/bin
For i in *
Do ln/usr/local/mysql/bin/$i/usr/bin/$i
Done

38. Get IP Address:

Copy Code code as follows:

Ifconfig eth0 |grep "inet addr:" |awk ' {print $} ' |cut-c 6-

39. Number of open files

Copy Code code as follows:

Lsof

40. Clear Zombie Process

Copy Code code as follows:

Ps-eal | awk ' {if ($ = = ' Z ') {print $}} ' | Kill-9

41. Print Unique rows

Copy Code code as follows:

awk '!a[$0]++ ' file

42. Print odd rows

Copy Code code as follows:

awk ' i=!i ' file
awk ' nr%2 ' file

43. Print a row after the matching line

Copy Code code as follows:

Seq 10 | awk '/4/{f=4};--f==0{print;exit} '

44. Print the following 10 lines after a line

Copy Code code as follows:

Cat File | grep-a100 string
Cat File | grep-b100 string #前面
Cat File | grep-c100 string #前后

Sed-n '/string/,+100p '

awk '/string/{f=100}--f>=0 '

45. Get the last parameter of the command line

Copy Code code as follows:

Echo ${!#}
Echo ${$#} #错误的尝试

46. Output redirection

If you wish, you can redirect the output of stderr and STDOUT to an output file, Bash provides special redirection symbols &>

Copy Code code as follows:

ls file nofile &>/dev/null

How do we redirect in the script? Nothing special, as with normal redirects.

Copy Code code as follows:

#!/bin/bash
#redirecting output to different locations
echo "Now redirecting all output to another location" &>/dev/null

The question is, what if we want to redirect all the output to a file? We do not want each output time to redirect it, is the so-called, the hermit has its own ingenious plan. We can use Exec to permanently redirect, as follows:

Copy Code code as follows:

#!/bin/bash
#redirecting output to different locations
EXEC 2>testerror
echo "This is the start of the script"
echo "Now redirecting all output to another location"

EXEC 1>testout
echo "This output should go to Testout file"
echo "But this should go" the Testerror file ">& 2

The output results are as follows:

Copy Code code as follows:

This is the start of the script
Now redirecting all output to another location
lalor@lalor:~/temp$ Cat Testout
This output should go to Testout file
lalor@lalor:~/temp$ Cat Testerror
But this should go the Testerror file
lalor@lalor:~/temp$

To redirect in an append way:

Copy Code code as follows:

EXEC 3 >> testout

To cancel redirection:

Copy Code code as follows:

EXEC 3>-

47. function

Anywhere defined variables are global variables, and if you want to define local variables, you need to add the Locals keyword
Functions in the shell can also be recursive

Copy Code code as follows:

#!/bin/bash

function factorial {
if [[$1-eq 1]]; Then
Echo 1
Else
Local temp=$[$1-1]
Local result= ' factorial $temp '
Echo $[$result * $]
Fi
}

result= ' factorial 5 '
Echo $result

Create a function library

To set a function in another file, and then load it to the current file through the source command

Using functions on the command line

Define the function in ~/.BASHRC

Passing an array to a function

Copy Code code as follows:

#!/bin/bash
#adding values in an array

function AddArray {
Local sum=0
Local NewArray
newarray= (' echo ' $@ ')
For value in ${newarray[*]}
Todo
sum=$[$sum + $value]
Done
Echo $sum
}

Myarray= (1 2 3 4 5)
echo "The original array is: ${myarray[*]}"
Arg1= ' echo ${myarray[*]} '
result= ' AddArray $arg 1 '
echo "The result is $result"

48. Regular Expressions

Matching regular expressions for Chinese characters: [U4E00-U9FA5]
Commentary: Matching Chinese is really a headache, with this expression will be easy to do
Match Double-byte characters (including Chinese characters): [^x00-xff]
Commentary: can be used to compute the length of a string (a double-byte character length meter 2,ascii 1 characters)
A regular expression that matches a blank row: ^ *$
Commentary: can be used to delete blank lines
Regular expression:< matching HTML tags (s*?) [^>]*>.*?</1>|<.*? />
Commentary: The online version is too bad, the above can only match the part of the complex nested tags still powerless
A regular expression that matches the end-end whitespace character: ^s*|s*$
Commentary: A useful expression that can be used to delete white-space characters (including spaces, tabs, page breaks, and so on) at the end of a line at the beginning
Regular expression matching an email address: w+ ([-+.] w+) *@w+ ([-.] w+) *.w+ ([-.] w+) *
Commentary: Form validation is useful
Regular expressions that match URL URLs: [a-za-z]+://[^s]*
Commentary: Online circulation of the version of the function is very limited, which can meet the basic requirements
Match account number is legal (beginning of letter, allow 5-16 bytes, allow alphanumeric underline): ^[a-za-z][a-za-z0-9_]{4,15}$
Commentary: Form validation is useful
Match domestic phone number: D{3}-d{8}|d{4}-d{7}
Commentary: Match form such as 0511-4405222 or 021-87888822
Matching Tencent QQ Number: [1-9][0-9]{4,}
Commentary: Tencent QQ number starting from 10000
Match China ZIP Code: [1-9]d{5} (?! D
Commentary: China postal code is 6 digits
Matching ID: d{15}|d{18}
Commentary: China's ID card is 15-or 18-digit
Matching IP address: d+.d+.d+.d+
Commentary: Useful when extracting IP addresses
Match a specific number:
^[1-9]d*$//Matching positive integer
^-[1-9]d*$//matching negative integers
^-? [1-9]d*$//matching integer
^[1-9]d*|0$//matching nonnegative integer (positive integer + 0)
^-[1-9]d*|0$//matching non positive integer (negative integer + 0)
^[1-9]d*.d*|0.d*[1-9]d*$//matching positive floating-point numbers
^-([1-9]d*.d*|0.d*[1-9]d*) $//matching negative floating-point number
^-? ([1-9]d*.d*|0.d*[1-9]d*|0?. 0+|0) $//matching floating-point number
^[1-9]d*.d*|0.d*[1-9]d*|0? 0+|0$//matching nonnegative floating-point number (positive floating-point number + 0)
^ (-([1-9]d*.d*|0.d*[1-9]d*)) |? 0+|0$//matching non-positive floating-point numbers (negative floating-point number + 0)
Commentary: useful when dealing with large amounts of data, pay attention to corrections when applied
Match a specific string:
^[a-za-z]+$//Match a string of 26 English letters
^[a-z]+$//Match a string of 26 uppercase letters
^[a-z]+$//Match string consisting of 26 lowercase letters
^[a-za-z0-9]+$//Match a string of numbers and 26 English letters
^w+$//Match A string of numbers, 26 English letters, or underscores