Svn automatic update

First, I want to talk a lot here,

Requirements:

1. The core requirement for writing any script is that there must be a stepping stone, which can be verified unilaterally.

2. A server is required to mount all front-end web sites

3. An alarm notification interface is required

My personal understanding of automatic launch:

1. Execute the launch script, and you can leave the script in the background.

2. How to achieve peace of mind, svn automated online implementation is not simple

I. Backup Block

The landlord is still hesitant to back up the data. He has two ideas.

(1) You do not need to back up data. You can use the-r + version number to roll back to the previous version, but there are drawbacks. For example, if a wrong package occurs during testing, this cannot be done by others alone.

(2) Each full-path web Backup will be rolled back quickly, but it is unnecessary. For example, if I went online this time and only used the jsp file, do I have to back up all the files, this will definitely affect efficiency

The landlord still chooses the second solution.

Ii. Online Block

1) svn has its own drawbacks when going online. For example, if you manually update online files, the next time you update this package, there will be a G problem, indicating that the file has not been updated successfully.

(2) If svn goes online without this file in the previous version library and you manually upload a file, an error will be reported when svn is updated.

(3) there are as many as 23 front-end web servers and six clusters. The paths and process names of each cluster are different. How to Determine the group to be updated each time?

(4) How can we streamline the number of rows of scripts in such a large web Cluster?

Iii. Detection Block

(1) The essence of automation is detection.

(2) check whether the 500 error is normal when all web pages in the front-end cluster are online.

(3) Check the nginx log at the front-end. Is the Error 500 normal?

(4) check whether all files have been updated successfully when svn goes online. Check the md5 code for the online files only.

Paste the code below

Part 1 launch

The landlord has a public function script, which is the core script.

Vim function. sh

#! /Bin/bash
One = $1
Two = $2
Three =/svnlog/$3
Date = 'date + % Y % m % d % H % m'
Function youyuanjar (){
Echo "---------------------------- resin restart --------------------------------" 2> & 1 | tee-a/tmp/shangxianlog/svn. $ one
Echo "----------------------------------- $ one ------------------------------------" 2> & 1 | tee-a/tmp/shangxianlog/svn. $ one
Ssh $ two "sh/usr/local/resin/bin/resin. sh stop-server youyuan" 2> & 1 | tee-a/tmp/shangxianlog/svn. $ one
Echo "$ one stopped" 2> & 1 | tee-a/tmp/shangxianlog/svn. $ one
Sleep 3
Ssh $ two "cp-rf/www/youyuan.com. 1/htdocs/opt/$ date"
E = 'cat $ three | grep-v Updated | awk-F "/" '{print $ (NF-1) "/" $ NF }''
For I in $ e
Do
Ssh $ two "rm-f/www/youyuan.com. 1/$ e"
If ["$? "-Eq 0]; then
Echo delet OK
Fi
Done
Ssh $ two "svn up $ svnversion/www/youyuan.com. 1/htdocs" 2> & 1 | tee-a/tmp/shangxianlog/svn. $ one
Echo "$ one svnup OK" 2> & 1 | tee-a/tmp/shangxianlog/svn. $ one
Sleep 15
B = ($ (cat $ three | grep-v Updated | awk-F "/" '{print $ NF}' | awk-F ". "'{print $1 }'))
C = ($ (cat $ three | grep-v Updated | awk-F "/" '{print $ (NF-1) "/" $ NF }'))
Echo $ c
For (d = 0; d <$ {# c [@]}; d ++ ))
Do
Touch/home/shangxianlog/$ {B [$ d]}
A = 'ssh $ two "md5sum/www/youyuan.com. 1/$ {c [$ d]}"'
Echo $ two $ a >>/ home/shangxianlog/$ {B [$ d]}
Done
Ssh $ two "sh/usr/local/resin/bin/resin. sh restart-server youyuan" 2> & 1 | tee-a/tmp/shangxianlog/svn. $ one
Sleep 20
Echo "$ one restart" 2> & 1 | tee-a/tmp/shangxianlog/svn. $ one
}
Function iosjar (){
Touch/tmp/shangxianlog/svni. $ one
Echo "---------------------------- resin restart --------------------------------" 2> & 1 | tee-a/tmp/shangxianlog/svni. $ one
Echo "----------------------------------- $ one ------------------------------------" 2> & 1 | tee-a/tmp/shangxianlog/svni. $ one
Ssh $ two "sh/usr/local/resin/bin/resin. sh stop-server ios" 2> & 1 | tee-a/tmp/shangxianlog/svni. $ one
Echo "$ one stopped" 2> & 1 | tee-a/tmp/shangxianlog/svni. $ one
Sleep 3
Ssh $ two "cp-rf/www/ios.youyuan.com/htdocs/opt/ios $ three"
C = 'cat $ three | egrep '(ios | iosui | css | lib)' | grep-v Updated | awk-F "/" '{print $ (NF-1) "/" $ NF }''
Echo $ c
For d in $ c
Do
Ssh $ two "rm-f/www/ios.youyuan.com/mongod"
If ["$? "-Eq 0]; then
Echo delet OK
Fi
Done
Ssh $ two "svn up $ svnversion/www/ios.youyuan.com/htdocs/WEB-INF/lib" 2> & 1 | tee-a/tmp/shangxianlog/svni. $ one
Ssh $ two "svn up $ svnversion/www/ios.youyuan.com/htdocs/WEB-INF/ios" 2> & 1 | tee-a/tmp/shangxianlog/svni. $ one
Ssh $ two "svn up $ svnversion/www/ios.youyuan.com/htdocs/WEB-INF/iosui" 2> & 1 | tee-a/tmp/shangxianlog/svni. $ one
Ssh $ two "svn up $ svnversion/www/ios.youyuan.com/htdocs/css/" 2> & 1 | tee-a/tmp/shangxianlog/svni. $ one
Echo "$ one svnup OK" 2> & 1 | tee-a/tmp/shangxianlog/svni. $ one
Ssh $ two "sh/usr/local/resin/bin/resin. sh restart-server ios" 2> & 1 | tee-a/tmp/shangxianlog/svni. $ one
Sleep 20
Echo "$ one restart" 2> & 1 | tee-a/tmp/shangxianlog/svni. $ one
}
Function ios (){
Touch/tmp/shangxianlog/svni. $ one
Echo "----------------------------------- $ oneios ------------------------------------" 2> & 1 | tee-a/tmp/shangxianlog/svni. $ one
Ssh $ two "cp-rf/www/ios.youyuan.com/htdocs/opt/ios $ three"
C = 'cat $ three | egrep '(ios | iosui | css)' | grep-v Updated | awk-F "/" '{print $ (NF-1) "/" $ NF }''
Echo $ c
For d in $ c
Do
Ssh $ two "rm-f/www/ios.youyuan.com/mongod"
If ["$? "-Eq 0]; then
Echo delet OK
Fi
Done
Ssh $ two "svn up $ svnversion/www/ios.youyuan.com/htdocs/WEB-INF/ios" 2> & 1 | tee-a/tmp/shangxianlog/svni. $ one
Ssh $ two "svn up $ svnversion/www/ios.youyuan.com/htdocs/WEB-INF/iosui" 2> & 1 | tee-a/tmp/shangxianlog/svni. $ one
Ssh $ two "svn up $ svnversion/www/ios.youyuan.com/htdocs/css/" 2> & 1 | tee-a/tmp/shangxianlog/svni. $ one
Sleep 3
Echo "53ios svnupok" 2> & 1 | tee-a/tmp/shangxianlog/svni. $ one
}
Function youyuan (){
Echo "----------------------------------- $ one ------------------------------------" 2> & 1 | tee-a/tmp/shangxianlog/svn. $ one
#1. Backup
Ssh $ two "cp-rf/www/youyuan.com. 1/htdocs/opt /"
#2. Before going online, delete all the files to be online this time to completely avoid some problems in the svn going online.
E = 'cat $ three | grep-v Updated | awk-F "/" '{print $ (NF-1) "/" $ NF }''
For I in $ e
Do
Ssh $ two "rm-f/www/youyuan.com. 1/$ e"
If ["$? "-Eq 0]; then
Echo delet OK
Fi
Done
#3. version updates
Ssh $ two "svn up $ svnversion/www/youyuan.com. 1/htdocs" 2> & 1 | tee-a/tmp/shangxianlog/svn. $ one
Echo "$ one svnup OK" 2> & 1 | tee-a/tmp/shangxianlog/svn. $ one
Sleep 15
#4. If no file is updated, the md5 code will be compared with the correct file version number of the stepping stone.
A = ($ (cat/svnlog/$ filename | grep-v Updated | awk '{print $2}' | awk-F "/" '{print $3 minutes /" $4 }'))
H = ($ (cat/svnlog/$ filename | grep-v Updated | awk-F "/" '{print $ NF}' | awk-F ". "'{print $1 }'))
For (z = 0; z <$ {a [@]}; z ++ ))
Do
B = 'ssh $ two "md5sum/www/youyuan.com. 1/$ {a [$ z]}" | awk '{print $ NF }''
Grep-w $ B/home/shangxianlog/$ {h [$ z]}
If ["$? "-Eq 0]; then
Echo good
Else
Sh/www/shell/syslog_2.sh $ two-/www/youyuan.com. 1/$ {a [$ z]}-uploaderror 'hostname' 1 2 1 6
Ssh $ two "rm-f/www/youyuan.com. 1/$ {a [$ z]}"
Ssh $ two "svn up/www/youyuan.com. 1/htdocs"
Fi
Done
# O = 'cat/tmp/shangxianlog/svn. $ one | grep "G" | wc-l'
# If ["$ o"-gt 0]; then
# B = 'cat/tmp/shangxianlog/svn. $ one | awk '/G/{print $ NF }''
# C = 'cat/svnlog/$ a | egrep-o "([0-9] {3 }. [0-9] {1, 3 }. [0-9] {1, 3 }. [0-9] {1, 3} | G) "| grep-B1 G | sed-n '1'
# C = 'cat/svnlog/$ a | egrep-o "(192. [0-9] {1, 3 }. [0-9] {1, 3 }. [0-9] {1, 3} | G) "| grep-B1 G | grep" 192 "'
# E = 'date + % H % M % s'
# For I in $ B
# Do
# Ssh $ two "cp-f $ I/tmp/$ e"
# If ["$? "= 0]; then
# Echo OK
# Ssh $ two "rm-f $ I"
# If ["$? "= 0]; then
# Echo OK
# Ssh $ two "svn up/www/youyuan.com. 1/htdocs" | tee-a/tmp/$ e
# Fi
# Fi
# Done
# Fi
Sleep 3
Echo "$ one svnupok" 2> & 1 | tee-a/tmp/shangxianlog/svn. $ one
}

How to call a function

There are four function call scripts.

Iosjar. sh

#! /Bin/bash
Source/opt/shangxian/function. sh
Iosjar

Ios. sh

#! /Bin/bash
Source/opt/shangxian/function. sh
Ios

Youyuanjar. sh

#! /Bin/bash
Source/opt/shangxian/function. sh
Youyuanjar

Youyuan. sh

#! /Bin/bash
Source/opt/shangxian/function. sh
Youyuan

Svn launch script

Svn. sh

#! /Bin/bash
Source/opt/shangxian/function. sh
Youyuanjar
[Root @ 02 shangxian] # cat youyuan. sh
#! /Bin/bash
Source/opt/shangxian/function. sh
Youyuan
[Root @ 02 shangxian] # cat svn. sh
#! /Bin/bash
### Create a log ###
Filename = 'date' + % Y % m % d _ % H % M % S. svnup. log''
Svnversion =''
Touch/svnlog/$ filename
# Ip = 'cat/tmp/ip.txt | awk-F "." '{print $ NF }''
# For I in $ ip
# Do
# Touch/tmp/shangxianlog/svn. $ I
# Done

A = ($ (cat/opt/shangxian/shangxiantest.txt | awk '{print $2 }'))
B = ($ (awk '$2 ~ /Youyuan/{print $1} '/opt/shangxian/shangxiantest.txt ))
D = ($ (awk '$2 ~ /Ios/{print $1} '/opt/shangxian/shangxiantest.txt ))
C = ($ (awk '$2 ~ /Youyuan/{print $1} '/opt/shangxian/shangxiantest.txt | awk-F "."' {print $ NF }'))
E = ($ (awk '$2 ~ /Ios/{print $1} '/opt/shangxian/shangxiantest.txt | awk-F "."' {print $ NF }'))

# Write the files to be launched into the log on the stepping stone so that they can be determined later.
/Usr/bin/svn up $ svnversion/svnsignal/htdocs 2> & 1 | tee-a/svnlog/$ filename

# Record the md5 code of the correct file on the current launch so that all the front-end servers can compare whether the update is successful.
F = ($ (cat/svnlog/$ filename | grep-v Updated | awk '{print $2}' | awk-F "/" '{print $3 minutes /" $4 }'))
G = ($ (cat/svnlog/$ filename | grep-v Updated | awk-F "/" '{print $ NF}' | awk-F ". "'{print $1 }'))
For (z = 0; z <$ {# a [@]}; z ++ ))
Do
Svn up/www/youyuan.com. 1/$ {a [$ z]}
Touch/home/shangxianlog/$ {h [$ z]}
Md5sum/www/youyuan.com. 1/$ {a [$ z]} >>/ home/shangxianlog/$ {h [$ z]}
Done

### Determine whether to restart ###
Restartsignal = 'cat/svnlog/$ filename | awk '/. jar $ |. conf $ | classes | lib/' | wc-l'
If [$ restartsignal-gt 0]
Then
# Use the second column of the/opt/shangxian/shangxiantest.txt file to determine which functions are used and pass the parameters to the function, because different sites are placed in a svn update script.
For (I = 0; I <$ {# I [@]}; I ++ ))
Do
If ["$ {a [I]}" = youyuan]; then
Sh/opt/shangxian/youyuanjar. sh $ {c [I] }$ {B [I]} $ filename
Else
Sh/opt/shangxian/iosjar. sh $ {e [$ I] }$ {d [$]} $ filename
Fi
Sleep 5
Done
Else
For (I = 0; I <$ {# I [@]}; I ++ ))
Do
If ["$ {a [I]}" = youyuan]; then
Sh/opt/shangxian/youyuan. sh $ {c [I] }$ {B [I]} $ filename
Else
Sh/opt/shangxian/ios. sh $ {e [$ I] }$ {d [$]} $ filename
Fi
Sleep 5
Done
Fi
# H = 'cat/svnlog/$ filename | grep-v Updated | awk-F "/" '{print $ NF}' | awk-F ". "'{print $1 }''
# K = 'awk '! A [$2 $3] ++ '/home/shangxianlog/$ h | wc-l'
# If [$ k-gt 1]; then
# I = 'awk '! A [$2 $3] + + '/home/shangxianlog/$ g | awk 'end {print}' | awk '{print $1 }''
# J = 'awk '! A [$2 $3] + + '/home/shangxianlog/$ g | awk 'end {print}' | awk '{print $ NF }''
# Y = 'awk '! A [$2 $3] + + '/home/shangxianlog/$ g | awk 'end {print}' | awk-F "/" '{print $1 usd/" $2 #/" $3 #/" $4 "/"}''
# Ssh $ I "rm-f $ j & svn up $ y"
# If ["$? "-Eq 0]; then
# Echo OK
# Fi
# Fi

Ii. Detection

Nginx500 error detected

#! /Bin/bash
A = ($ (tail-n20000/usr/local/nginx/logs/access. log | grep "\" 500 "| awk '{+ + a [$ (NF-2)]} END {for (I in a) print I, a [I]} '| sort-k2-rn | head-n10 | awk' {print $1 }'))
B = ($ (tail-n20000/usr/local/nginx/logs/access. log | grep "\" 500 "| awk '{+ + a [$ (NF-2)]} END {for (I in a) print I, a [I]} '| sort-k2-rn | head-n10 | awk' {print $2 }'))
For (I = 0; I <$ {# a [@]}; I ++ ))
Do
If ["$ {B [I]}"-gt 20]; then
# Ssh $ {B [I]} "/usr/local/resin/bin/resin. sh restart-server youyuan"
Sh/www/shell/syslog_2.sh nginx500error-must-$ {a [I]}-$ {B [I]} 1 2 1 6
Fi
Done

Resin500 error detected

#! /Bin/bash
Time = 'date + % Y-% m-% d "" % H'

For station in 23 41 42 43 44 51 53 123 132 130 131 128 211 126 127 138 139 56
Do

For line in 'Find/logs/youyuan.com .? /$ Station-name root-error.log'
Do
Echo $ station
Count = 'cat $ line | grep "$ time" | egrep '(500error | 500 error)' | wc-l'
Echo $ count
A = 'awk' END {print} '/home/$ station/500error.txt'
Echo $
Countjia = 'echo "scale = 0; $ count-$ a" | bc'
If ["$ countjia"-gt 20]; then
Sh/www/shell/syslog_2.sh 192.168.0. $ line-resin-500erroradd-$ countjia 'hostname' 1 2 1 6
Fi
Cat $ line | grep "$ time" | egrep '(500error | 500 error)' | wc-l>/home/$ station/500error.txt
Done
Done

This article is from the "batch CT Batch Data Synchronization" blog, please be sure to keep this source http://4249964.blog.51cto.com/4239964/1279530