Automated O & M: website svn Code Update (flask + saltstack) and svnflask

Automated O & M: website svn Code Update (flask + saltstack) and svnflask
Periodic summary: It took about a week to complete the automatic website upgrade function. Many problems were encountered and many problems were learned. Here is a summary. 1. Overall architecture: backend: nginx + uwsgi # nginx provides web Services. uwsgi parses python code. Foreground: flask + bootstrap + html # flask is a set of web development tools for python, bootstrap is a set of ready-made templates that do not need to be explained in html. Function: shell + saltstack + svn # core of the saltstack function, an automated O & M tool. Shell does not explain. svn code consistency tool. 2. Skills used: python-related: how to execute the shell command for calling the dictionary of the read/write list of files using flask: other forms of session flash seem to be unexpected. Remember to comment out the code together. 3. review and comment the code below######################################## ######################################## ################################################ ######################################## ######### Call flask-related modules from flask import Flask, render_template, session, redirect, url_for, flashfrom flask. ext. script import Managerfrom flask. ext. bootstrap import Bootstrapfrom flask. ext. moment import Momentfrom flask. ext. wtf import Form # The type and attributes of the Form to be called from wtforms import TextAreaField, StringField, SubmitField, SelectField, PasswordFieldfrom wtforms. validators import Required, DataRequired # Call the Basic module import OS, com Mands, subprocess, time, threading # Call the dictionary file and load from webpath import webpathfrom userdata import userdata app = Flask (_ name _) app. config ['secret _ key'] = 'who is your boss' manager = Manager (app) bootstrap = Bootstrap (app) moment = Moment (app) # function: obtain the current time in the specified format def gettime (): ISOTIMEFORMAT = '% Y-% m-% d % x' # specify the time format ttime = time. strftime (ISOTIMEFORMAT, time. localtime (time. time () # Get the current time return ttime # function: Convert the output to List d Ef make_list (listin, symbol): tem = listin. replace (symbol, '') # replace the specified content with null tem1 = tem. splitlines () # convert the input to the list by row to return tem1 # function: Output log def inputF (con): f = open ('seos. log', 'A') # open the file in append mode. If no file exists, create updatetime = gettime () uptime = str (updatetime) + '\ n' f. write (uptime) conx = con + '\ n' f. write (conx) f. close () # close the file and release the resource # function: get the name of the currently alive web server and convert it to the List def getweb (): # Run the shell command, obtain the surviving web Server p = subproces in saltstack S. popen ('Salt-run manage. up | grep web ', shell = True, stdout = subprocess. PIPE) tem0 = p. stdout. read () # Get the script execution result tem2 = make_list (tem0, '-') return tem2 # function: Delete the cache def rmcache_Newseos (webname) of the specified web-related module ): wbapp = 'wap weio 'tem2 = getweb () basicpath = webpath. get (webname) # execute the script and call the script on the client server of saltstack to delete the cache function res = subprocess. popen ('Salt \ '% s \' cmd. run \ 'sh % scleancache. sh % s \ ''% (webname, basicpath, wba Pp), shell = True, stdout = subprocess. PIPE) # function: Delete the cache of all web-related modules # This function is a function used before the thread. The function serves as above and is less efficient def rmomscache (): p = subprocess. popen ('Salt-run manage. up | grep web ', shell = True, stdout = subprocess. PIPE) path = 'oss/Runtime/* 'tem0 = p. stdout. read () tem2 = make_list (tem0, '-') for webname in tem2: basicpath = webpath. get (webname) tpath = '% s % s' % (basicpath, path) res = subprocess. popen ('Salt \ '% s \' cmd. run \ 'rm -Rf % s \ ''% (webname, tpath), shell = True, stdout = subprocess. PIPE) res_in = res. stdout. read () print res_in print tpath return 'delete oms cache! '# Function: delete def rmomscache_new (webname): path = 'oss/Runtime/*' basicpath = webpath. get (webname) tpath = '% s % s' % (basicpath, path) res = subprocess. popen ('Salt \ '% s \' cmd. run \ 'rm-rf % s \ ''% (webname, tpath), shell = True, stdout = subprocess. PIPE) # function: Create a thread set def getthreads (targets): allweb = getweb () threads = [] for webname in allweb: t = threading. thread (target = targets, args = (webname,) threa Ds. append (t) return threads # function: Upgrade website def update_web (path): f = open('resault.txt ', 'A') tem2 = getweb () for webname in tem2: basicpath = webpath. get (webname) tpath = '% s % s' % (basicpath, path) res = subprocess. popen ('Salt \ '% s \' cmd. run \ 'svn update % s \ ''% (webname, tpath), shell = True, stdout = subprocess. PIPE) res_in = res. stdout. read () f. write (str (res_in) res_list = make_list (res_in, '') f. close () r Esault = open('resault.txt '). read () f. close () OS .remove('resault.txt ') return resault # define Form class NameForm (Form): uppath = SelectField ('path', choices = [('Do', 'Do '), ('oms', 'oms')]) username = SelectField ('user', choices = [('admin', 'admin'), ('liujian ', 'liujian ')]) upkey = PasswordField ('Password', validators = [DataRequired ()]) submit = SubmitField ('update') @ app. errorhandler (404) def page_not_found (e ): Return render_template('404.html '), 404 @ app. errorhandler (500) def internal_server_error (e): return render_template('500.html '), 500 # main program @ app. route ('/', methods = ['get', 'post']) def index (): name = None uppath = None username = None upkey = None passwd = None rmcache = 'Does it need to delete the cache' form = NameForm () if form. validate_on_submit (): uppath = form. uppath. data upkey = form. upkey. data Username = form. username. data passwd = userdata. get (username) if upkey is None or upkey! = Passwd: flash ('your key is wrong! ') Session ['name'] = 'Please input right key. 'session ['rdcac'] = 'Does it need to delete the cache' return redirect (url_for ('index') else: utime = gettime () print '% s Start update web' % utime resaul = update_web (uppath) resault = resaul. replace ('\ n',' <br> ') session ['name'] = resault if uppath. startswith ('newseos '): Newseostime = gettime () print' % s Newseos is updated, start remove cache. '% New Seostime threads = getthreads (rmcache_Newseos) for t in threads: t. setDaemon (True) t. start () t. join () endtime = gettime () print '% s Newseos upgrade over! '% Endtime session ['rdcac'] = 'delete Newseos Cache. 'uplog = 'operator: '+ username +' \ n' + 'Update Path: '+ uppath +' \ n' + resaul inputF (uplog) return redirect (url_for ('index') elif uppath. startswith ('oms'): omstime = gettime () print '% s oms id updated, start remove cache. '% omstime threads = getthreads (rmomscache_new) for t in threads: t. setDaemon (True) t. start () t. join () endtime = gettime () Print '% s oms upgrade over! '% Endtime session ['rdcac'] = 'delete oms Cache. 'uplog = 'operator: '+ username +' \ n' + 'Update Path: '+ uppath +' \ n' + resaul inputF (uplog) return redirect (url_for ('index') else: session ['rmcache'] = 'No need delete cache. 'uplog = 'operator: '+ username +' \ n' + 'Update Path: '+ uppath +' \ n' + resaul inputF (uplog) return redirect (url_for ('index') return render_template('index.html ', form = Form, name = session. get ('name'), uppath = uppath, upkey = upkey, username = username, rmcache = session. get ('rmcache') if _ name _ = '_ main _': manager. run () Conclusion: The template-related code and the overall program structure are not displayed, and the necessity is not very great. Below are some of my reference and learning materials and links: Books: Flask Web development-Python-based Web application development practices-core books Python core programming version 2 links: the detail page of flask + uwsgi + nginx deployment in linux is displayed:

Page implementation functions:

Select Update PATH, update password verification, automatically delete cache, update result feedback, and update log records