Python's call to Linux shell commands and related advanced applications

Recently, according to the requirements of the boss, the data synchronized backup, combined with the tools provided by third parties. Third-party services in fact there is a Python demo, this would like to study the actual Python SDK for a demo development, but found that some of the build is really big, and the company's online version of the python is also really low enough to find another way.

The final solution is as follows: using the relevant management tools provided by the third party (Linux, of course), call the System tool via Python (I am not particularly cold on the shell, except for the text processing), and then redirect the script output to the log file. Easy to check the file upload success or failure of the specific situation processing.

Then it will be designed to three questions: 1, the list of uploaded files (due to the interface with the client key problem, but also processing, not an absolute path); 2, call the third-party provided package, with tools to upload; 3. Analysis and processing of output logs

For the first question, the function functions are as follows:

1 ImportOs,re2pathname = open ('Pathlist.txt','w+')3filename = open ('filelist.txt','w+')4 5 defFind_file (ARG, dirname, files):6      forFileinchFiles:7File_path =os.path.join (dirname, file)8         ifOs.path.isfile (file_path):9Pathname.write ('%s \ n'%File_path)Ten  One defkey (file): ATmppathname = open (file,'R') -      forLineinchTmppathname: -Line_tmpa = Re.split ('AAA', line) #由于key的规则后端已经设置好了, then in the script to do the corresponding processing, through the delimiter to deal with the actual requirements, what you need to give, through the list implementation theLINE_TMPB = str (line_tmpa[1]) -Filename.write ('%s'%LINE_TMPB) -  -Os.path.walk (R"/a/b/c/aaa", Find_file, ()) +  -Key'filelist.txt')

To the second part of the Python function nested shell commands:

　　

1 defBackup (file):2Tmppathname = open (file,'R')3      forKeyinchTmpfilename:4LocalFile ='/A/B/C/AA'+Key #其中key和localfile为第三方工具包需要的参数5os.environ['Key'] =key #通过os. Environ assigns parameter variables that need to be passed6os.environ['LocalFile'] =LocalFile7Os.system ('proxy-client--action put--bucket aaa--key $key--file $localfile')8 9Backup'filelist.txt')

The second part of the implementation is relatively simple, to the third part of the process of the log link because it is the first encounter to deal with the trouble, through the cat Log.txt and less log.txt found the results, respectively:

　　

，

Cat after the discovery progress bar is dynamic processing, but less to view the log file after the discovery of a lot of special symbols, I in this step of the processing took a bit of time (first encounter), have considered using regular to match, but found that the final output is indeed Nonetype, perhaps my kung fu has not home ==!. Always consider this looks like ^h how to deal with, check a half day found this goods unexpectedly is an output does not show the combination of keys ... is still a combination of keys, so in the processing time is not a simple gourd painting, write on ^h, but should be Ctrl + V and then continue to ctrl+h, so that the special character can be processed. So the third part of the code is as follows:

　　

1 #!/usr/bin/env python2 Importre,string3logname = open ('Log_online.txt','R')4Logtest = open ('Log_test.txt','w+')5  forLineinchlogname:6 #print type (line)8 #Print Line_a9Line_b = Line_a.split ('>')Ten #print Type (line_b) OneLogtest.write (line_b[-1]) #这里主要考虑到输出结果中并不一定都是上传成功的案例, so through the analysis of the law found that the delimiter > the last element is the upload result.  ALogtest.close ()

This is the test process was written, but also compare orders, the key is the thinking problem.

Share with us!

Python's call to Linux shell commands and related advanced applications