Python deletes expired log files for instance parsing,

Python deletes expired log files for instance parsing,

This article focuses on how to delete expired log files in python. The details are as follows.

1. Use Python to traverse directories

OS. the walk method can easily obtain all the files in the directory and returns a three-element tupple (dirpath, dirnames, filenames). dirpath indicates the directory path, dirnames is a list that contains the names of all sub-Directories In dirpath. filenames is a list that contains non-directory files. To obtain a full path, you need to use OS. path. join (dirpath, name ). for example, the structure of the test directory is:

Test ------------ file_c
|
----------- Dir_a1/file_a1
|
| ------- Dir_a2/file_a2
|
------------ Dir_b1/file_b1

Use the following code:

import os  for i in os.walk('test'):    print i 

Result:

('Test', ['dir _ a1', 'dir _ b1 '], ['file _ c1']) ('test/dir_a1 ', ['dir _ a2 '], ['file _ a1']) ('test/dir_a1/dir_a2', [], ['file _ a2 ']) ('test/dir_b1 ', [], ['file _ b1'])

To get a file with a path, you can do this:

for i in os.walk('test'):    #print i    for j in i[2]:      os.path.join(i[0],j) 

Result:

'Test/file_c1'
'Test/dir_a1/file_a1'
'Test/dir_a1/dir_a2/file_a2'
'Test/dir_b1/file_b1'

Of course, you can also use OS. path. isdir to perform recursive operations to obtain files in the directory:

def walk(dir):   ret = []   dir = os.path.abspath(dir)   for file in [file for file in os.listdir(dir) if not file in [".",".."]]:     nfile = os.path.join(dir,file)     if os.path.isdir(nfile):       ret.extend( walk(nfile) )     else:       ret.append( nfile )   return ret 

2. Exclude files to be retained

Determine whether to delete a file based on the file name and the file change time. path. getmtime (file) is used to obtain the time when the file is last changed. Of course, in addition to the "XXX" in file Method to Determine the file name, you can also use a regular expression.

def shouldkeep(file):   if '.py' in file:     return True   elif '.conf' in file:     return True   elif 'current' in file:     return True   elif 'rtb' in file and datetime.datetime.fromtimestamp( os.path.getmtime(file) ) > datetime.datetime.now() - datetime.timedelta(3):     return True   # the log webdebug/popterr/webaccess/controller_slow/game/checking_social which are modified 6 day ago should be removed   elif datetime.datetime.fromtimestamp( os.path.getmtime(file) ) < \      datetime.datetime.now() - datetime.timedelta(6)\      and ('webdebug' in file \      or 'potperr' in file\      or 'webaccess' in file\      or 'controller_slow' in file\      or 'game.' in file\      or 'checkin_social' in file\      ):     return False   elif datetime.datetime.fromtimestamp( os.path.getmtime(file) ) < \      datetime.datetime.now() - datetime.timedelta(2)\      and ('queue.master.info' in file):     return False   elif datetime.datetime.fromtimestamp( os.path.getmtime(file) ) > \      datetime.datetime.now() - datetime.timedelta(6):     return True   else:     return False 

files = walk('/var/server/log') for i in files:   if not shouldkeep(i):     print i, datetime.datetime.fromtimestamp( os.path.getmtime(i) )     os.remove( i ) 

Run the script once a day using crontab to regularly clean up expired files under/var/server/log every day.

Summary

The above is all the content parsed by the instance for deleting expired log files in python. I hope it will be helpful to you. If you are interested, you can continue to refer to other related topics on this site. If you have any shortcomings, please leave a message. Thank you for your support!