Parse Log download time script

Demand:

Feedback queue Backlog in queue, find bus log process_feedbacks_1.log have time out download error, and a large number of derivative samples downloaded too long
Current requirements: Calculate the download time for each derivative sample in the Process_feedbacks_1.log log
For example:
2014-12-22 10:29:58,073 __main__ DEBUG on_task_success, task_id = 15145129
2014-12-22 10:29:58,097 __main__ DEBUG receive AUTO success event, begin to Redispatch PROC task
2014-12-22 10:29:58,097 __main__ DEBUG begin to process derivative File[filename = 92f69bd61e4ad511a664caf7852018c5. D6336163]
2014-12-22 10:29:58,097 __main__ DEBUG auto_derivative_task_filetype:sample
2014-12-22 10:29:58,097 __main__ DEBUG in __download_derivative_file__ (). Output_file:/mnt/auto_derivative/92f69bd61e4ad511a664caf7852018c5. D6336163
2014-12-22 10:30:02,388 __main__ DEBUG Download OK, create file OK:%s/mnt/auto_derivative/ 92f69bd61e4ad511a664caf7852018c5. D6336163
2014-12-22 10:30:02,414 __main__ DEBUG begin to process derivative File[filename = 83613b2fe6b780b3bbb08470171dbfd6. B16EDA9B]
2014-12-22 10:30:02,415 __main__ DEBUG auto_derivative_task_filetype:sample
2014-12-22 10:30:02,415 __main__ DEBUG in __download_derivative_file__ (). Output_file:/mnt/auto_derivative/83613b2fe6b780b3bbb08470171dbfd6. b16eda9b
2014-12-22 10:30:03,806 __main__ DEBUG Download OK, create file OK:%s/mnt/auto_derivative/ 83613b2fe6b780b3bbb08470171dbfd6. b16eda9b

Calculate Time Difference
2014-12-22 10:29:58,097 __main__ DEBUG in __download_derivative_file__ (). Output_file:/mnt/auto_derivative/92f69bd61e4ad511a664caf7852018c5. D6336163
2014-12-22 10:30:02,388 __main__ DEBUG Download OK, create file OK:%s/mnt/auto_derivative/ 92f69bd61e4ad511a664caf7852018c5. D6336163

2014-12-22 10:30:02,415 __main__ DEBUG in __download_derivative_file__ (). Output_file:/mnt/auto_derivative/83613b2fe6b780b3bbb08470171dbfd6. b16eda9b
2014-12-22 10:30:03,806 __main__ DEBUG Download OK, create file OK:%s/mnt/auto_derivative/ 83613b2fe6b780b3bbb08470171dbfd6. b16eda9b

Recorded as Time.txt file, txt list content is
92f69bd61e4ad511a664caf7852018c5. D6336163, time interval
83613b2fe6b780b3bbb08470171dbfd6. b16eda9b, time interval

Simultaneous calculation of time.txt, the mean time difference

Script:

#!/usr/bin/env python

Import datetime
Import Os,sys
Import time

filename = sys.argv[1]
f = open (' Spend_time.txt ', ' W ')
F.write ("md5,spend_time (s) \ n")
def Time_clac (date1):
Date1_perfix,date1_suffix = Date1.split (', ')
Sec=time.mktime (Time.strptime (Date1_perfix, '%y-%m-%d%h:%m:%s '))
Sec_1 = sec*1000
Mis_sec = sec_1 + int (date1_suffix)
Return mis_sec

For line in open (filename, ' R '):
line = Line.strip (' \ n ')
If ' in __download_derivative_file__ () ' on line:
Start_time = Line.split (' __main__ ') [0]
Start_time_1 = Time_clac (start_time)
MD5 = line[-32:]
Continue
If ' download ok ' in line and MD5. Line:
End_time = Line.split (' __main__ ') [0]
End_time_1 = Time_clac (end_time)
Spend_time = (end_time_1-start_time_1)/1000
Print "%s,%s"% (md5,spend_time)
F.write ("%s,%s\n"% (md5,spend_time))

F.close ()

Parse Log download time script