Fast lookup of log content with binary matching timestamp

Objective

By using dichotomy, you can quickly match timestamps and find the log content you need to find based on timestamps.

Find Prerequisites:

1 because it is a dichotomy, the time of the log must be arranged in order from small to large or from large to small.

2 Modify the matching timestamp according to the lookup needs.

3 Modify the corresponding code according to the different timestamp format.

1 ImportOS2 Import Time3 ImportSYS4 5 classSearchlog (object):6 7     def __init__(Self, file_name='./log.txt'):8SELF.FP =Open (file_name)9Self.fp.seek (0, OS. Seek_end)#move the pointer to the endTenSelf.size = Self.fp.tell ()#the Tell () method tells us the size of the file OneSelf.fp.seek (0, OS. Seek_set)#the pointer moves to the beginning of the contents of the file . A  -     defSEARCH_CMP (self, timestamp, line):#define a method for matching timestamps -TMP = Line.split (' ') theRiqi=Tmp[0] -Shijian=tmp[1] -realtime=riqi+' '+shijian#Here you can modify the timestamp format to match -         #Print Realtime +         returnCMP (timestamp, realtime)#The CMP function is used to compare whether the two are the same, if returned as-1, then the timestamp>realtime, if equal to 0, are equal.  -              +              A     defSearch_linehead (self):#define a method that matches the beginning of each line at  -          whileSelf.fp.tell () >0: -Self.fp.seek ( -1, OS. Seek_cur)#move a pointer to the contents of the file in the current position. -val = self.fp.read (1)#read the contents of the pointer position -             ifval = ='\ n':#matches the value of the pointer content equal to the line break, which is equal to jumping out of the loop -                  Break 　　  inSelf.fp.seek ( -1, OS. Seek_cur)#No, just move on, move forward. Find -              to     defSearch_timestamp (self, TimeStamp, start_p =0): +  -Fp_start =start_p theFp_end =self.size *  $          whileFp_start < fp_end:#determines whether the start position of the pointer is less than the end positionPanax NotoginsengMID = Fp_start + (Fp_end-fp_start)/2#Find intermediate values (using dichotomy) -Self.fp.seek (Mid, Os. Seek_set)#navigate to middle value by seek the Self . Search_linehead () +line =Self.fp.readline () Aval = self. SEARCH_CMP (timestamp, line)#call this method to match the timestamp, the return value is 0 then it means that the timestamp was found the             ifval = =0: +                 Print "Find timestamp:%s"% Line -                 returnTrue $             elifval = = 1: $Fp_start = Self.fp.tell ()#if equal to 1, the timestamp of the current content is greater than the time stamp entered -             Else: -Fp_end = mid#If equal to the other, the timestamp of the current content is smaller than the input timestamp, the         returnFalse -             Wuyi     defSearch_firststamp (self, timestamp, start_p =0): the  -First_point =-1 Wuval =Self . Search_timestamp (TimeStamp, start_p) -  About         ifval = =True: $Point = Self.fp.tell ()#The logic here is this ==> -              whilePoint > 0:#if the Search_timestamp method matches the timestamp, and the pointer is more than 0, the pointer moves forward in the current position until it matches the newline character, and the line is taken out by the Readlie () method.  -Self.fp.seek ( -1, OS. Seek_cur)#at this point, to match whether the beginning of each line is equal to the timestamp, if equal, move the pointer from the beginning of the file content to the line that matches the timestamp, marking the first line of the timestamp in which the behavior is to be looked up.  - Self . Search_linehead () APoint =Self.fp.tell () +line =Self.fp.readline () the                 Print Line -                 ifLine.startswith (timestamp) = =True: $ Self.fp.seek (Point, OS. Seek_set) the                     Continue theFirst_point =Self.fp.tell () the                  Break the  -         returnFirst_point in  the     defSearch_endstamp (self, timestamp, start_p =0): the  AboutLast_point =-1 theval =Self . Search_timestamp (TimeStamp, start_p) the         ifval = = True:#That 's the same thing, but this one is reversed to find the last line of the timestamp to find.  theLast_point =Self.fp.tell () +              whileLast_point <self.size: -line =Self.fp.readline () the                 ifLine.startswith (timestamp) = =True:BayiLast_point =Self.fp.tell () the                     Continue the                 Print Line - Self.fp.seek (last_point, OS. Seek_set) -                  Break the  the             returnLast_point the  the     defSearch_dump (self, s_p, e_p):#write the found content to the log - Self.fp.seek (s_p, OS. Seek_set) theF_log=open ('Search_err.log','WB') the          whileSelf.fp.tell () <e_p: the             PrintSelf.fp.readline ()94 F_log.write (Self.fp.readline ()) the f_log.close () the  the     defSearch_deinit (self):98 self.fp.close () About  - if __name__=='__main__':101time_s =time.time ()102obj = Searchlog (sys.argv[3])103S_point = obj. Search_firststamp (sys.argv[1])104     ifS_point >-1: theE_point = obj. Search_endstamp (sys.argv[2], S_point)106     ifS_point >-1:107         PrintS_point, E_point108 obj. Search_dump (S_point, E_point)109     Else: the         Print "Not find"111     Printtime_s, Time.time () the 113Obj. Search_deinit ()

Fast lookup of log content with binary matching timestamp