Implement log Parser with 70 lines of code

Python is another masterpiece, feeling the power of Python. Implement log Parser with 70 lines of code

Function Description: Can be directly to the text date to the grouping and sorting function, finished the output paste into Excel can directly generate the chart, for troubleshooting some production environment problems have a great effect.

Code:

#encoding =utf-8from optparse Import Optionparserimport re def Get_args (): Def get_parser (): Usage = u "" "%prog- F filename-r Rule [-d] [-c] Purpose: Grouping and sorting text files according to specified patterns, mainly analyzing text logs Note: If there are groups in the regular expression, then the first group is extracted, if you do not want this, use the regular no capturing grouping (?:) Example: Log volume per minute in the statistics log, default by time sequence python group.py-f log.txt-r "\d\d\d\d\-\d\d\-\d\d \d\d:\d\d" Statistics log the number of occurrences of each IP, and by the number of occurrences in reverse order Arrange python group.py-f input.txt-r "\d+\.\d+.\d+.\d+"-c-d "" "Return Optionparser (usage) def add_option ( Parser): Parser.add_option ("-F", "--file", dest= "filename", help=u "text file required for grouping", metavar= "file") par Ser.add_option ("-R", "--rule", dest= "rule", help=u "grouped regular expression", metavar= "REGEX") parser.add_option ("-D", AC Tion= "Store_true", dest= "reverse", Default=false, help=u "reverse order") parser.add_option ("-C", action= "store_tr        UE ", dest=" Orderbycount ", Default=false, Help=u" Sort by quantity, default by matching string ") def get_options (parser): Options, args = Parser.parse_arGS () If not options.filename:parser.error (' no filename specified ') If not Options.rule:parser.erro R (' No group rule specified ') return options parser = Get_parser () add_option (parser) return get_options (parser) option s = Get_args () filename = Options.filenamerule = Options.rulereverse = Options.reverseorderbycount = Options.orderbycountregex = re.compile (rule, re. IGNORECASE) keys = {} def counter_key (key): Keys.setdefault (key, 0) Keys[key] + = 1 def print_keys (): Sor T_key = (Lambda d:d[1]) if orderbycount else (Lambda d:d[0]) Temp_items = sorted (Keys.items (), Key=sort_key, Reverse=re Verse) for item in Temp_items:key = item[0] Print key, Keys[key] def get_key (line): M = Regex.search (line) if M:return m.group () if regex.groups = = 0 Else m.group (1) return '!  notmatch! ' with open (filename) as F:for line in F:key = Get_key (line) Counter_key (key) Print_keys ()

