Python log parsing and regular expressions

Programmers are often confronted with log parsing work. The regular expression is an essential tool for processing logs.

"Line 622:01-01 09:04:16.727 <6> [pid:14399, cpu1 dabc_pwym_task] [HISTBF] update_freeze_data:dabc9:bl_level= 1740627:01-01 09:04:17.727 <6> [pid:14399, cpu1 dabc_pwym_task] [HISTBF] Update_freeze_data:dabc:bl_ level=1720 "

For example, for the above log, you need to find the log time, and to find the corresponding format of data. The main problems with bread here include:

1. Matching work. Need to find the real log, the second line above is not the real log;
2. Split work (split). Divide the log according to the space, find the log time;
3. Filtering work. Find the matching format to filter out the numbers 1740 and 1720.

For matching work, you need to find the line that starts with ' lines '. Use the search () function of Re.

ImportRestrrs=list () Strrs.append ("Line 622:01-01 09:04:16.727 <6> [pid:14399, cpu1 dabc_pwym_task] [HISTBF] Update_freeze_data:dabc9:b l_level=1740") Strrs.append ("app_log_cat new log begin") Strrs.append ("Line 627:01-01 09:04:17.727 <6> [pid:14399, cpu1 dabc_pwym_task] [HISTBF] Update_freeze_data:dabc:b l_level=1720") Regex= R' Line' forStrrinchStrrs:str_search=Re.match (Regex, STRR)ifStr_search:Print(True)Else:        Print(False)

The matching results are as follows

Truefalsetrue

For split work, you need to find the log time. Observing the above log, the space is used as the basis for segmentation.

Import'line622:01-01 09:04:16.727 <6> [pid:14399, cpu1 dabc_pwym_task] [HISTBF] "'   update_freeze_data:dabc9:bl_level=1740 '           ' \s '  = re.split (regex, STRR)print(str_split)

The segmented output is a list in which the time corresponding data is selected.

[' Line','622:','01-01','09:04:16.727','<6>','[pid:14399,','cpu1','Dabc_pwym_task]','[HISTBF]','Update_freeze_data:','dabc9:bl_level=1740']

For filtering work, you need to find the final data.

Import"" "line622:01-01 09:04:16.727 <6> [pid:14399, cpu1 dabc_pwym_task] [HISTBF]          update_freeze_data:dabc9:bl_level=1740          Line 627:01-01 09:04:17.727 <6> [pid:14399, cpu1 dabc_pwym_task] [HISTBF]           update_freeze_data:dabc:bl_level= 1720"" = R'dabc\d?:bl_level= (\d+)'=  Re.findall (Regex, STRR)print(str_select)

The filtered result is a list

['1740'1720']

Python log parsing and regular expressions