Python file Practice _ Find the IP in log and count

Requirements: monitoring logs, if there is an attack, the IP blacklist

Analysis:

1. Open log File

2. Take out the IP address

3, determine the number of times each IP appears, if more than 50 times, add blacklist

4. Read once per minute

Log style:

178.210.90.90--[04/jun/2017:03:44:13 +0800]"get/wp-includes/logo_img.php http/1.0"302 161"http://nnzhp.cn/wp-includes/logo_img.php" "mozilla/5.0 (Windows; U Windows NT 5.1; En-US) applewebkit/533.4 (khtml, like Gecko) chrome/5.0.375.99 safari/533.4" "10.3.152.221"178.210.90.90--[04/jun/2017:03:44:13 +0800]"Get/blog http/1.0"301 233"http://nnzhp.cn/wp-includes/logo_img.php" "mozilla/5.0 (Windows; U Windows NT 5.1; En-US) applewebkit/533.4 (khtml, like Gecko) chrome/5.0.375.99 safari/533.4" "10.3.152.221"178.210.90.90--[04/jun/2017:03:44:15 +0800]"get/blog/http/1.0"200 38278"http://nnzhp.cn/wp-includes/logo_img.php" "mozilla/5.0 (Windows; U Windows NT 5.1; En-US) applewebkit/533.4 (khtml, like Gecko) chrome/5.0.375.99 safari/533.4" "10.3.152.221"66.249.75.29--[04/jun/2017:03:45:55 +0800]"get/bbs/forum.php?mod=forumdisplay&fid=574&filter=hot http/1.1"200 17482"-" "mozilla/5.0 (compatible; googlebot/2.1; +http://www.google.com/bot.html)" "-"37.9.169.20--[04/jun/2017:03:47:59 +0800]"get/wp-admin/security.php http/1.1"302 161"HTTP://NNZHP.CN/WP-ADMIN/S

Realize:

ImportTimepoint= 0#position of the file pointer every time it is recorded whileTrue:#continuously read log for real-time updatesALL_IP =[] f=open ('Access.log', encoding='Utf-8')    #cannot use read to directly read the file, the file from the disk open loaded into memory, into the CPU analysis, if the file is too large, the memory will be full, the computer back to the card deadF.seek (Point)#move the file pointer, the counted IP no longer additional statistics     forLineinchF:#loop A file object directly, each time the file is a lineIP = Line.split ('-') [0].strip ()#Remove IPAll_ip.append (IP)#put IP into the listPoint = F.tell ()#the position of the pointer is recordedAll_ip_set = Set (ALL_IP)#The collection is inherently heavy.     forIinchAll_ip_set:#cyclic sets are more efficient than loop lists and have been removed        ifAll_ip.count (i) > 50:            Print('The blacklist IP is%s and%s occurs within one minute'%( i,all_ip.count (i))) F.close () Time.sleep (30L#read once per minute

Python File Practice _ Find IP in log and count