CC detection blocks bots and proxies!

The experiences of these days are merged into code.

Python, edited by me.

#! /Usr/bin/ENV Python #-*-coding: UTF-8-*-import OS, sys, timeimport commands, logging # The following is the action for manual commands # Time Format 17/OCT/2014: 10: 00: 00 # Time Format sat Oct 18 12:35:43 2014 # awk '$4> "[17/OCT/2014: 14: 00: 00 "& $4 <" [17/OCT/2014: 14: 02: 00 "'xxx. log | awk '{A [$7] ++} end {for (I in a) print a [I], i} '| sort-N | tail-N 10 | column-T | 500 # awk' $4> "[17/OCT/2014: 14: 00: 00 "& $4 <" [17/OCT/2014: 14: 02: 00 "'xxx. log | grep 'uc _ server/IM Ages/noavatar_small.gif '| awk' {A [$1] ++} end {for (I in a) print a [I], i} '| sort-N | tail-N 10 | column-T | 100 # defines the number of CC endpoints and the number of times the IP addresses access the CC URL when an alert is triggered. urlalert = 500 ipalert = 50 print 'urlaler is: ', urlalert,', ipalert is: ', ipalert # defines the log file address logpath = "XXX. log "# Processing interval of 120 seconds endtime = time. time () strendtime = time. ctime (endtime ). split () strbegintime = time. (endtime-120 ). split () # regular format, so that the python time format corresponds to the nginx log time format endtime = "% S/ % S/% s: % s "% (strendtime [2], strendtime [1], strendtime [4], strendtime [3]) begintime = "% S/% s: % s" % (strbegintime [2], strbegintime [1], strbegintime [4], strbegintime [3]) # Logging. basicconfig (Level = logging. info, format = '% (asctime) S % (filename) s [Line: % (lineno) D] % (levelname) S % (Message) s ', datefmt = '% A, % d % B % Y % H: % m: % s', filename = 'xxx. log', filemode = 'A') # General Command Processing Function def exec_cmd (CMD): (status, output) = Commands. getstatusoutput (CMD) Try: If int (Status) = 0: Print cmd + "is OK! "Logging.info (CMD +" --- is OK! ") Return output else: logging.info (CMD +" Cant't finish... ") print cmd +" can't finish .... "sys. exit (1) failed T: logging.info (CMD + "is wrong") print cmd + "is wrong! "Sys. exit (1) # obtain the most likely cc url list def ccdef_url (): Global urlalert, logpath, begintime, endtime # encapsulate the bash shell command, the search log is within two minutes before the current time. You can also use tail-F to save time. I do not understand the awk syntax, O & M engineers provide urlcmd = "awk '$4> \" ["+ begintime +" \ "& $4 <\" ["+ endtime +" \ "'" + logpath + "| awk '{A [$7] ++} end {for (I In) print a [I], I} '| sort-N | tail-N 10 | column-T "urloutput = exec_cmd (urlcmd) urllist = [] # standardize the output format and obtain the URL list for line in (urloutput. Split ('\ n'): urlcountlist = line. split () If int (urlcountlist [0])> urlalert: urllist. append (urlcountlist [1]) # logging.info ("cc's URL is:", urlcountlist [1]) return urllist # obtain the list of IP addresses that have accessed these URLs def ccdef_ip (urllist): Global ipalert, logpath, begintime, endtime iplist = [] for URL in urllist: # Change awk, obtain ipcmd = "awk '$4> \" ["+ begintime +" \ "& $4 <\" ["+ endtime +" \ "'" + logpath +" | grep "+ URL +" | WK '{A [$1] ++} end {for (I in a) print a [I], i} '| sort-N | tail-N 10 | column-T "ipoutput = exec_cmd (ipcmd) # standardize the output format and obtain the list of IP addresses greater than the warning value for line in (ipoutput. split ('\ n'): ipcountlist = line. split () If int (ipcountlist [0])> ipalert: iplist. append (ipcountlist [1]) # logging.info ("cc's URL is:", urlcountlist [1]) return iplist # Add the most suspicious attack IP address to the firewall def dropiptables (iplist): drop_ip_list = [] # first list the IP addresses of the existing iptables to achieve filtering and deduplication I Ptablesoutput = exec_cmd ('iptables-vnl ') for line in iptablesoutput. split ('\ n'): line = line. split () if not line [0]. isdigit (): Continue drop_ip_list.append (line [7]) for IP in iplist: If IP in drop_ip_list: continue dropcmd = "iptables-I input-s" + IP + "-J Drop" Print dropcmd # Do it !!! Exec_cmd (dropcmd) def main (): urllist = ccdef_url () iplist = ccdef_ip (urllist) print iplist dropiptables (iplist) passif _ name __= = "_ main _": Main ()

BASH Shell, written by O & M engineers, saves a lot of trouble. Take a good look at the shell script.

#!/bin/bash#while true#do        tail -f XXX.log > url.txt &        echo "wait 120s"        sleep 120s        kill -9 `ps -ef | grep "tail -f" | grep -v grep | awk ‘{print $2}‘ | xargs` > /dev/null        sleep 1        one=`head -1 url.txt | awk ‘{print $4}‘`        two=`tail -1 url.txt | awk ‘{print $4}‘`        #awk -v one=$one -v two=$two ‘$4>one&&$4<two‘ url.txt|awk ‘{a[$7]++}END{for(i in a)print a[i],i}‘|sort -n|tail -n 10|column -t        url_max=`awk -v one=$one -v two=$two ‘$4>one&&$4<two‘ url.txt|awk ‘{a[$7]++}END{for(i in a)print a[i],i}‘|sort -n|tail -n 10|column -t | awk ‘{if ($1 > 500) print $2}‘ | xargs`        echo $url_max >> url.log        for i in $url_max        do            ipaddress=`awk -v one=$one -v two=$two ‘$4>one&&$4<two‘ url.txt|grep "$i"|awk ‘{a[$1]++}END{for(i in a)print a[i],i}‘|sort -n|tail -n 10|column -t | awk ‘{if ($1 > 100) print $2}‘|xargs`            for ip in $ipaddress            do                iptables -L -n | grep "$ip"                if [ $? != 0 ]                then                    echo $ip >> url.log                    iptables -I INPUT -s $ip -j DROP                fi            done        done#done

 

CC detection blocks bots and proxies!