CC detection blocks bots and proxies!

Source: Internet
Author: User

The experiences of these days are merged into code.

Python, edited by me.

#! /Usr/bin/ENV Python #-*-coding: UTF-8-*-import OS, sys, timeimport commands, logging # The following is the action for manual commands # Time Format 17/OCT/2014: 10: 00: 00 # Time Format sat Oct 18 12:35:43 2014 # awk '$4> "[17/OCT/2014: 14: 00: 00 "& $4 <" [17/OCT/2014: 14: 02: 00 "'xxx. log | awk '{A [$7] ++} end {for (I in a) print a [I], i} '| sort-N | tail-N 10 | column-T | 500 # awk' $4> "[17/OCT/2014: 14: 00: 00 "& $4 <" [17/OCT/2014: 14: 02: 00 "'xxx. log | grep 'uc _ server/IM Ages/noavatar_small.gif '| awk' {A [$1] ++} end {for (I in a) print a [I], i} '| sort-N | tail-N 10 | column-T | 100 # defines the number of CC endpoints and the number of times the IP addresses access the CC URL when an alert is triggered. urlalert = 500 ipalert = 50 print 'urlaler is: ', urlalert,', ipalert is: ', ipalert # defines the log file address logpath = "XXX. log "# Processing interval of 120 seconds endtime = time. time () strendtime = time. ctime (endtime ). split () strbegintime = time. (endtime-120 ). split () # regular format, so that the python time format corresponds to the nginx log time format endtime = "% S/ % S/% s: % s "% (strendtime [2], strendtime [1], strendtime [4], strendtime [3]) begintime = "% S/% s: % s" % (strbegintime [2], strbegintime [1], strbegintime [4], strbegintime [3]) # Logging. basicconfig (Level = logging. info, format = '% (asctime) S % (filename) s [Line: % (lineno) D] % (levelname) S % (Message) s ', datefmt = '% A, % d % B % Y % H: % m: % s', filename = 'xxx. log', filemode = 'A') # General Command Processing Function def exec_cmd (CMD): (status, output) = Commands. getstatusoutput (CMD) Try: If int (Status) = 0: Print cmd + "is OK! "Logging.info (CMD +" --- is OK! ") Return output else: logging.info (CMD +" Cant't finish... ") print cmd +" can't finish .... "sys. exit (1) failed T: logging.info (CMD + "is wrong") print cmd + "is wrong! "Sys. exit (1) # obtain the most likely cc url list def ccdef_url (): Global urlalert, logpath, begintime, endtime # encapsulate the bash shell command, the search log is within two minutes before the current time. You can also use tail-F to save time. I do not understand the awk syntax, O & M engineers provide urlcmd = "awk '$4> \" ["+ begintime +" \ "& $4 <\" ["+ endtime +" \ "'" + logpath + "| awk '{A [$7] ++} end {for (I In) print a [I], I} '| sort-N | tail-N 10 | column-T "urloutput = exec_cmd (urlcmd) urllist = [] # standardize the output format and obtain the URL list for line in (urloutput. Split ('\ n'): urlcountlist = line. split () If int (urlcountlist [0])> urlalert: urllist. append (urlcountlist [1]) # logging.info ("cc's URL is:", urlcountlist [1]) return urllist # obtain the list of IP addresses that have accessed these URLs def ccdef_ip (urllist): Global ipalert, logpath, begintime, endtime iplist = [] for URL in urllist: # Change awk, obtain ipcmd = "awk '$4> \" ["+ begintime +" \ "& $4 <\" ["+ endtime +" \ "'" + logpath +" | grep "+ URL +" | WK '{A [$1] ++} end {for (I in a) print a [I], i} '| sort-N | tail-N 10 | column-T "ipoutput = exec_cmd (ipcmd) # standardize the output format and obtain the list of IP addresses greater than the warning value for line in (ipoutput. split ('\ n'): ipcountlist = line. split () If int (ipcountlist [0])> ipalert: iplist. append (ipcountlist [1]) # logging.info ("cc's URL is:", urlcountlist [1]) return iplist # Add the most suspicious attack IP address to the firewall def dropiptables (iplist): drop_ip_list = [] # first list the IP addresses of the existing iptables to achieve filtering and deduplication I Ptablesoutput = exec_cmd ('iptables-vnl ') for line in iptablesoutput. split ('\ n'): line = line. split () if not line [0]. isdigit (): Continue drop_ip_list.append (line [7]) for IP in iplist: If IP in drop_ip_list: continue dropcmd = "iptables-I input-s" + IP + "-J Drop" Print dropcmd # Do it !!! Exec_cmd (dropcmd) def main (): urllist = ccdef_url () iplist = ccdef_ip (urllist) print iplist dropiptables (iplist) passif _ name __= = "_ main _": Main ()

BASH Shell, written by O & M engineers, saves a lot of trouble. Take a good look at the shell script.

#!/bin/bash#while true#do        tail -f XXX.log > url.txt &        echo "wait 120s"        sleep 120s        kill -9 `ps -ef | grep "tail -f" | grep -v grep | awk ‘{print $2}‘ | xargs` > /dev/null        sleep 1        one=`head -1 url.txt | awk ‘{print $4}‘`        two=`tail -1 url.txt | awk ‘{print $4}‘`        #awk -v one=$one -v two=$two ‘$4>one&&$4<two‘ url.txt|awk ‘{a[$7]++}END{for(i in a)print a[i],i}‘|sort -n|tail -n 10|column -t        url_max=`awk -v one=$one -v two=$two ‘$4>one&&$4<two‘ url.txt|awk ‘{a[$7]++}END{for(i in a)print a[i],i}‘|sort -n|tail -n 10|column -t | awk ‘{if ($1 > 500) print $2}‘ | xargs`        echo $url_max >> url.log        for i in $url_max        do            ipaddress=`awk -v one=$one -v two=$two ‘$4>one&&$4<two‘ url.txt|grep "$i"|awk ‘{a[$1]++}END{for(i in a)print a[i],i}‘|sort -n|tail -n 10|column -t | awk ‘{if ($1 > 100) print $2}‘|xargs`            for ip in $ipaddress            do                iptables -L -n | grep "$ip"                if [ $? != 0 ]                then                    echo $ip >> url.log                    iptables -I INPUT -s $ip -j DROP                fi            done        done#done

 

CC detection blocks bots and proxies!

Related Article

Contact Us

The content source of this page is from Internet, which doesn't represent Alibaba Cloud's opinion; products and services mentioned on that page don't have any relationship with Alibaba Cloud. If the content of the page makes you feel confusing, please write us an email, we will handle the problem within 5 days after receiving your email.

If you find any instances of plagiarism from the community, please send an email to: info-contact@alibabacloud.com and provide relevant evidence. A staff member will contact you within 5 working days.

A Free Trial That Lets You Build Big!

Start building with 50+ products and up to 12 months usage for Elastic Compute Service

  • Sales Support

    1 on 1 presale consultation

  • After-Sales Support

    24/7 Technical Support 6 Free Tickets per Quarter Faster Response

  • Alibaba Cloud offers highly flexible support services tailored to meet your exact needs.