Network scan + Dede CMS fingerprint recognition example

Recently, a requirement is to specify IP segments and then identify dedecms Web applications from these IP segments.

First, this requirement should be divided into the following three points:

(1) scan the Web port from the IP segment. In this case, port 80 is used by default.

(2) obtain the IP address list from (1) and query the domain name, that is, obtain the domain name bound to each IP address.

(3) Dede fingerprint recognition for each item in the Domain Name List

For step (1), you can use socket for testing. To speed up scanning, you need to set the appropriate timeout time. (Sockfd. setTimeout (0.8) I am using 0.8 s)

With regard to step (2), I directly use the interface for Reverse Domain Name Lookup on the Internet IP address. However, the speed is relatively slow, but the expected results can be achieved.

For step 3, it is related to the web fingerprint recognition technology.

Common web fingerprint recognition technologies include the following:

(1) KEYWORDS found on the webpage (such as powered by XXX)

(2) identify the MD5 value of a specific file, such as the MD5 value of favicon. ICO.

(3) Specify the URL keyword

(4) Tag mode of the specified URL

In fact, for the specified CMS identification, I think the robots file is also very helpful, so here I used to detect the content in robots with identification.

This is the robots.txt of the general dededesite:

User-agent: * Disallow: /plus/feedback_js.phpDisallow: /plus/mytag_js.phpDisallow: /plus/rss.phpDisallow: /plus/search.phpDisallow: /plus/recommend.phpDisallow: /plus/stow.phpDisallow: /plus/count.php

Because of the time, there is no version test. Let's share my code below:

Dede_hunter.py

# Coding = utf-8import requests, JSON, urllib, sys, osfrom bs4 import beautifulsoupimport socketimport timeimport re ''' IP Lookup domain name class Demo: obtain the list of domain names bound to 202.20.2.1 ipre = ipreverse (); ipre. getdomainslist ('2017. raise 2.1 ') ''' class ipreverse (): # obtain the page content def getpage (self, IP, page): r = requests. get ("http://dns.aizhan.com/index.php? R = index/domains & IP = % S & page = % d "% (IP, page) return R # obtain the maximum page number def getmaxpage (self, ip ): R = self. getpage (IP, 1) json_data ={} json_data = R. JSON () If json_data = none: Return none maxcount = json_data [U 'conut'] maxpage = int (INT (maxcount)/20) + 1 return maxpage # retrieve Domain Name List def getdomainslist (self, ip): maxpage = self. getmaxpage (IP) If maxpage = none: Return none result = [] for X in xrange (1, maxpage + 1): R = Self. getpage (IP, x) result. append (R. JSON () [U "domains"]) return result ''' network scan class is given an IP segment scan specified port Demo: Given 202.203.208.8/24, scan port 80 myworkshop = firewall () ip_list = mylogs. webbench ('2017. 203.208.0 ', '2017. 203.208.255 ') ''' class evaluate (): # verify whether the specified IP address and port are enabled def porttings (self, IP, Port = 80): Server = (IP, Port) sockfd = socket. socket (socket. af_inet, socket. sock_stream) sockfd. setTimeout (0.8) ret = sockfd. connect_e X (server) # print RET if not RET: sockfd if 0 is returned successfully. close () print '% s: % s is opened... '% (IP, Port) return true else: sockfd. close () return false # convert the string IP address to ipdef ip2num (self, ip) of the number: Lp = [int (x) for X in IP. split ('. ')] return Lp [0] <24 | lp [1] <16 | lp [2] <8 | lp [3] # convert the numeric IP address to the string def num2ip (self, num): IP = ['',''] IP [3] = (Num & 0xff) IP [2] = (Num & 0xff00)> 8 IP [1] = (Num & 0xff0000)> 16 IP [0] = (Num & 0xff000000)> 24 return '% S. % S. % S. % s' % (IP [0], IP [1], IP [2], IP [3]) # Calculate the input IP Range def iprange (self, IP1, ip2): num1 = self. ip2num (IP1) num2 = self. ip2num (ip2) TMP = num2-num1 if TMP <0: Return none else: Return num1, num2, TMP # scan function def webhandler (self, startip, endip, Port = 80): ip_list = [] res = () RES = self. iprange (startip, endip) If res <0: Print 'endip must be bigger than startone' return non E sys. exit () else: for X in xrange (INT (RES [2]) + 1): startipnum = self. ip2num (startip) startipnum = startipnum + X if self. portshares (self. num2ip (startipnum), Port): ip_list.append (self. num2ip (startipnum) return ip_list'''check dedecms1.robots.txt 2. detection page powered by words '''class detectdedecms (): detects robots.txt def detectingrobots (self, URL): robots_content = ("disallow:/plus/feedback_js.php" or "disallow:/plus /Mytag_js.php "or" disallow:/plus/RSS. PHP "or" disallow:/plus/search. PHP "or" disallow:/plus/recommend. PHP "or" disallow:/plus/stow. PHP "or" disallow:/plus/count. PHP ") robots_url =" % S/% s "% (url,'robots.txt ') robots_page = requests. get (robots_url) If robots_page.status_code! = 200: Return falsecontent = robots_page.contentif content. Count (robots_content )! = 0: Return trueelse: Return false # powered by Dede detection def detectingpoweredby (self, raw_page): Soup = beautifulsoup (raw_page) pattern = Re. Compile (r 'dedecms .*? ') Try: text = soup. A. textexcept exception, E: Return falseif pattern. findall (text )! = []: Return trueelse: Return falsedef getresult (self, URL): url = 'HTTP: // % s' % urltry: r = requests. get (URL) raw_page = R. contentexcept exception, E: Return falseif (not r) or (R. status_code! = 200) or (not raw_page): Return falseis_robots_exists = self. detectingrobots (URL) is_poweredby_exists = self. detectingpoweredby (raw_page) If is_poweredby_exists or is_robots_exists: Return trueelse: Return falseclass worker (): def _ init _ (self, IP1, ip2): Self. startip = ip1self. endip = ip2def dojob (Self): myworkshop = reverse () ipreverse = ipreverse () dededetector = detectdedecms () domain_list = [] tmp_list = [] dede_res = [] ip_list = myworkshop. webparts (self. startip, self. endip) for X in ip_list: tmp_list = ipreverse. getdomainslist (x) If tmp_list = none: continuedomain_list = domain_list + tmp_listfor X in domain_list: If not X: continuefor I in X: If dededetector. getresult (I): dede_res.append (I) else: continuereturn dede_resif _ name _ = '_ main _': Begin = time. time () dede_res = [] myworker = worker ('2017. 235.5.52 ', '2017. 235.5.52 ') dede_res = myworker. dojob () Current = time. time ()-beginprint 'cost: % s' % STR (current) If dede_res = []: print 'No 'else: print' Detected Results:', dede_res

Test that the given IP address is:

219.235.5.52

More than 150 domain names are bound to this IP address .......

The result is as follows:

Verify to see if it is accurate ??

Recognition successful! However, the execution above also showed that the time was indeed very high, and my campus network was about s at 2 m .........

Zookeeper

Network scan + Dede CMS fingerprint recognition example