python爬蟲學習（一）

標籤：ret   app   --   imp   logs   gen   div   cookie   secure   

基於python2.7get與post：

url = "http://zzk.cnblogs.com"urllib.urlopen(url)----->get方法name = urllib.urlencode({"k":"b"})urllib.urlopen(url+name)----->pst方法

開發人員工具中form表單的method選項為post，那麼必須使用post方法。

urllib：

import urllibimport rereponse = urllib.urlopen("https://www.baidu.com")         #開啟指定的網頁，返回網頁所有資訊reponse_code = reponse.getcode()                          #擷取狀態代碼reponse_body = reponse.read()                             #擷取網頁內容#直接儲存網頁地址的內容到指定的檔案save = urllib.urlretrieve("https://www.baidu.com", filename="/home/guido/python/baidu.html")images = re.findall(r"src=‘(.*?\.jpg)‘", reponse_body)    #利用Regex匹配資料urllib.urlretrieve(images[0], filename="/home/guido/python/baidu_images.html")

拼接連結格式

import urllibparament = urllib.urlencode({"t":"b", "w":"ios"})url = ("http://zzk.cnblogs.com/s?"+parament)print(url)執行結果：http://zzk.cnblogs.com/s?t=b&w=ios

　　

urllib2：

import urllib2url = "http://www.phpno.com"
#偽造瀏覽器要求標頭send_headers = {        "Accept":"text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,*/*;q=0.8",        "Accept-Encoding":"gzip, deflate, sdch",        "Accept-Language":"zh-CN,zh;q=0.8",        "Cache-Control":"max-age=0",        "Connection":"keep-alive",        "Cookie":"ASPSESSIONIDCCTRDBQT=OJNFDDEANPLCEFLECFILODNN; Hm_lvt_39dcd5bd05965dcfa70b1d2457c6dcae=1484820976,1484821014,1484821053; Hm_lpvt_39dcd5bd05965dcfa70b1d2457c6dcae=1484821053",        "Host":"www.nm3dp.com",        "Referer":"https://www.baidu.com/link?url=Q_AEn1rb05AX6miw616Tx5bIWILq5K_FpUQl_eyJ7TS&wd=&eqid=cb712bbf00052caf00000003588091e9",        "Upgrade-Insecure-Requests":"1",        "User-Agent":"Mozilla/5.0 (Windows NT 6.3; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/45.0.2454.101 Safari/537.36"}req = urllib2.Request(url, headers=send_headers)     #合并瀏覽器向伺服器發送的內容r = urllib2.urlopen(req)print(r.read())

　　

 

python爬蟲學習（一）