Python代理實現__Python

來源:互聯網
上載者:User



這兩天在看python代理IP實現,終於敲定了,哈哈,看代碼

from bs4 import BeautifulSoupimport requestsimport randomimport urllib2def get_ip_list(url, headers):    web_data = requests.get(url, headers=headers)    soup = BeautifulSoup(web_data.text, 'lxml')    ips = soup.find_all('tr')    ip_list = []    for i in range(1, len(ips)):        ip_info = ips[i]        tds = ip_info.find_all('td')        ip_list.append(tds[1].text + ':' + tds[2].text)    return ip_listdef get_random_ip(ip_list):    proxy_list = []    for ip in ip_list:        proxy_list.append('http://' + ip)    proxy_ip = random.choice(proxy_list)    proxies = {'http': proxy_ip}    return proxiesif __name__ == '__main__':    url = 'http://www.xicidaili.com/nn/'    headers = {        'User-Agent': 'Mozilla/5.0 (Windows NT 6.1; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/53.0.2785.143 Safari/537.36'    }    ip_list = get_ip_list(url, headers=headers)    proxies = get_random_ip(ip_list)    print(proxies)    enable_proxy = Trueproxy_handler = urllib2.ProxyHandler(proxies)null_proxy_handler = urllib2.ProxyHandler({})if enable_proxy:    opener = urllib2.build_opener(proxy_handler)else:    opener = urllib2.build_opener(null_proxy_handler)    urllib2.install_opener(opener)response = urllib2.urlopen('http://www.baidu.com', timeout=10)print(response.read())


聯繫我們

該頁面正文內容均來源於網絡整理,並不代表阿里雲官方的觀點,該頁面所提到的產品和服務也與阿里云無關,如果該頁面內容對您造成了困擾,歡迎寫郵件給我們,收到郵件我們將在5個工作日內處理。

如果您發現本社區中有涉嫌抄襲的內容,歡迎發送郵件至: info-contact@alibabacloud.com 進行舉報並提供相關證據,工作人員會在 5 個工作天內聯絡您,一經查實,本站將立刻刪除涉嫌侵權內容。

A Free Trial That Lets You Build Big!

Start building with 50+ products and up to 12 months usage for Elastic Compute Service

  • Sales Support

    1 on 1 presale consultation

  • After-Sales Support

    24/7 Technical Support 6 Free Tickets per Quarter Faster Response

  • Alibaba Cloud offers highly flexible support services tailored to meet your exact needs.