Sample code for batch obtaining and verifying HTTP proxy through python

This article describes how to obtain and verify HTTP proxy Python scripts in batches. For more information about how to obtain and verify HTTP proxy Python scripts in batches, see the following article, for more information, see

HTTP brute force cracking, credential stuffing, and some common skills, such:

1. when scanning Renren's network, I encountered a single account error twice and required to enter the verification code. the other party did not implement the IP policy.

I used to maintain the 0.1 million (username, password) queue to bypass the verification code. When a user name or password combination requires a verification code, the cracking sequence will be suspended and placed at the end of the queue for the next test to continue cracking the passwords of other accounts.

This ensures that 2/3 of the time is being cracked and scanned.

2. when I cracked a system account of Meituan, I encountered certain access restrictions for a single IP address, and the request frequency should not be too fast. So I hung up 72 HTTP proxies to solve this problem. It seems that requests from each IP address are normal, but in fact, the efficiency is quite impressive from the perspective of the whole program.

In this article, I sent my own HTTP script snippets, but there are only a few lines. The anonymous proxy is crawled from here: http://www.xici.net.co/nn/

First, obtain the proxy list:

from bs4 import BeautifulSoupimport urllib2of = open('proxy.txt' , 'w')for page in range(1, 160):  html_doc = urllib2.urlopen('http://www.xici.net.co/nn/' + str(page) ).read()  soup = BeautifulSoup(html_doc)  trs = soup.find('table', id='ip_list').find_all('tr')  for tr in trs[1:]:    tds = tr.find_all('td')    ip = tds[1].text.strip()    port = tds[2].text.strip()    protocol = tds[5].text.strip()    if protocol == 'HTTP' or protocol == 'HTTPS':      of.write('%s=%s:%s\n' % (protocol, ip, port) )      print '%s=%s:%s' % (protocol, ip, port)of.close()

Then verify whether the proxy is available because I used to crack the account of the Meituan network system, so I used the Meituan page tag:

#encoding=gbkimport httplibimport timeimport urllibimport threadinginFile = open('proxy.txt', 'r')outFile = open('available.txt', 'w')lock = threading.Lock()def test():  while True:    lock.acquire()    line = inFile.readline().strip()    lock.release()    if len(line) == 0: break    protocol, proxy = line.split('=')    headers = {'Content-Type': 'application/x-www-form-urlencoded',      'Cookie': ''}    try:      conn = httplib.HTTPConnection(proxy, timeout=3.0)      conn.request(method='POST', url='http://e.meituan.com/m/account/login', body='login=ttttttttttttttttttttttttttttttttttttt&password=bb&remember_username=1&auto_login=1', headers=headers )      res = conn.getresponse()      ret_headers = str( res.getheaders() )       html_doc = res.read().decode('utf-8')      print html_doc.encode('gbk')      if ret_headers.find(u'/m/account/login/') > 0:        lock.acquire()        print 'add proxy', proxy        outFile.write(proxy + '\n')        lock.release()      else:        print '.',    except Exception, e:      print eall_thread = []for i in range(50):  t = threading.Thread(target=test)  all_thread.append(t)  t.start()  for t in all_thread:  t.join()inFile.close()outFile.close()

The above is the details of the sample code for batch obtaining and verifying HTTP proxy through python. For more information, see other related articles in the first PHP community!