芝麻HTTP:設定Selenium+Chrome代理,seleniumchrome
微博登入限制了錯誤次數···加上Cookie大批帳號被封需要從Cookie池中 剔除被封的帳號··· 需要使用代理··· 無賴百度了大半天都是特麼的啥玩意兒???結果換成了 Google手到擒來 分分鐘解決(那麼問題來了?百度除了賣假藥還會幹啥?)
Selenium+Chrome認證代理不能通過options處理。只能換個方法使用擴充解決
原文地址:https://stackoverflow.com/questions/29983106/how-can-i-set-proxy-with-authentication-in-selenium-chrome-web-driver-using-pyth#answer-30953780 (Stack Overflow 這是個好地方啊)
# -*- coding: utf-8 -*-# @Time : 2017/11/15 9:50# @Author : 哎喲臥槽# @Site : # @File : pubilc.py# @Software: PyCharmimport stringimport zipfiledef create_proxyauth_extension(proxy_host, proxy_port, proxy_username, proxy_password, scheme='http', plugin_path=None): """代理認證外掛程式 args: proxy_host (str): 你的Proxy 位址或者網域名稱(str類型) proxy_port (int): 代理連接埠號碼(int類型) proxy_username (str):使用者名稱(字串) proxy_password (str): 密碼 (字串) kwargs: scheme (str): 代理方式 預設http plugin_path (str): 擴充的絕對路徑 return str -> plugin_path """ if plugin_path is None: plugin_path = 'vimm_chrome_proxyauth_plugin.zip' manifest_json = """ { "version": "1.0.0", "manifest_version": 2, "name": "Chrome Proxy", "permissions": [ "proxy", "tabs", "unlimitedStorage", "storage", "<all_urls>", "webRequest", "webRequestBlocking" ], "background": { "scripts": ["background.js"] }, "minimum_chrome_version":"22.0.0" } """ background_js = string.Template( """ var config = { mode: "fixed_servers", rules: { singleProxy: { scheme: "${scheme}", host: "${host}", port: parseInt(${port}) }, bypassList: ["foobar.com"] } }; chrome.proxy.settings.set({value: config, scope: "regular"}, function() {}); function callbackFn(details) { return { authCredentials: { username: "${username}", password: "${password}" } }; } chrome.webRequest.onAuthRequired.addListener( callbackFn, {urls: ["<all_urls>"]}, ['blocking'] ); """ ).substitute( host=proxy_host, port=proxy_port, username=proxy_username, password=proxy_password, scheme=scheme, ) with zipfile.ZipFile(plugin_path, 'w') as zp: zp.writestr("manifest.json", manifest_json) zp.writestr("background.js", background_js) return plugin_path
使用方法:
from selenium import webdriverfrom common.pubilc import create_proxyauth_extensionproxyauth_plugin_path = create_proxyauth_extension( proxy_host="XXXXX.com", proxy_port=9020, proxy_username="XXXXXXX", proxy_password="XXXXXXX")co = webdriver.ChromeOptions()# co.add_argument("--start-maximized")co.add_extension(proxyauth_plugin_path)driver = webdriver.Chrome(executable_path="C:\chromedriver.exe", chrome_options=co)driver.get("http://ip138.com/")print(driver.page_source)
無認證代理:
options = webdriver.ChromeOptions()options.add_argument('--proxy-server=http://ip:port') driver = webdriver.Chrome(executable_path="C:\chromedriver.exe", chrome_options=0ptions)driver.get("http://ip138.com/")print(driver.page_source)