給HttpClient添加Socks代理

給HttpClient添加Socks代理

本文描述http client使用socks代理過程中需要注意的幾個方面：1，socks5支援使用者密碼授權；2，支援https；3，支援讓Proxy 伺服器解析DNS；

使用代理建立Socket

從原理上來看，不管用什麼http用戶端（httpclient，okhttp），最終都要轉換到java.net.Socket的建立上去，看到代碼：
package java.net;
 public Socket(Proxy proxy) {
    ...
 }

這是JDK中對網路請求使用Socks代理的入口方法。（http代理是在http協議層之上的，不在此文討論範圍之內）。
HttpClient要實現socks代理，就需要塞進去一個Proxy對象，也就是定製兩個類：org.apache.http.conn.ssl.SSLConnectionSocketFactory 和org.apache.http.conn.socket.PlainConnectionSocketFactory，分別對應https和http。
 代碼如下：
    private class SocksSSLConnectionSocketFactory extends SSLConnectionSocketFactory {

        public SocksSSLConnectionSocketFactory(SSLContext sslContext, HostnameVerifier hostnameVerifier) {
            super(sslContext, hostnameVerifier);
        }

        @Override
        public Socket createSocket(HttpContext context) throws IOException {
            ProxyConfig proxyConfig = (ProxyConfig) context.getAttribute(ProxyConfigKey);
            if (proxyConfig != null) {//需要代理
                return new Socket(proxyConfig.getProxy());
            } else {
                return super.createSocket(context);
            }
        }

        @Override
        public Socket connectSocket(int connectTimeout, Socket socket, HttpHost host, InetSocketAddress remoteAddress,
                                    InetSocketAddress localAddress, HttpContext context) throws IOException {
            ProxyConfig proxyConfig = (ProxyConfig) context.getAttribute(ProxyConfigKey);
            if (proxyConfig != null) {//make proxy server to resolve host in http url
                remoteAddress = InetSocketAddress
                        .createUnresolved(host.getHostName(), host.getPort());
            }
            return super.connectSocket(connectTimeout, socket, host, remoteAddress, localAddress, context);
        }
    }

和
    private class SocksSSLConnectionSocketFactory extends SSLConnectionSocketFactory {

        public SocksSSLConnectionSocketFactory(SSLContext sslContext, HostnameVerifier hostnameVerifier) {
            super(sslContext, hostnameVerifier);
        }

        @Override
        public Socket createSocket(HttpContext context) throws IOException {
            ProxyConfig proxyConfig = (ProxyConfig) context.getAttribute(ProxyConfigKey);
            if (proxyConfig != null) {
                return new Socket(proxyConfig.getProxy());
            } else {
                return super.createSocket(context);
            }
        }

        @Override
        public Socket connectSocket(int connectTimeout, Socket socket, HttpHost host, InetSocketAddress remoteAddress,
                                    InetSocketAddress localAddress, HttpContext context) throws IOException {
            ProxyConfig proxyConfig = (ProxyConfig) context.getAttribute(ProxyConfigKey);
            if (proxyConfig != null) {//make proxy server to resolve host in http url
                remoteAddress = InetSocketAddress
                        .createUnresolved(host.getHostName(), host.getPort());
            }
            return super.connectSocket(connectTimeout, socket, host, remoteAddress, localAddress, context);
        }
    }

然後在建立httpclient對象時，給HttpClientConnectionManager設定socketFactoryRegistry
            Registry<ConnectionSocketFactory> socketFactoryRegistry = RegistryBuilder.<ConnectionSocketFactory>create()
                .register(Protocol.HTTP.toString(), new SocksConnectionSocketFactory())
                .register(Protocol.HTTPS.toString(), new SocksSSLConnectionSocketFactory(sslContext, NoopHostnameVerifier.INSTANCE))
                .build();

        PoolingHttpClientConnectionManager connectionManager = new PoolingHttpClientConnectionManager(socketFactoryRegistry);
   

讓Proxy 伺服器解析網域名稱

情境：運行httpClient的進程所在主機可能並不能上公網，大部分時候，也無法進行DNS解析，這時通常會出現網域名稱無法解析的IO異常，下面介紹怎麼避免在用戶端解析網域名稱。

上面有一行代碼非常關鍵：
remoteAddress = InetSocketAddress
                        .createUnresolved(host.getHostName(), host.getPort());

變數host是你發起http請求的目標主機和連接埠資訊，這裡建立了一個未解析(Unresolved)的SocketAddress，在socks協議握手階段，InetSocketAddress資訊會原封不動的發送到Proxy 伺服器，由Proxy 伺服器解析出具體的IP地址。
Socks的協議描述中有個片段：
  The SOCKS request is formed as follows:

        +----+-----+-------+------+----------+----------+
        |VER | CMD |  RSV  | ATYP | DST.ADDR | DST.PORT |
        +----+-----+-------+------+----------+----------+
        | 1  |  1  | X'00' |  1  | Variable |    2    |
        +----+-----+-------+------+----------+----------+

    Where:

          o  VER    protocol version: X'05'
          o  CMD
            o  CONNECT X'01'
            o  BIND X'02'
            o  UDP ASSOCIATE X'03'
          o  RSV    RESERVED
          o  ATYP  address type of following address
            o  IP V4 address: X'01'
            o  DOMAINNAME: X'03'
            o  IP V6 address: X'04'

代碼按上面方法寫，協議握手發送的是ATYP=X'03'，即採用網域名稱的地址類型。否則，HttpClient會嘗試在用戶端解析，然後發送ATYP=X'01'進行協商。當然，大多數時候HttpClient在解析網域名稱的時候就掛了。

https中需要注意的問題

在使用httpclient訪問https網站的時候，經常會遇到javax.net.ssl包中的異常，例如：
Caused by: javax.net.ssl.SSLException: Received fatal alert: internal_error
    at sun.security.ssl.Alerts.getSSLException(Unknown Source) ~[na:1.7.0_80]
    at sun.security.ssl.Alerts.getSSLException(Unknown Source) ~[na:1.7.0_80]

一般需要做幾個設定：

建立不校正憑證鏈結的SSLContext
        SSLContext sslContext = null;
        try {
            sslContext = new SSLContextBuilder().loadTrustMaterial(null, new TrustStrategy() {
                @Override
                public boolean isTrusted(X509Certificate[] chain, String authType)
                        throws CertificateException {
                    return true;
                }

            }).build();
        } catch (Exception e) {
            throw new com.aliyun.oss.ClientException(e.getMessage());
        }
        ...
        new SocksSSLConnectionSocketFactory(sslContext, NoopHostnameVerifier.INSTANCE)

建立不校正網域名稱的HostnameVerifier
public class NoopHostnameVerifier implements javax.net.ssl.HostnameVerifier {

    public static final NoopHostnameVerifier INSTANCE = new NoopHostnameVerifier();

    @Override
    public boolean verify(final String s, final SSLSession sslSession) {
        return true;
    }
}

如何使用使用者密碼授權？

java SDK中給Socks代理授權有點特殊，不是按socket來的，而是在系統層面做的全域配置。比如，可以通過下面代碼設定一個全域的Authenticator：
Authenticator.setDefault(new MyAuthenticator("userName", "Password"));
...
class MyAuthenticator extends java.net.Authenticator {
    private String user ;
    private String password ;
 
    public MyAuthenticator(String user, String password) {
      this.user = user;
      this.password = password;
    }
 
    protected PasswordAuthentication getPasswordAuthentication() {
      return new PasswordAuthentication(user, password.toCharArray());
    }
  }

這種方法很簡單，不過有些不方便的地方，如果你的產品中需要串連不同的Proxy伺服器，而他們的使用者名稱密碼是不一樣的，那麼這個方法就不適用了。

基���ThreadLocal的Authenticator
public class ThreadLocalProxyAuthenticator extends Authenticator{
    private ThreadLocal<PasswordAuthentication> credentials = null;
    private static class SingletonHolder {
        private static final ThreadLocalProxyAuthenticator instance = new ThreadLocalProxyAuthenticator();
    }
    public static final ThreadLocalProxyAuthenticator getInstance() {
        return SingletonHolder.instance;
    }
      public void setCredentials(String user, String password) {
        credentials.set(new PasswordAuthentication(user, password.toCharArray()));
    }
    public static void clearCredentials() {
        ThreadLocalProxyAuthenticator authenticator = ThreadLocalProxyAuthenticator.getInstance();
        Authenticator.setDefault(authenticator);
        authenticator.credentials.set(null);
    }
    public PasswordAuthentication getPasswordAuthentication() {
        return credentials.get();
    }
}

這個類意味著，授權資訊只會儲存到當前調用者的線程中，其他線程的調用者無法訪問，在建立Socket的線程中設定密鑰和清理密鑰，就可以做到授權按照Socket串連進行隔離。Java TheadLocal相關知識本文不贅述。

按串連隔離的授權
 class ProxyHttpClient extends CloseableHttpClient{
    private CloseableHttpClient httpClient;
    public ProxyHttpClient(CloseableHttpClient httpClient){
        this.httpClient=httpClient;
    }
    protected CloseableHttpResponse doExecute(HttpHost target, HttpRequest request, HttpContext context) throws IOException, ClientProtocolException {
            ProxyConfig proxyConfig = //這裡擷取當前串連的代理配置資訊
            boolean clearCredentials = false;
            if (proxyConfig != null) {
                if (context == null) {
                    context = HttpClientContext.create();
                }
                context.setAttribute(ProxyConfigKey, proxyConfig);
                if (proxyConfig.getAuthentication() != null) {
                    ThreadLocalProxyAuthenticator.setCredentials(proxyConfig.getAuthentication());//設定授權資訊
                    clearCredentials = true;
                }
            }
            try {
                return httpClient.execute(target, request, context);
            } finally {
                if (clearCredentials) {//清理授權資訊
                    ThreadLocalProxyAuthenticator.clearCredentials();
                }
            }
        }
 }

另外，線程是可以複用的，因為每次調用完畢後，都清理了授權資訊。
 這裡有個一POJO類ProxyConfig，儲存的是socks代理的IP連接埠和使用者密碼資訊。
public class ProxyConfig {
    private Proxy proxy;
    private PasswordAuthentication authentication;
}

本文永久更新連結地址：https://www.bkjia.com/Linux/2018-03/151224.htm