HttpClient 4.3.6 tutorial chapter 2nd connection management (translation), httpclient4.3.6

HttpClient 4.3.6 tutorial chapter 2nd connection management (translation), httpclient4.3.6
Chapter 2 connection management 2nd it takes a lot of time to establish a persistent connection between a host and the other end. In addition, it takes a lot of time to exchange multiple information packages between two terminals. Handshake is especially important for low-level HTTP messages. If you repeatedly use public connections when executing multiple requests, the data throughput can be greatly improved.
By default, HTTP/1.1 allows HTTP connections to be reused by multiple requests. HTTP/1.0 is also compatible with terminals that use a clear mechanism for multiple requests to prioritize active connections. The HTTP proxy can also maintain active idle connections for a certain period of time to avoid subsequent requests from the same target host. This ability to maintain active connections usually involves persistent connections. HttpClient fully supports "persistent connection ".
2.2 The Http connection route HttpClient can establish a connection to the host or the route [contains complex intermediate connections-also known as hops (bounce)]. HttpClient identifies different route connections (flat, path, and layer ). A proxy link is used to connect multiple intermediate proxy services to the target host.
A "flat route" is created when a connection is in progress, the first connection, or only a proxy connection is used ". "Channel routing" will be created for the first connection and through the proxy link ". A route cannot generate a path when it leaves the proxy. When a layer-based protocol ends an existing connection, a "layer-based routing" is established ". When a target path is terminated or a connection no longer acts as a proxy is terminated, the Protocol is layered.
2.2.1 The RouteInfo interface of route computing indicates the information of a specified target host path, involving one or more intermediate steps or hops ). HttpRoute is a specific RouteInfo implementation. It cannot be changed (it is immutable ). HttpTracker is a variable RouteInfo execution condition. It is used for HttpClient to internally track the remaining hops pointing to the final route target (bounce ). HttpTracker will be updated if the next hop to the target is successfully executed. HttpRouteDirector is a help class that can be used to calculate the next step of routing. This class is used inside HttpClient.
HttpRoutePlanner is an interface that represents a policy and is used to calculate a complete route based on the execution context. HttpClient includes two default HttpRoutePlanner implementations. SystemDefaultRoutePlanner is based on java.net. ProxySelector. By default, the JVM proxy settings are loaded (one of the settings is selected in the browser running on the system feature or application ). The DefaultProxyRoutePlanner implementation does not take advantage of any Java System features, nor use any system or browser proxy settings. It always calculates routes through the same default proxy service.

2.2.2 secure HTTP connection if the information being transmitted between two terminals cannot be read or tampered by unauthorized users, the HTTP connection is considered safe. SSL/TLS protocols are widely used in HTTP Transmission Security. However, other encryption methods are also used. Generally, HTTP transmission is layered over SSL/TLS encrypted connections.

2.3 HTTP connection management 2.3.1 managing connections and connection managers HTTP connections are complex, stateful, and thread-safe. It needs to be managed as appropriate. An HTTP connection can only be used by one execution thread at a time. The HttpClientConnectionManager interface is a special entity used by HttpClient to manage HTTP connections. The HTTP Connection Manager is designed to act as a factory for creating new HTTP connections to manage the lifecycle and synchronization entry of persistent connections, so that only one thread can access one connection at a time. The internal HTTP Connection Manager works with the ManagedHttpClientConnection instance to act as a proxy service for a real connection to manage the connection status and control the execution of I/O. If a management connection is released by a consumer or is explicitly closed, the underlying connection is separated from the proxy service and returned to the manager. Although this service consumer will keep the reference of the proxy service instance, it is no longer allowed to perform any I/O operations and will not intentionally or unintentionally change the real connection status.
Here is an example of getting a connection from the Connection Manager:

HttpClientContext context = HttpClientContext.create();HttpClientConnectionManager connMrg = new BasicHttpClientConnectionManager();HttpRoute route = new HttpRoute(new HttpHost("localhost", 80));// Request new connection. This can be a long process ConnectionRequest connRequest = connMrg.requestConnection(route, null);// Wait for connection up to 10 secHttpClientConnection conn = connRequest.get(10, TimeUnit.SECONDS);try {    // If not open    if (!conn.isOpen()) {        // establish connection based on its route info        connMrg.connect(conn, route, 1000, context);        // and mark it as route complete        connMrg.routeComplete(conn, route, context);    }    // Do useful things with the connection.} finally {    connMrg.releaseConnection(conn, null, 1, TimeUnit.MINUTES);}

If necessary, the connection can be terminated early by ConnectionRequest # cancel. This causes thread blocking to be removed in the ConnectionRequest # get () method.

2.3.2 simple connection management BasicHttpClientConnectionManager is a simple Connection Manager that can only maintain one connection at a time. Although this class is thread-safe, it can only be used by one execution thread. BasicHttpClientConnectionManager tries to reuse the connection for subsequent requests with the same route. If the route of the persistent connection does not match the connection request, it closes the existing connection for the specified route and re-opens it. If the connection has been allocated, a java. lang. IllegalStateException will be thrown.
The implementation of the Connection Manager should be used in an EJB container.
2.3.3 The pool Connection Manager PoolingHttpClientConnectionManager is a more complex implementation. It manages a client connection pool and provides services for thread connection requests. Connections are aggregated on the basis of each route. For a routing request, if the manager already has an available persistent connection in the pool, it will not create a new connection, but will lease the connection in the pool.
PoolingHttpClientConnectionManager maintains the maximum number of connections based on each route. By default, no more than two parallel connections are created for each implementation, and no more than 20 connections are created for each specified route. For many practical applications, these restrictions may be too restrictive, especially when they use the HTTP Transport Protocol for their servers.
This example demonstrates how to adjust the connection pool parameters:

PoolingHttpClientConnectionManager cm = new PoolingHttpClientConnectionManager();// Increase max total connection to 200cm.setMaxTotal(200);// Increase default max connection per route to 20cm.setDefaultMaxPerRoute(20);// Increase max connections for localhost:80 to 50HttpHost localhost = new HttpHost("locahost", 80);cm.setMaxPerRoute(new HttpRoute(localhost), 50);CloseableHttpClient httpClient = HttpClients.custom()        .setConnectionManager(cm)        .build();

2.3.4 Connection Manager close when an HttpClient instance is no longer needed and is about to leave the scope of its role, close its connection manager to ensure that all connections remain active after the manager is closed, and the connected system resources will be released.

CloseableHttpClient httpClient = <...>httpClient.close();

2.4 thread request execution after a pool Connection Manager is configured, such as PoolingClientConnectionManager, HttpClient can execute parallel multiple requests using multiple threads.
PoolingClientConnectionManager allocates connections based on its configuration. If a specified route connection has been rented, the connection request will be blocked until a connection is released back to the pool. You can set a positive value for 'HTTP. conn-manager.timeout 'to ensure that the connection manager does not block the connection request operation indefinitely. If the connection request cannot obtain the service within the specified time, the ConnectionPoolTimeoutException exception will be thrown.

PoolingHttpClientConnectionManager cm = new PoolingHttpClientConnectionManager();CloseableHttpClient httpClient = HttpClients.custom()        .setConnectionManager(cm)        .build();// URIs to perform GETs onString[] urisToGet = {    "http://www.domain1.com/",    "http://www.domain2.com/",    "http://www.domain3.com/",    "http://www.domain4.com/"};// create a thread for each URIGetThread[] threads = new GetThread[urisToGet.length];for (int i = 0; i < threads.length; i++) {    HttpGet httpget = new HttpGet(urisToGet[i]);    threads[i] = new GetThread(httpClient, httpget);}// start the threadsfor (int j = 0; j < threads.length; j++) {    threads[j].start();}// join the threadsfor (int j = 0; j < threads.length; j++) {    threads[j].join();}

HttpClient instances are thread-safe and can be shared among multiple threads. It is strongly recommended that each thread maintain its own dedicated HttpContext instance.

static class GetThread extends Thread {    private final CloseableHttpClient httpClient;    private final HttpContext context;    private final HttpGet httpget;    public GetThread(CloseableHttpClient httpClient, HttpGet httpget) {        this.httpClient = httpClient;        this.context = HttpClientContext.create();        this.httpget = httpget;    }    @Override    public void run() {        try {            CloseableHttpResponse response = httpClient.execute(                    httpget, context);            try {                HttpEntity entity = response.getEntity();            } finally {                response.close();            }        } catch (ClientProtocolException ex) {            // Handle protocol errors        } catch (IOException ex) {            // Handle I/O errors        }    }}

2.5 The classic I/O blocking mode of the connection recovery policy has one major drawback: When I/O operations are blocked, the network socket only affects I/O events. When a connection is released back to the manager, it remains active, but it does not listen to the socket status and any I/O events. If the connection is closed on the server side, the client connection will not detect this change in the connection status (and the appropriate response is being closed at the end.
HttpClient tries to alleviate this problem by testing whether the connection is "stale", because it will be disabled by the server, before that, the connection will be used for the HTTP request being executed. The "stale" connection detection is not effective and will increase the execution of each request by 10 to 10 milliseconds. For idle connections, the only effective solution is that each socket model does not contain a thread, and a dedicated listening thread is used to evict expired inactive persistent connections. This listening thread periodically calls the ClientConnectionManager # closeExpiredConnections () method to close all expired connections and evict closed connections from the pool. After the specified period expires, you can also call the ClientConnectionManager # closeIdleConnections () method to close all connections at will.

public static class IdleConnectionMonitorThread extends Thread {        private final HttpClientConnectionManager connMgr;    private volatile boolean shutdown;        public IdleConnectionMonitorThread(HttpClientConnectionManager connMgr) {        super();        this.connMgr = connMgr;    }    @Override    public void run() {        try {            while (!shutdown) {                synchronized (this) {                    wait(5000);                    // Close expired connections                    connMgr.closeExpiredConnections();                    // Optionally, close connections                    // that have been idle longer than 30 sec                    connMgr.closeIdleConnections(30, TimeUnit.SECONDS);                }            }        } catch (InterruptedException ex) {            // terminate        }    }        public void shutdown() {        shutdown = true;        synchronized (this) {            notifyAll();        }    }    }

2.6 The HTTP specification does not specify how long a persistent connection can remain active at most. Some HTTP servers use a non-standard Keep-Alive (Keep active) header to tell the client that they plan to Keep the connection active on the server side (in seconds ). If this information is available, HttpClient will use this information. If the Keep-Alive header does not appear in the response, HttpClient assumes that the connection can remain active indefinitely. However, many HTTP servers are usually configured to discard persistent connections after an inactive period. In order to save the system, the client is often not notified. The default policy seems too optimistic. You may want to provide a custom active strategy.

ConnectionKeepAliveStrategy myStrategy = new ConnectionKeepAliveStrategy() {    public long getKeepAliveDuration(HttpResponse response, HttpContext context) {        // Honor 'keep-alive' header        HeaderElementIterator it = new BasicHeaderElementIterator(                response.headerIterator(HTTP.CONN_KEEP_ALIVE));        while (it.hasNext()) {            HeaderElement he = it.nextElement();            String param = he.getName();            String value = he.getValue();            if (value != null && param.equalsIgnoreCase("timeout")) {                try {                    return Long.parseLong(value) * 1000;                } catch(NumberFormatException ignore) {                }            }        }        HttpHost target = (HttpHost) context.getAttribute(                HttpClientContext.HTTP_TARGET_HOST);        if ("www.naughty-server.com".equalsIgnoreCase(target.getHostName())) {            // Keep alive for 5 seconds only            return 5 * 1000;        } else {            // otherwise keep alive for 30 seconds            return 30 * 1000;        }    }};CloseableHttpClient client = HttpClients.custom()        .setKeepAliveStrategy(myStrategy)        .build();

2.7 connect to the socket Factory

The HTTP connection uses the internal java.net. Socket object to process data transmitted from the wire. However, they rely on interfaces to create, initialize, and connect the socket. At runtime, The HttpClient user is allowed to configure the specified socket initialization code. PlainConnectionSocketFactory is a default job for creating and initializing flat (unencrypted) sockets.

The process of creating a socket is decoupled from connecting it to a host. Therefore, when a connection operation is blocked, you should close the socket.

HttpClientContext clientContext = HttpClientContext.create();PlainConnectionSocketFactory sf = PlainConnectionSocketFactory.getSocketFactory();Socket socket = sf.createSocket(clientContext);int timeout = 1000; //msHttpHost target = new HttpHost("localhost");InetSocketAddress remoteAddress = new InetSocketAddress(        InetAddress.getByAddress(new byte[] {127,0,0,1}), 80);sf.connectSocket(timeout, socket, target, remoteAddress, null, clientContext);

2.7.1 Secure socket layering

LayeredConnectionSocketFactory is an extension of the ConnectionSocketFactory interface. The layered socket factory has the ability to create layered sockets on an existing flat socket. The layered socket is first used by the proxy service to create a secure socket. HttpClient includes SSLSocketFactory to implement SSL/TLS layering. Please note that HttpClient does not use any custom encryption function. It relies entirely on standard Java Cryptography (JCE) and Secure Sockets (JSEE) extensions.

2.7.2 integrated connection management a custom connection socket factory can be associated with a special protocol system, such as HTTP or HTTPS, to create custom connection management.

ConnectionSocketFactory plainsf = <...>LayeredConnectionSocketFactory sslsf = <...>Registry<ConnectionSocketFactory> r = RegistryBuilder.<ConnectionSocketFactory>create()        .register("http", plainsf)        .register("https", sslsf)        .build();HttpClientConnectionManager cm = new PoolingHttpClientConnectionManager(r);HttpClients.custom()        .setConnectionManager(cm)        .build();

2.7.3 SSL/TLS Customization

HttpClient uses SSLConnectionSocketFactory to create an SSL connection. SSLConnectionSocketFactory allows high customization. It can regard the javax.net. ssl. SSLContext instance as a parameter and use it to create custom SSL connection configurations.

KeyStore myTrustStore = <...>SSLContext sslContext = SSLContexts.custom()        .useTLS()        .loadTrustMaterial(myTrustStore)        .build();SSLConnectionSocketFactory sslsf = new SSLConnectionSocketFactory(sslContext);

Custom SSLConnectionSocketFactory requires more in-depth understanding of the SSL/TLS protocol, which is beyond the scope of this document. For more information about javax.net. ssl. SSLContext and related tools, see Java Secure Socket Extension (link: http://docs.oracle.com/javase/1.5.0/docs/guide/security/jsse/JSSERefGuide.html ).

2.7.4 Host Name check

In addition to hosting and customer identity authentication at the SSL/TLS protocol level, once a connection is created, HttpClient can also selectively check whether the target host name matches the X.509 certificate stored on the server. This test can provide additional guarantees for the server to believe in the reliability of materials. The X509HostnameVerifier interface represents a policy for hostname verification. HttpClient contains three X509HostnameVerifier implementations. Note: Do not confuse the Host Name check with the SSL hosting check.

StrictHostnameVerifier (validator for exact host names ):The exact host name validator works in a format similar to Sun Java 1.4, Sun Java 5, and Sun Java 6. It is also closely related to IE6. This implementation complies with RFC 2818 because wildcards must be processed. The host name must either match the first CN or any subject-alts. Wildcard characters can appear in CN and any subject-alts.
BrowserCompatHostnameVerifier (Browser compatible host name validator ):This host name validator works in browsers similar to Curl and Firefox. The host name must either match the first CN or any subject-alts. Wildcard characters can appear in CN and any subject-alts. The only difference between BrowserCompatHostnameVerifier and StrictHostnameVerifier is that BrowserCompatHostnameVerifier's wildcard (such as "* .foo.com") matches all subdomains, including "a. B .foo.com ".
AllowAllHostnameVerifier (allows the validator for all host names ):This host name validator will essentially turn off the host inspection. This implementation is non-operational (no-op) and will always throw a javax.net. ssl. SSLException exception.

The default HttpClient is implemented using BrowserCompatHostnameVerifier. You can specify different host name validators as needed.

SSLContext sslContext = SSLContexts.createSystemDefault();SSLConnectionSocketFactory sslsf = new SSLConnectionSocketFactory(        sslContext,        SSLConnectionSocketFactory.STRICT_HOSTNAME_VERIFIER);

2.8 HttpClient Proxy Server Configuration

Although HttpClient knows the complex routing system and proxy service links, it only supports Simple locating or a quit hop proxy connection.

HttpRoutePlanner routePlanner = new HttpRoutePlanner() {    public HttpRoute determineRoute(            HttpHost target,            HttpRequest request,            HttpContext context) throws HttpException {        return new HttpRoute(target, null,  new HttpHost("someproxy", 8080),                "https".equalsIgnoreCase(target.getSchemeName()));    }};CloseableHttpClient httpclient = HttpClients.custom()        .setRoutePlanner(routePlanner)        .build();    }}

Translator: lianghongge