Jetty Development Guide: HTTP client

Introduction

The jetty HTTP client module provides easy-to-use APIs, tool classes, and a high-performance, asynchronous implementation to execute HTTP and HTTPS requests.
The jetty HTTP client module requires Java version 1.7 or later. Java 1.8 applications can use lambda expressions in some HTTP client APIs.
Jetty HTTP client is implemented and provides an asynchronous API, so it will not be blocked due to I/O time, so it is more efficient in thread utilization, it is suitable for Load Testing and parallel computing.
However, sometimes all you need to do is execute a GET request to a resource, and the HTTP client also provides an asynchronous API. The thread that initiates the request will be blocked until the request processing is complete.
Externally, Jetty HTTP client provides:
1) supports redirection. The redirection encoding, such as 302 or 303, is automatically followed;
2) Cookies support; cookies sent by the server are sent back to the server in matching requests;
3) authentication support; http "Basic" and "Digest" fever are supported, and others can be added;
4) Support for forwarding protocols.

Initialization

The main class name is Org. eclipse. jetty. client. httpclient is the same as jetty 7 and jetty 8 (although it is not backward compatible with the same name class in jetty 7 and jetty 8 ).
You can regard an httpclient instance as a browser instance. Like a browser, it can initiate requests to different domains. It manages redirection, cookies, and authentication. You can configure it with a proxy and provide you with a response to the request you initiate.
To use httpclient, you must initialize it, configure it, and then start it:

// Instantiate HttpClientHttpClient httpClient = new HttpClient(); // Configure HttpClient, for example:httpClient.setFollowRedirects(false); // Start HttpClienthttpClient.start();

You can create multiple httpclient instances. The reason may be that you want to specify different configuration parameters (for example, one instance is configured as a forward proxy, and the other is not ), or because you want two instances to perform like two different browsers, there are different cookies, different authentication certificates, and so on.
When you use the parameter constructor to create an httpclient instance, you can only perform simple HTTP requests, and you will not be able to perform HTTPS requests.
To fulfill HTTPS requests, you should first create an sslcontextfactory, configure it, and pass it to the httpclient constructor. When an sslcontextfactory is created, httpclient can fulfill HTTP and HTTPS requests to any domain.

// Instantiate and configure the SslContextFactorySslContextFactory sslContextFactory = new SslContextFactory(); // Instantiate HttpClient with the SslContextFactoryHttpClient httpClient = new HttpClient(sslContextFactory); // Configure HttpClient, for example:httpClient.setFollowRedirects(false); // Start HttpClienthttpClient.start();

API introduction blocking API

To fulfill an HTTP request, the simpler method is:

ContentResponse response = httpClient.GET(<a target=_blank href="http://domain.com/path?query">http://domain.com/path?query</a>);

Method httpclient. Get (...) to fulfill an http get request to a given URI. After successful, a contentresponse is returned.
The contentresponse object contains the HTTP Response Information: Status Code, headers, and possible content. The content length is limited to 2 MB by default. The following "response content processing" describes how to process larger content.
If you want to customize a request, for example, by initiating a head request instead of a get, and simulating a browser user proxy, you can use this method:

ContentResponse response = httpClient.newRequest("http://domain.com/path?query")        .method(HttpMethod.HEAD)        .agent("Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:17.0) Gecko/20100101 Firefox/17.0")        .send();

The following is a simplified method:

Request request = httpClient.newRequest("http://domain.com/path?query");request.method(HttpMethod.HEAD);request.agent("Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:17.0) Gecko/20100101 Firefox/17.0");ContentResponse response = request.send();

You first create a request object using httpclient. newrequest (...) and then customize it. After the request object is customized, you call request. Send (). After the request is processed, contentresponse is returned.
A simple POST request also has a short method:

ContentResponse response = httpClient.POST("http://domain.com/entity/1")        .param("p", "value")        .send();

The post parameter value is automatically URL encoded.
Jetty HTTP client automatically follows redirection, so it automatically processes this typical Web mode post/redirect/get, and the response object contains the response content of the GET request. Following redirection is a feature that allows you to activate/stop each request or globally.
A line is also required for file upload. Use the java. NiO. File class of JDK 7:

ContentResponse response = httpClient.newRequest("http://domain.com/upload")        .file(Paths.get("file_to_upload.txt"), "text/plain")        .send();

You can also add a timeout value:

ContentResponse response = httpClient.newRequest("http://domain.com/path?query")        .timeout(5, TimeUnit.SECONDS)        .send();

In the preceding example, after more than five seconds, the request is terminated and a java. util. Concurrent. timeoutexception exception is thrown.

Asynchronous API

So far, we have demonstrated how to use the jetty HTTP client blocking API, that is, the thread initiating the request is blocked until the request is processed completely. In this section, we will look at the asynchronous and non-blocking APIs of jetty HTTP client, it is very suitable for all scenarios where big data download, concurrent processing of requests/responses, performance, effective threads, and resource utilization are a key factor.
Asynchronous APIs depend on the call to callback listener at each stage of request and Response Processing. These listener are implemented by the application and can fulfill any application logic. The listener is called in the thread that processes the request or response. Therefore, if the application code in these listener takes a long time, the request or response processing will be blocked.
If you need to perform time-consuming operations in a listener, you must use your own thread and remember to copy any data provided by listener in depth, because when listener returns, the data may be recycled, cleared, or destroyed.
Requests and responses are executed in two different threads, so they can be executed in parallel. A typical example of parallel processing is a echo server, where large uploads and large echo downloads are performed simultaneously. Note: Remember that the response can be processed and completed before the request. A typical example is a large upload (for example, an error) that is triggered by the server for a quick response ): when the request content is uploaded, the response has arrived and been completed.
The Application Thread calls request. Send (completelistener) to handle the request until or when the request is fully processed or returned due to blocking in I/O (so the request is never blocked ). If it will block in I/O, the thread requests the I/O system to issue an event when I/O is complete and then returns. When such an event is triggered, a thread from the httpclient thread pool will resume Response Processing.
The response is processed by the thread. These threads are either the I/O system threads that have triggered the bytecode to be prepared, or a thread from the httpclient thread pool (through httpclient. isdispatchio () attribute control ). The response is continuously processed until the response is processed or blocked in I/O. If it is blocked in I/O, the thread requests the I/O system to issue a time after I/O is ready and then returns. When such an event is triggered, a thread from the httpclient thread pool will resume Response Processing.
The thread that completes the final processing after the request and response are processed (usually the thread that processes the response, but it may also be the request processing thread-if the request takes more time than the response processing), it will take the next request for processing.
An asynchronous GET request that discards the response content can be implemented as follows:

httpClient.newRequest("http://domain.com/path")        .send(new Response.CompleteListener()        {            @Override            public void onComplete(Result result)            {                // Your logic here            }        });

Method Request. Send (response. completelistener) returns void without blocking. After the request/Response Processing is complete, response. completelistener will be notified, and the result parameter can get the response object.
You can use the lambda expression of JDK 8 to write the same code:

httpClient.newRequest("http://domain.com/path")        .send((result) -> { /* Your logic here */ });

You can also specify a total timeout value for it:

Request request = httpClient.newRequest("http://domain.com/path")        .timeout(3, TimeUnit.SECONDS)        .send(new Response.CompleteListener()        {            @Override            public void onComplete(Result result)            {                // Your logic here            }        });

In the preceding example, a timeout value of 3 seconds is specified for Request/Response Processing.
HTTP client APIs use listener extensively to provide hooks for all possible requests and response times. In the lambda expressions of JDK 8, they become easier to use:

httpClient.newRequest("http://domain.com/path")        // Add request hooks        .onRequestQueued((request) -> { ... })        .onRequestBegin((request) -> { ... })        ... // More request hooks available         // Add response hooks        .onResponseBegin((response) -> { ... })        .onResponseHeaders((response) -> { ... })        .onResponseContent((response, buffer) -> { ... })        ... // More response hooks available         .send((result) -> { ... });

This makes jetty HTTP client very suitable for HTTP load testing, for example, you can precisely know the time spent in each step of request/Response Processing (so you know where the request/response time is actually consumed ).
For more information about request events, see request. listener. For more information about response events, see response. listener.

Content Processing request content processing

Jetty HTTP client provides many ready-made tools to process request content.
You can provide the request content in these formats: String, byte [], bytebuffer, Java. NIO. file. path, inputstream, and provide your org. eclipse. jetty. client. API. contentprovider implementation. The following example uses Java. NiO. file. Paths to provide the request content:

ContentResponse response = httpClient.newRequest("http://domain.com/upload")        .file(Paths.get("file_to_upload.txt"), "text/plain")        .send();

This is equivalent to using the pathcontentprovider tool class as follows:

ContentResponse response = httpClient.newRequest("http://domain.com/upload")        .content(new PathContentProvider(Paths.get("file_to_upload.txt")), "text/plain")        .send();

Similarly, you can use fileinputstream through the inputstreamcontentprovider tool class:

ContentResponse response = httpClient.newRequest("http://domain.com/upload")        .content(new InputStreamContentProvider(new FileInputStream("file_to_upload.txt")), "text/plain")        .send();

Because inputstream is blocked, request sending is blocked. You can consider using asynchronous APIs.
If you have read the content to the memory, you can use the bytescontentprovider tool class to pass it as byte:

byte[] bytes = ...;ContentResponse response = httpClient.newRequest("http://domain.com/upload")        .content(new BytesContentProvider(bytes), "text/plain")        .send();

If the request content is not immediately available, you can use deferredcontentprovider:

DeferredContentProvider content = new DeferredContentProvider();httpClient.newRequest("http://domain.com/upload")        .content(content)        .send(new Response.CompleteListener()        {            @Override            public void onComplete(Result result)            {                // Your logic here            }        }); // Content not available yet here ... // An event happens, now content is availablebyte[] bytes = ...;content.offer(ByteBuffer.wrap(bytes)); ... // All content has arrivedcontent.close();

Another way to provide the request content is to use outputstreamcontentprovider. When the request content is available, allow the application to write the request content to outputstream provided by outputstreamcontentprovider:

OutputStreamContentProvider content = new OutputStreamContentProvider(); // Use try-with-resources to close the OutputStream when all content is writtentry (OutputStream output = content.getOutputStream()){    client.newRequest("localhost", 8080)            .content(content)            .send(new Response.CompleteListener()            {                @Override                public void onComplete(Result result)                {                    // Your logic here                }            });     ...     // Write content    writeContent(output);}// End of try-with-resource, output.close() called automatically to signal end of content

Response content processing

Jetty HTTP client allows applications to process response content in multiple ways.
The first method is to cache the response content in the memory. With the blocking API, the maximum cache of the content in a contentresponse is 2mib.
If you want to control the length of the response content (for example, to the default value smaller than 2 MIB), you can use org. Eclipse. Jetty. Client. util. futureresponselistener:

Request request = httpClient.newRequest("http://domain.com/path"); // Limit response content buffer to 512 KiBFutureResponseListener listener = new FutureResponseListener(request, 512 * 1024); request.send(listener); ContentResponse response = listener.get(5, TimeUnit.SECONDS);

If the response Content Length escapes and the response is terminated, an exception is thrown by the get () method.
If you are using an asynchronous API, you can use the bufferingresponselistener tool class:

httpClient.newRequest("http://domain.com/path")        // Buffer response content up to 8 MiB        .send(new BufferingResponseListener(8 * 1024 * 1024)        {            @Override            public void onComplete(Result result)            {                if (!result.isFailed())                {                    byte[] responseContent = getContent();                    // Your logic here                }            }        });

The second method is the most efficient (because it avoids content copying) and allows you to specify a response. contentlistener or a subclass to process the arrived content:

ContentResponse response = httpClient        .newRequest("http://domain.com/path")        .send(new Response.Listener.Empty()        {            @Override            public void onContent(Response response, ByteBuffer buffer)            {                // Your logic here            }        });

The third method allows you to wait for a response and use the inputstreamresponselistener tool to output the content:

InputStreamResponseListener listener = new InputStreamResponseListener();httpClient.newRequest("http://domain.com/path")        .send(listener); // Wait for the response headers to arriveResponse response = listener.get(5, TimeUnit.SECONDS); // Look at the responseif (response.getStatus() == 200){    // Use try-with-resources to close input stream.    try (InputStream responseContent = listener.getInputStream())    {        // Your logic here    }}

Other features cookies supported

Native support of jetty HTTP client cookie. The httpclient instance receives cookies from the HTTP Response and stores them in java.net. cookiestore. This class belongs to JDK. When a new request is created, the cookie cache is checked. If a matched cookie (that is, coolie does not escape and matches the domain and Request Path) exists, these cookies are added to the request.
Applications can program the cookie cache to find the configured COOKIE:

CookieStore cookieStore = httpClient.getCookieStore();List<HttpCookie> cookies = cookieStore.get(URI.create(<a target=_blank href="http://domain.com/path">http://domain.com/path</a>));

Applications can also set cookies programmatically if they return a response from an http:

CookieStore cookieStore = httpClient.getCookieStore();HttpCookie cookie = new HttpCookie("foo", "bar");cookie.setDomain("domain.com");cookie.setPath("/");cookie.setMaxAge(TimeUnit.DAYS.toSeconds(1));cookieStore.add(URI.create("http://domain.com"), cookie);

You can remove cookies that you do not want to use again:

CookieStore cookieStore = httpClient.getCookieStore();URI uri = URI.create("http://domain.com");List<HttpCookie> cookies = cookieStore.get(uri);for (HttpCookie cookie : cookies)    cookieStore.remove(uri, cookie);

If you want to completely disable cookie processing, you can install an httpcookiestore. Empty instance:

httpClient.setCookieStore(new HttpCookieStore.Empty());

You can activate cookie filtering by installing a cookie cache that implements the filtering logic:

httpClient.setCookieStore(new GoogleOnlyCookieStore()); public class GoogleOnlyCookieStore extends HttpCookieStore{    @Override    public void add(URI uri, HttpCookie cookie)    {        if (uri.getHost().endsWith("google.com"))            super.add(uri, cookie);    }}

In the preceding example, only cookies from the Google.com domain or subdomain are retained.

Authentication support

Jetty HTTP client supports "Basic" and "Digest" authentication mechanisms, which are defined in RFC 2617.
You can configure the authentication certificate in the HTTP client instance as follows:

URI uri = new URI("http://domain.com/secure");String realm = "MyRealm";String user = "username";String pass = "password"; // Add authentication credentialsAuthenticationStore auth = httpClient.getAuthenticationStore();auth.addAuthentication(new BasicAuthentication(uri, realm, user, pass)); ContentResponse response = httpClient        .newRequest(uri)        .send()        .get(5, TimeUnit.SECONDS);

Successful authentication is cached, but you can clear them and force re-Authentication:

httpClient.getAuthenticationStore().clearAuthenticationResults();

Proxy Support

The HTTP client of jetty can be configured with a proxy.
Two types of proxies are supported by the native: HTTP proxy (through the org. eclipse. jetty. client. httpproxy) and socks 4 proxy (through the org. eclipse. jetty. client. provided by socks4proxy ). Other implementations can be written through the subclass proxyconfiguration. Proxy.
A typical configuration is as follows:

ProxyConfiguration proxyConfig = httpClient.getProxyConfiguration();HttpProxy proxy = new HttpProxy("proxyHost", proxyPort);// Do not proxy requests for localhost:8080proxy.getExcludedAddresses().add("localhost:8080"); httpClient.setProxyConfiguration(proxyConfig); ContentResponse response = httpClient.GET(uri);

You specify the host and port of the proxy, set the address that you do not want to be proxy, and then set the proxy configuration on the httpclient instance.