HttpClient in Layman's ~

This article emphasizes the use, the simple narration httpclient, actually in the network programming, the Java-based realization almost all is wraps the socket the communication, then simulates the various kinds of protocol; HttpClient is actually simulating the browser to initiate the server-side request, And this more similar to the JS request or page post, get, but this kind of data return generally need to get meaningful data, it is convenient to do other interaction, otherwise get a page result, all is the label, after all, is not a browser, So we use httpclient more to make the interaction of the system more simple, this article from how to use HttpClient to start the description to the performance of the optimization method cut into:

1. HttpClient Client Invocation example, and what needs to be done on the server side.

2, security how to control.

3, httpclient in the concurrency of a higher number of calls under how to solve the problem.

1. HttpClient Client Invocation example, and what needs to be done on the server side.

First of all, what needs to be done on the server side, HttpClient simulation is a browser, to the server side of the data interaction, it is the same as the browser to initiate the request, accept the requested operation, but it and the browser and the server interaction the biggest difference is that it has no login action, Of course, you can also complete the cookie by simulating the login, but this code is hard to write, and the account must be in your code to specify the login account to obtain the cookie, if the program is self-use or can be, if a lot of people use a bit messy, Because each person's password you also have to use a certain method to pass to the server side, but no cookie obviously will be intercepted by the server to a direct login interface, so that you do not get the data you want, so we first put aside the security issue, then is to open some of the URL, That is, the URL does not pass through the filter, or to open some directories alone access, security issues, the second chapter to discuss, OK, if the server side open a URL path, client access is like a browser to access a URL as simple, with httpclient how to access it?

Before using, it is necessary to understand that httpclient is provided by Apache, so we need to introduce the relevant package first, it basically needs several packages to use it:

Commons-logging, Commons-httpclient, Commons-codec specific versions as well as the introduction of the method should be based on the project and the project packaging methods to decide, at present, the introduction of MAVEN is a more convenient method, and then introduced in the code before:

Import org.apache.commons.httpclient.*;

Import org.apache.commons.httpclient.*;

< by the way. * This argument, some people say that using this * is very slow, for the current JVM can only say that it is nonsense, one of the only advantage of introducing is that you can quickly know that this class is under the package, but definitely not improve performance, the JVM at compile time has long determined what is needed, Which is not required, if the same package is introduced too much, even if each individual write, the JVM will also give the memory structure changed to *,JVM will not because a class introduced a few *, will modify their link structure, initialization is determined by the parent-child integration relationship and the packaging relationship, and the operation is determined by the optimizer >

Let's start with a very simple example of a GET request:

1. try {
2. HttpClient client = new
3. HttpClient ();//Define Client Object
5. Client.gethttpconnectionmanager (). Getparams (). Setconnectiontimeout (2000);//Set connection time-out to 2 seconds (connection initialization time)
7. GetMethod method = new GetMethod ("http://www.google.com.hk/");//Visit Google's homepage
9. int StatusCode =
10. Client.executemethod (method);//status, General 200 is OK State, other conditions will throw such as 404,500,403 errors
12. if (statusCode! = HTTPSTATUS.SC_OK) {
14. SYSTEM.OUT.PRINTLN ("Remote access failed. ");
15. }
17. System.out.println (Method.getresponsebodyasstring ());//Output feedback Results
19. Client.gethttpconnectionmanager (). closeidleconnections (1);
20. }catch (...)
21. {.....}

Copy Code

Note that the above feedback may not be the same as you use a socket to simulate some systems, because the returned content is not any value, is the page label, when you interact with another system, it does response data, it can return the specified JSON, XML and other formats, the use is very useful , the following will also mention its benefits; note that with the Get method, the parameter is placed above the URL, to pass the non-English characters past, you need to encode the data, such as:

1. String url = "Http://www.xxx.xxx.com/xxx?name="
2. + Urlencoder.encode ("Xie Yu", "GBK") + "&othername=" + Urlencoder.encode ("Xie Yu",
3. "GBK");

Copy Code

One of the urlencoder uses Apache or JDK to bring.

By the way, write a post example:

2. try {
4. HttpClient client = new HttpClient ();//Define Client Object
8. Client.getparams (). Setparameter (Httpmethodparams.http_content_charset,
10. "GBK");//Specify the transfer character set to GBK format
14. Client.gethttpconnectionmanager (). Getparams (). Setconnectiontimeout (2000);//Set connection time-out to 2 seconds (connection initialization time)
18. Postmethod method = new Postmethod ("http://www.xxx.xxx.com/aaa/bbb.do");
22. Method.setrequestbody (New namevaluepair[]
24. {
28. New Namevaluepair ("name", "Xie Yu"), New Namevaluepair ("Othername", "Xie Yu")
30. });
32. int StatusCode = Client.executemethod (method);
34. if (statusCode! = HTTPSTATUS.SC_OK) {
38. SYSTEM.OUT.PRINTLN ("Remote access failed. ");
40. }
44. System.out.println (Method.getresponsebodyasstring ());//Output feedback Results
48. Client.gethttpconnectionmanager (). closeidleconnections (1);
50. }catch (...)
52. {.....}

Copy Code

As you can see, the example of post is similar to get, the only difference is that the method of passing in the parameter post takes the method of individually writing parameters, and get is above the URL.

If the above two examples are running on your machine, then good, we begin to discuss its security issues:

2, security how to control.

In fact, since the interface is a human definition, that is, the protocol is its own control, we are based on a lightweight programming model, so that the server-side and client code has become very simple, simple, security issues, who can call, if your interface is very open, then this is not a problem, just control the concurrency can be , but if there is a security risk, then there is a problem, this need to define good one encryption method, the granularity of encryption can be determined according to the actual situation, this transfer is best not to use simple encryption, generally there are two ways:

One: The use of irreversible encryption algorithm (such as: MD5, note MD5 in the Chinese data encryption, the use of different character set transcoding to encrypt the result is not the same), irreversible encryption algorithm, it is necessary to agree on the two sides of a protocol, in the parameters of the transfer of an encrypted token value, the rest of the parameters are still passed, Encryption process for the use of a key and the data itself is combined, and there are some dynamic changes, the encrypted data as a parameter to the receiver, the receiver uses the same method to get a ciphertext, two ciphertext comparison, if the comparison is consistent, the authentication is successful, if the contrast is inconsistent, the authentication fails This is a relatively simple method, some also use reversible and irreversible at the same time to use, first the data in accordance with the reversible algorithm encryption, and then calculate token, but more complex.

Second: Reversible encryption algorithm, but this reversible, need to have a key, the best non-symmetric key, and preferably the two sides of the key can change with a value, rather than a fixed key. Asymmetric key is more complex, if you want to use, this may be more troublesome, high security level can be considered, if your key itself can be a change in some way, the use of symmetry is basically enough, such as: Blowfish is still good, but no key is not used, Similar to the Base64 is too simple, of course you can also say, reversible and irreversible mixing to improve the level of security.

3, httpclient How to solve the problem under the call of high concurrency

The previous mention of the interaction between the HttpClient simulation system, if the interaction between the system is not high, is very easy to move, but HttpClient is the Web container as a Web request exists, under the HTTP protocol, is a stateless protocol, That is, connect-request-feedback-break a few basic actions, fortunately, now the Web container has keep-alive features, including a lot of load balancing devices: LB, LVS, Nginx, Apache, JBoss, Tomcat, etc. are supported, although support, But looking at the code above, it turns out that each request will be re-established, so how can they not create the connection again? Alternatively, do not create the connection repeatedly before the server is disconnected, a connection can be used multiple requests, not one request will be disconnected once, to establish a connection requires three handshake process, and more network overhead, so you understand.

The truth is very simple, in fact, and the link database, the above request client object and method object as a shared variable, the average efficiency will increase by about twice times, note that this is a cyclic test, rather than multithreading.

But for the higher concurrency, we can not use method only, because it can not be concurrent, so we have to use multiple, in multiple shared objects, if the control of expropriation, there is a connection pool problem, but this connection pool relative to the database connection pool is much simpler, because, retry action, Apache has been packaged for you, you just need to find and distribute the order, how to reduce the competition is the problem of algorithms and strategies.

However, let the client to write such a piece of code is not a bit too much, of course, you are willing to write is also possible, in fact, Apache provides us with a back is asynchronous httpclient (in fact, this is known as async is not the real asynchronous IO mode), that is, this part of the packaging, For the visitor is still synchronous, but at the IO level is non-blocking, this is in line with the server side of the keep-alive, as the server side of the same time to a site to request multiple resources, we want to be a connection, not multiple links, in fact, in many browsers (such as Chrome, FF) can monitor the server-side resources it requests at the same time, then how to implement asynchronous IO with HttpClient? In fact, it is quite simple, here is a simple example:

First you need to add a package for asynchronous IO:

1, async-http-client package, can be downloaded here: https://oss.sonatype.org/content ...-http-client/1.6.2/

2, Log4j's bag, this does not need me to say, all know where

3, Slf4j-spi of the package, currently with more than 1.5 version of more.

4, slf4j-log4j package, you can see, SLF4J is on the basis of log4j packaging.

OK, just these few, and then look at the following code, by using it, performance can be significantly improved:

1. Asynchttpclient client = new Asynchttpclient ();
2. try {
3. future<response> f = client.prepareget ("http://www.google.com.hk/"). Execute ();
5. System.out.println (F.get (). Getresponsebody ("Big5"));//Google output encoding set for BIG5, reverse parse the results when used
6. }catch (...)
7. {....}

Copy Code

This code is not super simple, can be described above in the three ways:

1. Call directly

2. Use GetMethod or Postmethod objects as shared objects repeatedly.

3. Using Asynchttpclient

With these three methods, you can test performance with one call, multiple loops, concurrent calls, and the performance of the latter is much higher than the performance of the first method.

HttpClient in Layman's ~