Httpclient getting started

In general, we use the IE or Navigator browser to access a Web server, which is used to browse the page to view information or submit
Some data. The accessed pages are only some common pages, which can be used only after users log on, or require authentication and encrypted transmission, such as HTTPS. Currently
The browser we use will not solve these problems. However, you may need to access such pages through programs in some cases, such as "Stealing" some data from others' webpages and using some websites.
For example, if we want to know the location of a mobile phone number and we do not have such data, we have to use the existing websites of other companies to complete this function, this time
We need to submit the mobile phone number to the webpage and parse the desired data from the returned page. If the other side is just a very simple page, our program will be very simple and there is no need to make a big splash in this article.
Here is a waste of tongue. However, considering some service authorization issues, the pages provided by many companies may not be accessible through a simple URL, but must be registered and then logged on.
Use the page that provides services, which involves cookie processing. We know that currently popular dynamic web technologies such as ASP and JSP do not process session information through cookies.
. In order for our programs to use the service pages provided by others, the program is required to log on to the service page first, and then access the service page. In this process, you need to process the cookies on your own.
How terrible is java.net. httpurlconnection to accomplish these functions! Besides, this is just a common feature of the stubborn web server.
"Stubborn "! Is it like uploading files over HTTP? There is no headache. It is easy to solve these problems with "it!

We cannot list all possible stubbornness. We will deal with several of the most common problems. Of course, as mentioned earlier, if we use
Java.net. httpurlconnection is terrible to solve these problems. Therefore, before we start, we should first introduce an open source project, which is
Httpclient in the Apache open-source organization, which belongs to the Jakarta Commons project. The current version is 2.0rc2. There is already one under commons
Net sub-projects, but the httpclient was separately proposed, it is obvious that access to the HTTP server is not easy.

The commons-httpclient project is specially designed to simplify communication programming between HTTP clients and servers. It can be used
This makes it easy to solve the headache. For example, if you no longer care about HTTP or HTTPS communication, tell it that you want to use https, and hand over the rest
Httpclient is complete for you. This article will introduce how to use httpclient to solve the problems we often encounter when writing HTTP client programs.
Let readers get familiar with this project more quickly. At first, we will give a simple example to read the content of a web page, and then gradually solve the problem of moving forward? /Font>

1. Read webpage (HTTP/https) Content

The following is a simple example to access a page.

/*

* Created on 2003-12-14 by liudong

*/

Package HTTP. Demo;

Import java. Io. ioexception;

Import org. Apache. commons. httpclient .*;

Import org. Apache. commons. httpclient. Methods .*;

/**

* The simplest HTTP client is used to demonstrate how to access a page through get or post.

* @ Author liudong

*/

Public class simpleclient {

Public static void main (string [] ARGs) throws ioexception

{

Httpclient client = new httpclient ();

// Set the proxy server address and port

// Client. gethostconfiguration (). setproxy ("proxy_host_addr", proxy_port );

// Use the get method. If the server needs to connect over https, you only need to replace the HTTP in the following URL with HTTPS

Httpmethod method = new getmethod ("http://java.sun.com ");

// Use the post Method

// Httpmethod method = new postmethod ("http://java.sun.com ");

Client.exe cutemethod (method );

// Print the status returned by the server

System. Out. println (method. getstatusline ());

// Print the returned information

System. Out. println (method. getresponsebodyasstring ());

// Release the connection

Method. releaseconnection ();

}
}

In this example, first create an HTTP client (httpclient) instance, and then select get or
Post, execute the submit method on the httpclient instance, and finally read the results returned by the server from the selected commit method. This is the basics of using httpclient.
Process. In fact, using a single line of code can handle the entire request process, which is very simple!

2. Submit parameters to the webpage in get or post Mode

In fact, in the previous simplest example, we have introduced how to use get or post to request a page. This section is different from this section.
Set the parameters required for the page when submitting the request. We know that if it is a GET request method, all parameters will be placed directly behind the URL of the page and separated by question marks and page addresses. Each parameter is separated,
Example: http://java.sun.com? Name = liudong & mobile = 123456, but it may be a little bit difficult to use the post method. The example in this section shows how to query the city where the mobile phone number is located. The Code is as follows:

/*

* Created on 2003-12-7 by liudong

*/

Package HTTP. Demo;

Import java. Io. ioexception;

Import org. Apache. commons. httpclient .*;

Import org. Apache. commons. httpclient. Methods .*;

/**

* Parameter submission demo

* This program is connected to a page used to query the location of the mobile phone number.

* Query the province and city where code segment 1330227 is located.

* @ Author liudong

*/

Public class simplehttpclient {

Public static void main (string [] ARGs) throws ioexception

{

Httpclient client = new httpclient ();

Client. gethostconfiguration (). sethost ("www.imobile.com.cn", 80, "HTTP ");

Httpmethod method = getpostmethod (); // use post to submit data

Client.exe cutemethod (method );

// Print the status returned by the server

System. Out. println (method. getstatusline ());

// Print the result page

String response =

New String (method. getresponsebodyasstring (). getbytes ("8859_1 "));

// Print the returned information

System. Out. println (response );

Method. releaseconnection ();

}

/**

* Use get to submit data

* @ Return

*/

Private Static httpmethod getgetmethod (){

Return new getmethod ("/simcard. php? Simcard = 1330227 ");

}

/**

* Use post to submit data

* @ Return

*/

Private Static httpmethod getpostmethod (){

Postmethod post = new postmethod ("/simcard. php ");

Namevaluepair simcard = new namevaluepair ("simcard", "1330227 ");

Post. setrequestbody (New namevaluepair [] {simcard });

Return post;

}

}

In the preceding example. The get method only needs to add parameter information after the URL, while the POST method needs to set the parameter name and its corresponding value through the namevaluepair class.

3. Process page redirection

In JSP/servlet programming, the response. sendredirect method is to use the redirection server in the HTTP protocol.
. It works with <JSP: Forward
...> The difference is that the latter implements page Jump on the server, that is, the application container loads the content of the page to be redirected and returns it to the client. The former returns a status code, these Status Codes
The possible values are shown in the following table. Then, the client reads the URL of the page to jump to and reloads the new page. It is such a process, so we need to pass
The httpmethod. getstatuscode () method determines whether the returned value is a value in the following table to determine whether a jump is required. If you have confirmed that you need to jump to the page, you can
To obtain the new address by reading the location attribute in the HTTP header.

Status Code
Constant corresponding to httpservletresponse
Detailed description

301
SC _moved_permanently
The page has been permanently moved to another address.

302
SC _moved_temporarily
The page is temporarily moved to another address.

303
SC _see_other
The client request address must be accessed through another URL

307
SC _temporary_redirect
Same as SC _moved_temporarily

The following code snippet demonstrates how to handle page redirection.

Client.exe cutemethod (post );

System. Out. println (post. getstatusline (). tostring ());

Post. releaseconnection ();

// Check for redirection

Int statuscode = post. getstatuscode ();

If (statuscode = httpstatus. SC _moved_temporarily) |

(Statuscode = httpstatus. SC _moved_permanently) |

(Statuscode = httpstatus. SC _see_other) |

(Statuscode = httpstatus. SC _temporary_redirect )){

// Read the new URL

Header header = post. getResponseHeader ("location ");

If (header! = NULL ){

String newuri = header. getvalue ();

If (newuri = NULL) | (newuri. Equals ("")))

Newuri = "/";

Getmethod redirect = new getmethod (newuri );

Client.exe cutemethod (redirect );

System. Out. println ("Redirect:" + redirect. getstatusline (). tostring ());

Redirect. releaseconnection ();

} Else

System. Out. println ("invalid redirect ");

}

We can write two JSP pages by ourselves, one of which is redirected to another by using the response. sendredirect method to test the above example.

4. Enter the user name and password to log on.

This section should be said to be the most common problem encountered in HTTP client programming. The content of many websites is only visible to registered users. In this case, the requirements must be met.
You can view the desired page only after successfully logging on with the correct username and password. Because the HTTP protocol is stateless, that is, the connection validity period is limited to the current request, and the connection is closed after the request content ends.
. In this case, the cookie mechanism must be used to save user login information. Take JSP/Servlet as an example. When a browser requests a JSP or servlet page
The application server returns a parameter named JSESSIONID (varies with application servers). The value is a long and unique string cookie, which is the current access
Ask the session ID of the site. Each time the browser accesses other pages of the site, it must carry cookie information such as JSESSIONID. The application server obtains
Session information.

For websites that require user login, user information is generally stored in the session of the server after the user login is successful. In this way, when other pages are accessed
The server reads the session ID of the current request based on the cookie sent by the browser to obtain the corresponding session information. Then, it can determine whether the user information exists in the session information.
Access the page. Otherwise, you must enter your account and password to log on to the logon page. This is a common way to use JSP to develop websites to process user logon.

In this way, for the HTTP client, if you want to access a protected page, you must simulate the work done by the browser. The first step is to request the login
Page, and then read the cookie value. Request the logon page again and add each parameter required for the logon page. Finally, the page that is ultimately required for the request. Of course, all requests except the first request must be included
Cookie information so that the server can determine whether the current request has been verified. But if you use httpclient, you don't even need to add a line of code.
You must first pass the login information to execute the login process, and then directly access the desired page, which is no different from accessing a common page, because the httpclient class has done everything you should do,
Great! The following example implements such an access process.

/*

* Created on 2003-12-7 by liudong

*/

Package HTTP. Demo;

Import org. Apache. commons. httpclient .*;

Import org. Apache. commons. httpclient. Cookie .*;

Import org. Apache. commons. httpclient. Methods .*;

/**

* Shows an example of a logon form.

* @ Author liudong

*/

Public class formlogindemo {

Static final string logon_site = "localhost ";

Static final int maid = 8080;

Public static void main (string [] ARGs) throws exception {

Httpclient client = new httpclient ();

Client. gethostconfiguration (). sethost (logon_site, logon_port );

// Simulate the login. jsp-> main. jsp

Postmethod post = new postmethod ("/Main. jsp ");

Namevaluepair name = new namevaluepair ("name", "LD ");

Namevaluepair pass = new namevaluepair ("password", "LD ");

Post. setrequestbody (New namevaluepair [] {name, pass });

Int status = client.exe cutemethod (post );

System. Out. println (post. getresponsebodyasstring ());

Post. releaseconnection ();

// View Cookie Information

Cookiespec = cookiepolicy. getdefaspec SPEC ();

Cookie [] cookies = cookiespec. Match (logon_site, logon_port, "/", false, client. getstate (). getcookies ());

If (cookies. Length = 0 ){

System. Out. println ("NONE ");

} Else {

For (INT I = 0; I <cookies. length; I ++ ){

System. Out. println (Cookies [I]. tostring ());

}

}

// Access the required page main2.jsp

Getmethod get = new getmethod ("/main2.jsp ");

Client.exe cutemethod (get );

System. Out. println (get. getresponsebodyasstring ());

Get. releaseconnection ();

}

}

5. Submit XML format parameters

The parameter for submitting XML format is very simple. It is just a contenttype problem during submission. The following example shows how to read XML Information from a file and submit it to the server, this process can be used to test web services.

Import java. Io. file;

Import java. Io. fileinputstream;

Import org. Apache. commons. httpclient. httpclient;

Import org. Apache. commons. httpclient. Methods. entityenclosingmethod;

Import org. Apache. commons. httpclient. Methods. postmethod;

/**

* Used to demonstrate the example of submitting XML format data

*/

Public class postxmlclient {

Public static void main (string [] ARGs) throws exception {

File input = new file ("test. xml ");

Postmethod post = new postmethod ("http: // localhost: 8080/httpclient/XML. jsp ");

// Set the request content to be directly read from the file

Post. setrequestbody (New fileinputstream (input ));

If (input. Length () <integer. max_value)

Post. setrequestcontentlength (input. Length ());

Else post. setrequestcontentlength (entityenclosingmethod. content_length_chunked );

// Specify the request content type

Post. setRequestHeader ("Content-Type", "text/XML; charset = GBK ");

Httpclient = new httpclient ();

Int result = httpclient.exe cutemethod (post );

System. Out. println ("response status code:" + result );

System. Out. println ("response body :");

System. Out. println (post. getresponsebodyasstring ());

Post. releaseconnection ();

}

}

6. upload files over HTTP

Httpclient uses a separate httpmethod subclass to process file uploads. This class is multipartpostmethod, which has encapsulated the file upload details, all we need to do is to tell it that we want to upload the full path of the file. The following code snippet demonstrates how to use this class.

Multipartpostmethod filepost = new multipartpostmethod (TargetUrl );

Filepost. addparameter ("FILENAME", targetfilepath );

Httpclient client = new httpclient ();

// Because the file to be uploaded may be large, set the maximum connection timeout here

Client. gethttpconnectionmanager (). getparams (). setconnectiontimeout (5000 );

Int status = client.exe cutemethod (filepost );

In the code above, targetfilepath is the path of the file to be uploaded.

7. Access the authentication enabled page

We often encounter such a page. when accessing it, a dialog box in the browser is displayed asking you to enter the user name and password.
Method is different from the form-based user authentication described earlier. This is an HTTP Authentication Policy. httpclient supports three authentication methods: basic, summary, and NTLM authentication. Its
Basic authentication is the simplest, generic, but insecure. Digest Authentication
The authentication method added in 1.1, while NTLM is defined by Microsoft rather than general specifications. The latest version of NTLM is more secure than digest authentication.

The following example shows how to access a page protected by authentication:

Import org. Apache. commons. httpclient. httpclient;

Import org. Apache. commons. httpclient. usernamepasswordcredentials;

Import org. Apache. commons. httpclient. Methods. getmethod;

Public class basicauthenticationexample {

Public basicauthenticationexample (){

}

Public static void main (string [] ARGs) throws exception {

Httpclient client = new httpclient ();

Client. getstate (). setcredentials (

"Www.verisign.com ",

"Realm ",

New usernamepasswordcredentials ("username", "password ")

);

Getmethod get = new getmethod ("https://www.verisign.com/products/index.html ");

Get. setdoauthentication (true );

Int status = client.exe cutemethod (get );

System. Out. println (status + "" + Get. getresponsebodyasstring ());

Get. releaseconnection ();

}

}

8. Use httpclient in multi-threaded Mode

Multiple Threads simultaneously access httpclient. For example, multiple files can be downloaded from one site. For the same httpconnection
At the same time, only one thread can be accessed. To ensure that there is no conflict in the multi-threaded working environment, httpclient uses a multi-threaded connection manager
Class: multithreadedhttpconnectionmanager. To use this class, you only need to input
Yes. The Code is as follows:

Multithreadedhttpconnectionmanager connectionmanager =

New multithreadedhttpconnectionmanager ();

Httpclient client = new httpclient (connectionmanager );

You can access the client instance in the future.