Java implementation simulates browser access to Web site

In general, we use IE or navigator browser to access a Web server, to browse the page to view information or submit some data and so on. Some of the pages visited are just plain pages, some require users to log in to use them, or require authentication and are transmitted in encrypted form, such as HTTPS. The browsers we use today are not going to pose a problem with these situations. However, you may need to access some of these pages at some point, such as "stealing" some data from someone else's Web page, or using pages provided by certain sites to do something, such as saying we want to know where a cell phone number belongs and we don't have that data ourselves. So we have to use other companies already have the website to complete this function, this time we need to submit a mobile phone number to the page and from the returned page to resolve the data we want. If the other side is just a simple page, then our program will be very simple, this article there is no need to make a big waste of breath here. However, given some service licensing issues, many companies provide pages that are often not accessible through a simple URL, but must be registered and then logged in to use the service provided page, this time involves the processing of the cookie problem. We know that the current popular dynamic web technologies such as ASP, JSP are not all through the cookie processing session information. In order for our program to use the service page provided by others, we ask the program to log in and then visit the service page, this process will need to process the cookie itself, think about how horrible it is when you use java.net.HttpURLConnection to accomplish these functions! And it's just a very common "stubborn" in what we call a stubborn Web server! What about uploading files over HTTP? Don't need a headache, these problems with "it" will be easy to solve!

We cannot enumerate all possible stubbornness, and we will deal with several of the most common problems. Of course, as mentioned earlier, if we use the java.net.HttpURLConnection to solve these problems is very scary, so before we begin to introduce an open source project, This project is the httpclient in the Apache Open source organization, which belongs to the Jakarta Commons project, and the current version is 2.0RC2. Commons already has a net sub-project, but put httpclient alone, it can be seen that HTTP server access is not easy.

The Commons-httpclient project is specifically designed to simplify the communication programming of HTTP clients and servers. Through it can let the original very troublesome things now easy to solve, such as you no longer control HTTP or HTTPS communication mode, tell it you want to use HTTPS, the rest of the matter to httpclient for you to complete. This article describes how to use httpclient to solve a number of problems that we often encounter when writing HTTP client programs, so that readers can familiarize themselves with the project more quickly we start by giving a simple example to read the contents of a Web page. Then step through the steps to solve all the problems ahead.

1. Read Web page (HTTP/HTTPS) content
Here is a simple example we give to access a page

 PackageHttp.demo;Importjava.io.IOException;Importorg.apache.commons.httpclient.*;Importorg.apache.commons.httpclient.methods.*; Public classSimpleClient { Public Static voidMain (string[] args)throwsIOException {HttpClient client=NewHttpClient (); //setting the proxy server address and port//client.gethostconfiguration (). SetProxy ("Proxy_host_addr", Proxy_port); //using the Get method, if the server needs to be connected over HTTPS, you only need to change the HTTP in the URL below to HTTPSHttpMethod method =NewGetMethod ("http://java.sun.com";); //using the Post method//HttpMethod method = new Postmethod ("http://java.sun.com";); Client.executemethod (method); //status returned by the print serverSystem.out.println (Method.getstatusline ()); //Print the returned informationSystem.out.println (method.getresponsebodyasstring ()); //Release Connectionmethod.releaseconnection (); }} 

In this example, you first create an instance of the HTTP client (HttpClient), and then select the method to commit is get or post, and finally execute the commit on the HttpClient instance, and finally read the results of the server feedback from the selected submission method. This is the basic process of using httpclient. In fact, with a line of code can also handle the entire request process, very simple!

2. Submit parameters to a Web page in Get or post
In fact, in one of the simplest examples we have described how to use GET or post to request a page, this section differs from the set of parameters required for setting the page at commit, and we know that if it is a GET request, then all parameters are placed directly on the page The URL is followed by a question mark and the page address, each parameter is separated by &, for example: http://java.sun.com/?name=liudong&mobile=123456, but when using the Post method, a little bit of trouble. The example in this section shows how to query the city where the phone number is located, with the following code:

 PackageHttp.demo;Importjava.io.IOException;Importorg.apache.commons.httpclient.*; Importorg.apache.commons.httpclient.methods.*;  Public classsimplehttpclient { Public Static voidMain (string[] args)throwsIOException {HttpClient client=NewHttpClient (); Client.gethostconfiguration (). Sethost ("www.imobile.com.cn", "http"); HttpMethod Method= Getpostmethod ();//submit data using PostClient.executemethod (method); //status returned by the print serverSystem.out.println (Method.getstatusline ()); //Print results pageString response =NewString (Method.getresponsebodyasstring (). GetBytes ("8859_1")); //Print the returned informationSystem.out.println (response);     Method.releaseconnection (); }          Private StaticHttpMethod Getgetmethod () {return NewGetMethod ("/simcard.php?simcard=1330227"); }          Private StaticHttpMethod Getpostmethod () {Postmethod post=NewPostmethod ("/simcard.php"); Namevaluepair Simcard=NewNamevaluepair ("Simcard", "1330227"); Post.setrequestbody (Newnamevaluepair[] {simcard}); returnPost; } } 

In the above example page http://www.imobile.com.cn/simcard.php need a parameter is Simcard, this parameter value is the cell phone number segment, that is, the first seven digits of the mobile phone number, the server will return the submitted mobile phone number corresponding to the province, City as well as other detailed information. The Get submission method only needs to add parameter information after the URL, and post requires the Namevaluepair class to set the parameter name and its corresponding value

https://www.ibm.com/developerworks/cn/opensource/os-httpclient/Reference Additional Articles

Java implementation simulates browser access to Web site