"StackOverflow problem Selection" How to use Java.net.URLConnection to send and receive HTTP requests

problemHow do I use java.net.URLConnection to send and receive HTTP requests? What are the best practices for handling HTTP requests?
discussion: Essence Answer

The first statement, the following code, is the basic example. If you are more rigorous, you should also include code that handles various exceptions (such as Ioexceptions, NullPointerException, arrayindexoutofboundsexception)
PrepareFirst, you need to set the URL of the request and CharSet (encoding), and which parameters are required, depending on the requirements of the respective URL.

String url = "http://example.com"; String charset = "UTF-8"; String param1 = "value1"; String param2 = "value2";//... String query = String.Format ("param1=%s¶m2=%s", Urlencoder.encode (param1, CharSet), Urlencoder.encode (Param2, CharSet ));

The request parameter must be in a format such as Name=value, with a & connection between each parameter. In general, you have to use Urlencoder#encode () to encode the parameters of the above example also used String#format (), this is just for convenience, I prefer to use this method to complete the concatenation of strings.
send an HTTP GET request (optional: with parameters)This is still a tedious thing to do. The default way is as follows:

URLConnection connection = new URL (url + "?" + query). OpenConnection (); Connection.setrequestproperty ("Accept-charset", charset); InputStream response = Connection.getinputstream ();//...

Between the URL and the parameter, to use? Number of connections. The Accept-charset in the header of the request that tells the server the encoding of the parameter you are sending. If you do not send any parameters, you can also ignore Accept-charset. If you don't need to set any headers, you can also use Url#openstream () instead of openconnection. In either case, assuming the server side is HttpServlet, then your GET request will trigger its doget () method, which can get the arguments you pass through Httpservletrequest#getparameter ().
send an HTTP POST request with parametersSetting Urlconnection#setdooutput () is an implicit way to set the request method to post. The standard HTTP POST form, whose content-tyep is application/x-www-form-urlencoded, puts the requested content into the body. This is the following code:

URLConnection connection = new URL (URL). OpenConnection (); Connection.setdooutput (true); Triggers POST.connection.setRequestProperty ("Accept-charset", Charset); Connection.setrequestproperty (" Content-type "," application/x-www-form-urlencoded;charset= "+ charset); try (outputstream output = Connection.getoutputstream ()) {    output.write (Query.getbytes (CharSet));} InputStream response = Connection.getinputstream ();//...

Reminder: When you want to submit an HTML form, be sure to put <input type= "hidden" the value of such elements, in the form of Name=value also submitted. In addition, there are <input type= "submit" > such elements, as well. Because, typically, the server also needs this information to confirm which button triggered the commit action.
You can also use httpurlconnection instead of URLConnection, and then call Httpurlconnection#setrequestmethod () to set the request to the post type.

HttpURLConnection httpconnection = (httpurlconnection) new URL (URL). OpenConnection (); Httpconnection.setrequestmethod ("POST");//...

Similarly, if the server is HttpServlet, it will trigger its dopost () method, which can get the post parameters via Httpservletrequest#getparameter ().
really triggering the sending of an HTTP requestYou can explicitly send requests through urlconnection#connect (), but when you call the method that gets the response information, the request is automatically sent. For example, when you use Urlconnection#getinputstream (), the request is triggered automatically, so you don't have to call the Connect () method one more time. In my example above, it is also a direct call to the getInputStream () method.
Get HTTP response information1) HTTP response code: First default you use HttpURLConnection

int status = Httpconnection.getresponsecode ();

2) HTTP response Header (headers)

</pre><pre name= "code" class= "Java" >for (entry<string, list<string>> header: Connection.getheaderfields (). EntrySet ()) {    System.out.println (Header.getkey () + "=" + Header.getvalue ());}

3) HTTP response encoding: When the charset parameter is included in the Content-type, the response content is based on the encoding specified by the charset parameter. Therefore, when decoding the response information, also follow this encoding format.

String ContentType = Connection.getheaderfield ("Content-type"); String charset = Null;for (String Param:contentType.replace ("", ""). Split (";")) {    if (Param.startswith ("charset=")) {        charset = param.split ("=", 2) [1];        break;    }} if (charset! = null) {    try (BufferedReader reader = new BufferedReader (new InputStreamReader (response, CharSet))) {
   for (String line; (line = Reader.readline ()) = null;) {            // ... System.out.println (Line)}}}    else {    //It ' s likely binary content, use Inputstream/outputstream.}

maintenance of the sessionServer session, usually based on cookies. You can manage cookies through COOKIEHANDLERAPI. Initialize a cookiemanager before sending the HTTP request, and then set the parameter to Cookiepolicy.accept_all.

First set the default cookie manager. Cookiehandler.setdefault (new Cookiemanager (null, Cookiepolicy.accept_all));//all the following subsequent Urlconnections would use the same cookie manager. URLConnection connection = new URL (URL). OpenConnection ();//... connection = new URL (URL). OpenConnection ();//... connection = new URL (URL). OpenConnection ();//...

Note that this method is not available for all scenarios. If you fail in this way, you can try to set a cookie yourself: you need to get the Set-cookie parameter from the response header, and then set the cookie to the next request.

Gather All cookies are on the first request. URLConnection connection = new URL (URL). OpenConnection (); list<string> cookies = Connection.getheaderfields (). Get ("Set-cookie"),//...//Then use the same cookie on all subs Equent requests.connection = new URL (URL). OpenConnection (); for (String cookie:cookies) {    Connection.addrequestproperty ("Cookie", Cookie.split (";", 2) [0]);} // ...

The above split (";", 2) [0], is to remove some cookie information unrelated to the server (such as Expores,path, etc.). Also available cookie.substring (0, Cookie.indexof (';')) Achieve the same goal
Processing of StreamsRegardless of whether you set a fixed length for Content through Connection.setrequestproperty ("Content-length", ContentLength), httpurlconnection before sending a request, The body of the entire request is cached by default. If you send a larger post request (such as uploading a file), it may cause outofmemoryexception. To avoid this problem, you can set the Httpurlconnection#setfixedlengthstreamingmode ()

Httpconnection.setfixedlengthstreamingmode (contentlength);

However, if the content length is unknown, you can use Httpurlconnection#setchunkedstreamingmode (). In this way, the header transfer-encoding will become chunked, your request will be chunked sent, for example, the following example, the request body, will be 1KB block, chunked send

Httpconnection.setchunkedstreamingmode (1024);

user-agentSometimes, the request you send may only be returned in a browser, but not in other ways. This may be related to the user-agent in the request header. The user-agent information sent by URLConnection is java/1.6.0_19, which is the version of Java+jre, by default. You can rewrite this message:

Connection.setrequestproperty ("User-agent", "mozilla/5.0" (Windows; U Windows NT 5.1; En-us; rv:1.9.2.3) gecko/20100401 "); Do as if you ' re using Firefox 3.6.3.

Here's a more complete list of browser user-agent
Error HandlingIf the HTTP response code is 4xx (client exception) or 5xx (server-side exception), you can obtain the information through Httpurlconnection#geterrorstream (), the server may put some useful error information in this area.

InputStream error = ((httpurlconnection) connection). Geterrorstream ();

If The HTTP response code is-1, then something went wrong with connection and response handling, did not see the meaning of this passage. The httpurlconnection implementation is somewhat buggy with keeping connections alive. Want to turn it off by setting the Http.keepalive system property to False. You can do this programmatically in the beginning of your application By:System.setProperty ("Http.keepalive", "false");
Uploading FilesIn general, you will need to set the post content to Multipart/form-data (the relevant RFC document: RFC2388)

String param = "value"; File Textfile = new file ("/path/to/file.txt"); File Binaryfile = new file ("/path/to/file.bin"); String boundary = long.tohexstring (System.currenttimemillis ()); Just generate some unique random value. String CRLF = "\ r \ n"; Line separator required by Multipart/form-data. URLConnection connection = new URL (URL). OpenConnection (); Connection.setdooutput (true); Connection.setrequestproperty ("Content-type", "multipart/form-data;    boundary= "+ boundary); try (outputstream output = Connection.getoutputstream (); PrintWriter writer = new PrintWriter (new OutputStreamWriter (output, CharSet), true);)    {//Send normal param.    Writer.append ("--" + boundary). Append (CRLF); Writer.append ("Content-disposition:form-data;    Name=\ "Param\" "). Append (CRLF); Writer.append ("Content-type:text/plain;    charset= "+ charset). Append (CRLF);    Writer.append (CRLF). Append (param). Append (CRLF). Flush ();    Send text file.    Writer.append ("--" + boundary). Append (CRLF); Writer.append ("content-disposition:form-data; Name=\ "Textfile\";    Filename=\ "" + textfile.getname () + "\" "). Append (CRLF); Writer.append ("Content-type:text/plain; charset= "+ charset). Append (CRLF);    Text file itself must is saved in this charset!    Writer.append (CRLF). Flush ();    Files.copy (Textfile.topath (), output); Output.flush ();    Important before continuing with writer! Writer.append (CRLF). Flush (); CRLF is important!    It indicates end of boundary.    Send binary file.    Writer.append ("--" + boundary). Append (CRLF); Writer.append ("Content-disposition:form-data; Name=\ "Binaryfile\";    Filename=\ "" + binaryfile.getname () + "\" "). Append (CRLF);    Writer.append ("Content-type:" + urlconnection.guesscontenttypefromname (Binaryfile.getname ())). append (CRLF);    Writer.append ("Content-transfer-encoding:binary"). Append (CRLF);    Writer.append (CRLF). Flush ();    Files.copy (Binaryfile.topath (), output); Output.flush ();    Important before continuing with writer! Writer.append (CRLF). Flush (); CRLF is important!    It indicates end of boundary.    End of Multipart/form-data. Writer.append ("--" + Boundary + "--"). Append (CRLF). Flush ();}

Assuming that the server is still a httpservlet, its dopost () method will handle the request, and the server gets the content you sent through Httpservletrequest#getpart () (note, not getparameter ()). GetPart () is a relatively new method that was introduced after Servlet 3.0. If you are a version prior to Servlet 3.0, you can use Apache Commons fileupload to parse the multipart/form-data request. You can refer to the example here
Last WordsWith a lot of verbose, Apache provides a toolkit that helps us to do these things more conveniently Apache httpcomponents HttpClient:

• HttpClient Tutorial
• HttpClient Examples

Google also has a similar toolkit
parsing and extracting HTML contentIf you want to parse the contents of the extracted HTML, you can use Jsoup and other parsers

• The pros and cons of some of the more famous HTML parsers
• How to scan and parse Web pages in Java

StackOverflow:Http://stackoverflow.com/questions/2793150/using-java-net-urlconnection-to-fire-and-handle-http-requests


Column Introduction: very like StackOverflow, can always find a solution to the problem of incurable diseases. Accidentally found that the site has a list of heat. So select some of the higher heat problems, and then according to their own understanding, the discussion of the people to comb out. Therefore, these articles are not true translation, but in accordance with their own understanding of a number of additions and deletions, polishing, hoping to put the above discussion, more streamlined and effective sharing to everyone. if you want to reprint, please specify the original addressHttp://blog.csdn.net/lizeyang

"StackOverflow problem Selection" How to use Java.net.URLConnection to send and receive HTTP requests