17.2.3 use URL and urlconnection The URL (Uniform Resource Locator) object represents a unified resource locator, which is a pointer to the Internet "resource. A resource can be a simple file or directory, or a reference to a more complex object, such as a query of a database or search engine. Generally, a URL consists of a protocol name, host, port, and resource. The format is as follows:
Protocol: // host: Port/resourcename |
For example, the following URL:
Http://www.oneedu.cn/Index.htm |
JDK also provides a URI (Uniform resource identifiers) class. Its instance represents a Uniform Resource Identifier. Java URI cannot be used to locate any resource. Its only function is to parse it. The URL contains an input stream that can be opened to the resource. Therefore, we can understand the URL as a special case of Uri. The URL class provides multiple constructors for creating URL objects. Once a URL object is obtained, you can call the following method to access the resources corresponding to the URL: String GetFile (): gets the Resource Name of this URL.
String gethost (): Get the host name of this URL.
String getpath (): Obtain the path of this URL.
Int getport (): gets the port number of this URL.
String getprotocol (): gets the protocol name of this URL.
String getquery (): Obtain the query string of this URL.
Urlconnection openconnection (): returns a urlconnection object, which indicates the connection to the remote object referenced by the URL.
Inputstream openstream (): Open the connection to this URL and return an inputstream used to read the URL resource. The previous methods in the URL object are easy to understand, and the openstream () provided by the object can read the inputstream of the URL resource, by using this method, you can easily read remote resources-or even download multiple threads. As follows:ProgramAs shown in: Program list: codes/17/17-2/mutildown. Java
// Define the thread for downloading content from start to end
Class downthread extends thread
{
// Define the length of the byte array (the bamboo for taking water)
Private Final int buff_len = 32;
// Define the download start point
Private long start;
// Define the download end point
Private long end;
// Download the input stream corresponding to the Resource
Private inputstream is;
// Output the downloaded bytes to Raf.
Private randomaccessfile Raf;
// Constructor: input stream, output stream, download start point, and end point
Public downthread (long start, long end
, Inputstream is, randomaccessfile RAF)
{
// Output the location of the bytes that the thread is responsible for downloading
System. Out. println (start + "---->" + end );
This. Start = start;
This. End = end;
This. Is = is;
This. RAF = Raf;
}
Public void run ()
{
Try
{
Is. Skip (start );
Raf. Seek (start );
// Define the cache array for reading the content of the input stream (bamboo)
Byte [] buff = new byte [buff_len];
// This thread is responsible for downloading the resource size
Long contentlen = end-start;
// Defines the maximum number of reads required to download this thread.
Long times = contentlen/buff_len + 4;
// The actual number of bytes read
Int hasread = 0;
For (INT I = 0; I <times; I ++)
{
Hasread = is. Read (buff );
// Exit the loop if the number of bytes read is smaller than 0!
If (hasread <0)
{
Break;
}
Raf. Write (buff, 0, hasread );
}
}
Catch (exception ex)
{
Ex. printstacktrace ();
}
// Use finally blocks to close the input and output streams of the current thread
Finally
{
Try
{
If (is! = NULL)
{
Is. Close ();
}
If (RAF! = NULL)
{
Raf. Close ();
}
}
Catch (exception ex)
{
Ex. printstacktrace ();
}
}
}
}
Public class mutildown
{
Public static void main (string [] ARGs)
{
Final int down_thread_num = 4;
Final string out_file_name = "down.jpg ";
Inputstream [] isarr = new inputstream [down_thread_num];
Randomaccessfile [] outarr = new randomaccessfile [down_thread_num];
Try
{
// Create a URL object
URL url = new URL ("http://images.china-pub.com/
+ "Ebook35001-40000/35850/shupi.jpg ");
// Open the first input stream with this URL object
Isarr [0] = URL. openstream ();
Long filelen = getfilelength (URL );
System. Out. println ("network resource size" + filelen );
// Create the first randomaccessfile output stream with the output file name
Outarr [0] = new randomaccessfile (out_file_name, "RW ");
// Create an empty file of the same size as the downloaded Resource
For (INT I = 0; I <filelen; I ++)
{
Outarr [0]. Write (0 );
}
// The number of bytes to be downloaded per Thread
Long numperthred = filelen/down_thread_num;
// Remaining remainder of the entire downloaded Resource
Long left = filelen % down_thread_num;
For (INT I = 0; I <down_thread_num; I ++)
{
// Open an input stream and a randomaccessfile object for each thread,
// Each thread is responsible for downloading different parts of the resource.
If (I! = 0)
{
// Open multiple input streams with URLs
Isarr [I] = URL. openstream ();
// Create multiple randomaccessfile objects with the specified output file
Outarr [I] = new randomaccessfile (out_file_name, "RW ");
}
// Start multiple threads to download network resources.
If (I = down_thread_num-1)
{
// The last thread downloads the specified numperthred + Left byte
New downthread (I * numperthred, (I + 1) * numperthred + left
, Isarr [I], outarr [I]). Start ();
}
Else
{
// Each thread is responsible for downloading certain numperthred bytes
New downthread (I * numperthred, (I + 1) * numperthred,
Isarr [I], outarr [I]). Start ();
}
}
}
Catch (exception ex)
{
Ex. printstacktrace ();
}
}
// Define a method to obtain the length of a specified network resource
// Define a method to obtain the length of a specified network resource
Public static long getfilelength (URL) throws exception
{
Long length = 0;
// Open the urlconnection corresponding to the URL.
Urlconnection con = URL. openconnection ();
// Obtain the length of the URL resource.
Long size = con. getcontentlength ();
Length = size;
Return length;
}
} |
The above program defines the downthread class, which reads all the bytes from start to end from inputstream and writes them to the randomaccessfile object. The run () of this downthread class is a simple input and output implementation. The main method in the mutildown class in the program is responsible for implementing multi-threaded download by following the steps below: Create a URL object. Obtains the size of the resource to which the specified URL object points (implemented by the getfilelength method). The urlconnection class is used here. This class represents the communication link between the Java application and the URL. The following is a more detailed introduction to urlconnection. Create an empty file of the same size as the network resource on the local disk. Calculate the part of the network resource that each thread should download (starting from the byte and ending with the byte ). Create and start multiple threads in sequence to download specified parts of network resources. The above program has implemented the core of multi-thread downloadCodeTo implement resumable download, you also need to add a configuration file (you can find that all the resumable download tools will generate two files starting from download: one is an empty file of the same size as the network resources, and the other is a configuration file). The configuration file records the bytes that each thread has downloaded. When the network is disconnected, the download starts again, each thread can be downloaded backward Based on the recorded position in the configuration file. The openconnection () method of the URL returns a urlconnection object, which indicates the communication link between the application and the URL. The program can send a request to the URL through the urlconnection instance to read the resources referenced by the URL. Generally, the following steps are required to create a connection with a URL, send a request, and read resources referenced by the URL: Create a urlconnection object by calling the openconnection () method of the URL object. Set urlconnection parameters and common request attributes. If you only send a GET request, use the connect method to establish the actual connection with the remote resource. If you need to send a POST request, you need to obtain the output stream corresponding to the urlconnection instance to send request parameters. The remote resource becomes available. The program can access the header field of the Remote resource or read the data of the Remote resource through the input stream. Before establishing an actual connection to a remote resource, the program can set the request header field as follows: Setallowuserinteraction: set the value of the allowuserinteraction request header field of the urlconnection.
Setdoinput: set the value of the doinput request header field of the urlconnection.
Setdooutput: set the value of the dooutput request header field of the urlconnection.
Setifmodifiedsince: set the value of the ifmodifiedsince request header field of the urlconnection.
Setusecaches: set the value of the usecaches request header field of the urlconnection. In addition, you can use the following method to set or add a common header field: Setrequestproperty (string key, string value): set the value of the key request header field of the urlconnection to value. The following code is used:
Conn. setrequestproperty ("accept ","*/*") |
Addrequestproperty (string key, string value): adds a value to the key request header field of the urlconnection. This method does not overwrite the value of the original request header field, the new value is appended to the original request header field. When remote resources are available, the program can use the following methods to access header fields and content: Object getcontent (): Get the content of the urlconnection.
String getheaderfield (string name): obtains the value of the specified response header field.
Getinputstream (): returns the input stream corresponding to the urlconnection to obtain the content of the urlconnection response.
Getoutputstream (): returns the output stream corresponding to the urlconnection, which is used to send request parameters to urlconnection. If you want to use the input stream to read the urlconnection response content and the output stream to send request parameters, you must use the output stream before using the input stream. The getheaderfield method is used to return the corresponding value based on the response header field. Some header fields are frequently accessed, so Java provides the following methods to access the values of specific response header fields: Getcontentencoding: obtains the value of the content-encoding response header field.
Getcontentlength: Get the value of the Content-Length response header field.
Getcontenttype: Get the value of the Content-Type response header field.
Getdate (): Get the value of the date response header field.
Getexpiration (): Get the value of the expires response header field.
Getlastmodified (): Get the value of the last-modified response header field. The following example shows how to send a GET request and a POST request to a Web site and obtain a response from the Web site. Program list: codes/17/17-2/testgetpost. Java
Public class testgetpost
{
/**
* Send a GET request to a specified URL
* @ Param URL the request URL
* @ Param request parameter. The request parameter should be in the form of name1 = value1 & name2 = value2.
* @ Return URL indicates the response of the Remote resource.
*/
Public static string sendget (string URL, string PARAM)
{
String result = "";
Bufferedreader in = NULL;
Try
{
String urlname = URL + "? "+ Param;
URL realurl = new URL (urlname );
// Open the connection with the URL
Urlconnection conn = realurl. openconnection ();
// Set common request attributes
Conn. setrequestproperty ("accept ","*/*");
Conn. setrequestproperty ("connection", "keep-alive ");
Conn. setrequestproperty ("User-Agent ",
"Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; sv1 )");
// Establish the actual connection
Conn. Connect ();
// Obtain all response header fields
Map <string, list <string> map = conn. getheaderfields ();
// Traverse all response header fields
For (string key: map. keyset ())
{
System. Out. println (Key + "--->" + map. Get (key ));
}
// Define the bufferedreader input stream to read the URL response
In = new bufferedreader (
New inputstreamreader (conn. getinputstream ()));
String line;
While (line = in. Readline ())! = NULL)
{
Esult + = "\ n" + line;
}
}
Catch (exception E)
{
System. Out. println ("an exception occurred when sending a GET request! "+ E );
E. printstacktrace ();
}
// Use finally blocks to close the input stream
Finally
{
Try
{
If (in! = NULL)
{
In. Close ();
}
}
Catch (ioexception ex)
{
Ex. printstacktrace ();
}
}
Return result;
}
/**
* Send a POST method request to a specified URL
* @ Param URL the request URL
* @ Param request parameter. The request parameter should be in the form of name1 = value1 & name2 = value2.
* @ Return URL indicates the response of the Remote resource.
*/
Public static string sendpost (string URL, string PARAM)
{
Printwriter out = NULL;
Bufferedreader in = NULL;
String result = "";
Try
{
URL realurl = new URL (URL );
// Open the connection with the URL
Urlconnection conn = realurl. openconnection ();
// Set common request attributes
Conn. setrequestproperty ("accept ","*/*");
Conn. setrequestproperty ("connection", "keep-alive ");
Conn. setrequestproperty ("User-Agent ",
"Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; sv1 )");
// You must set the following two rows to send a POST request:
Conn. setdooutput (true );
Conn. setdoinput (true );
// Obtain the output stream corresponding to the urlconnection object
Out = new printwriter (conn. getoutputstream ());
// Send Request Parameters
Out. Print (PARAM );
// Flush the buffer of the output stream
Out. Flush ();
// Define the bufferedreader input stream to read the URL response
In = new bufferedreader (
New inputstreamreader (conn. getinputstream ()));
String line;
While (line = in. Readline ())! = NULL)
{
Result + = "\ n" + line;
}
}
Catch (exception E)
{
System. Out. println ("an exception occurred when sending the POST request! "+ E );
E. printstacktrace ();
}
// Use finally blocks to close output streams and input streams
Finally
{
Try
{
If (OUT! = NULL)
{
Out. Close ();
}
If (in! = NULL)
{
In. Close ();
}
}
Catch (ioexception ex)
{
Ex. printstacktrace ();
}
}
Return result;
}
// Provides the primary method to test sending GET requests and post requests
Public static void main (string ARGs [])
{
// Send a GET request
String S = testgetpost. sendget ("http: // localhost: 8888/ABC/
Login. jsp ", null );
System. Out. println (s );
// Send a POST request
String S1 = testgetpost. sendpost ("http: // localhost: 8888/ABC/a. jsp ",
"User = Li Gang & pass = ABC ");
System. Out. println (S1 );
}
} |
When sending a GET request in the above program, you only need to put the request parameters after the URL string? The program can directly call the Connect Method of the urlconnection object, as shown in the bold text code in the sendget method in the program. If the program needs to send a POST request, you must first set the values of the doin and doout request header fields, and then use the output stream corresponding to urlconnection to send request parameters, as shown in the bold text code in the sendpost () method in the program. Whether sending a GET request or a POST request, the program obtains the urlconnection response in the same way: if the program can determine that the remote response is a response stream, it can use the response stream to read the response; if the program cannot determine that the remote response is a response stream, use the byte stream to read the response. The two URLs in the above program send requests are Web applications deployed on the local machine by the author. For details about how to create a web application, refer to the author's Lightweight J2EE enterprise application practices for compiling JSP pages. Because the program can directly send a request to the server in this way-equivalent to submitting a logon table page in the Web application, this allows the program to constantly change the user name and password to submit logon requests, it is called brute-force cracking until the logon is successful. |
