HttpClient, using C # to manipulate the Web

We know. NET class library provides classes such as HttpWebRequest, which allows us to program and interact with the Web server. But in the actual use we often encounter the following requirements, the Basic class does not directly provide the corresponding function (WebClient class contains these functions, just a little trouble with the one o'clock---Thank you, the user Dong Wu's reminder ):

• The text encoding of the HTML obtained by HttpWebResponse is converted so that it does not appear garbled;
• Automatically maintain cookie,referer and other related information during session;
• Simulate HTML form submission;
• Uploading files to the server;
• For binary resources, get directly the returned byte array (byte[]), or save as a file

To solve these problems, I developed the HttpClient class. Here's how to use it:

• Gets the encoded converted string
  
  HttpClient client=new HttpClient (URL);
  String html=client. GetString ();
  
  The GetString () function internally looks for the HTTP Headers, as well as the META tag of the HTML, to try to find the encoded information of the obtained content. If you can't find it, it will use the client. Defaultencoding, this property defaults to Utf-8, or it can be set manually.
• Automatically keep cookies, Referer
  
  HttpClient client=new HttpClient (URL1, NULL, TRUE);
  String html1=client. GetString ();
  Client. URL=URL2;
  String html2=client. GetString ();
  
  Here httpclient the third parameter, when Keepcontext is set to True, HttpClient automatically records the server's actions on the cookie each time the interaction occurs, and the URL of the previous request is referer. In this example, when you get the HTML2 , URL1 will be used as a referer, and the server will be passed cookies that are set by the server when acquiring HTML1. Of course, you can also directly provide the first request to send the cookies and referer when constructing the httpclient:
  
  HttpClient client=new HttpClient (URL, new Webcontext (cookies, Referer), true);
  
  Or, modify this information at any time during use:
  
  Client. Context.cookies=cookies;
  Client. Context.referer=referer;
• Simulating HTML form Submissions
  
  HttpClient client=new HttpClient (URL);
  Client. Postingdata.add (fieldName1, filedValue1);
  Client. Postingdata.add (fieldName2, fieldValue2);
  String html=client. GetString ();
  
  The above code is equivalent to submitting a form with two input. When Postingdata is not empty, or if you attach a file to upload (see the upload and file below), HttpClient automatically changes the Httpverb to post and attaches the appropriate information to the request.
• Uploading files to the server
  
  HttpClient client=new HttpClient (URL);
  Client. Attachfile (FileName, fieldName);
  Client. Attachfile (ByteArray, FileName, fieldName);
  String html=client. GetString ();
  
  The fieldName in this is equivalent to FieldName in <input type= "file" Name= "/>" fieldName. FileName is, of course, the file path you want to upload. You can also directly provide a byte[] as a file content, but even then you must provide a file name to meet the HTTP specification requirements.
• Different forms of return
  
  String: String html = client. GetString ();
  Stream: Stream stream = client. GetStream ();
  byte array: byte[] data = client. GetBytes ();
  Save to file: client. SaveAsFile (FileName);
  Alternatively, you can also directly manipulate httpwebresponse:httpwebresponse res = client. GetResponse ();
  
  Each invocation of one of these methods causes an HTTP request to be issued, that is, you cannot get two return forms of a response at the same time.
  In addition, after they have been called any one, you can pass the client. Responseheaders to get the HTTP headers returned by the server.
• Download the specified part of the resource (for the breakpoint continuation, multi-threaded download)
  
  HttpClient client=new HttpClient (URL);
  Make head request, get resource length
  int length=client. Headcontentlength ();
  
  Get only the latter half of the content
  Client. STARTPOINT=LENGTH/2;
  Byte[] Data=client. GetBytes ();
  
  Headcontentlength () only sends an HTTP HEAD request. Depending on the HTTP protocol, head is equivalent to get, but only the HTTP header is returned, not the resource principal content. In other words, with this method, you can't get the length of a resource that needs to be received via post, and if you do have this requirement, it is recommended that you get content-length from the Responseheader by GetResponse ().
  
  

There are other features to add in the plan, such as breakpoint continuation, multi-threaded download, download progress Update event mechanism, and so on, is thinking how to integrate with the current code, looking forward to your feedback.

You can download the entire code for the current version from here.

Note: you should add a reference to System.Web.dll when you use it, and add a "using System.Web;" Before using this class of code, otherwise you will not be able to compile (thanks to Hyke's reminders).

[update:2007 year August 11]

• Fixed a bug related to file upload;
• Follow your advice and add XML annotations to public methods and attributes;
• Added support for breakpoint continuation (also consider how to make it easier to use).
• Fixed a post-related bug

HttpClient, using C # to manipulate the Web