Use HttpWebrequest to simulate the website (with Baidu demo) and httpwebrequestdemo

Use HttpWebrequest to simulate the website (with Baidu demo) and httpwebrequestdemo

 

This is the first official article in the blog Park.

 

×

No surprisesIn the future, we will develop in the web direction. Some time ago, we retried the webqq robot we had previously done. This is a farewell to winform and reinforces the knowledge of C.

This article describes how to simulate http requests. (The style of a blog is self-written, and it looks nice to have a wood (• too many others)

First View a GET request

 

public string GetHtml(string url, string Referer, Encoding encode, bool SaveCookie)        {            HttpWebRequest req = WebRequest.Create(url) as HttpWebRequest;            req.Method = "GET";            req.CookieContainer = this.CookieContainer;            req.Proxy = null;            if (!string.IsNullOrEmpty(Referer))                req.Referer = Referer;            using (HttpWebResponse hwr = req.GetResponse() as HttpWebResponse)            {                if (SaveCookie)                {                    this.CookieCollection = hwr.Cookies;                    this.CookieContainer.GetCookies(req.RequestUri);                }                using (StreamReader SR = new StreamReader(hwr.GetResponseStream(), encode))                {                    return SR.ReadToEnd();                }            }        }

 

Then View POST again

 public string PostHtml(string url, string Referer, string data, Encoding encode, bool SaveCookie)        {            HttpWebRequest req = WebRequest.Create(url) as HttpWebRequest;            req.CookieContainer = this.CookieContainer;            req.ContentType = "application/x-www-form-urlencoded";            req.Method = "POST";            req.UserAgent = "Mozilla/5.0 (Windows NT 5.1; rv:30.0) Gecko/20100101 Firefox/30.0";            req.Proxy = null;            req.ProtocolVersion = HttpVersion.Version10;            if (!string.IsNullOrEmpty(Referer))                req.Referer = Referer;            byte[] mybyte = Encoding.Default.GetBytes(data);            req.ContentLength = mybyte.Length;            using (Stream stream = req.GetRequestStream())            {                stream.Write(mybyte, 0, mybyte.Length);            }            using (HttpWebResponse hwr = req.GetResponse() as HttpWebResponse)            {                if (SaveCookie)                {                    this.CookieCollection = hwr.Cookies;                    this.CookieContainer.GetCookies(req.RequestUri);                }                using (StreamReader SR = new StreamReader(hwr.GetResponseStream(), encode))                {                    return SR.ReadToEnd();                }            }        }

 

 

Summary

1. This is part of an httphelp class I encapsulated. get does not need anything like UserAgent or ContentType.

2. Set ContentType and UserAgent in POST requests.

3. The default request is GET.

 

 

How Keep CookieSave CookieContainer after each request and assign it to the httpwebrequest of the next request.

 

Differences HttpWebRequest. CookieContainer and HttpWebResponse. CookieCollection

This section describes how to obtain a response after an http request.

CookieContainer is all cookies in the current domain

CookieCollection is all cookies related to this request.

 

Referer What is

That is, the source of this request, from which page the http request is sent

The server is used to check whether the request is valid.

 

Proxy?

I don't know much about the request proxy. I forget someone can tell me.

Set proxy = null before the request to allow the request to skip the check proxy. The http request speed immediately increases to a higher level! I have a deep understanding, especially when I used to get rid of wifi.

 

 

 

 

Here is an http request helper class I encapsulated. If you are interested, refer:

 

Public class HttpHelp {public CookieContainer {get; set;} public CookieCollection {get; set;} public HttpHelp () {this. cookieCollection = new CookieCollection (); this. cookieContainer = new CookieContainer ();} public static string GetHtml (string url, string Referer, Encoding encode) {return new HttpHelp (). getHtml (url, Referer, encode, false);} public static s Tring PostHtml (string url, string Referer, string data, Encoding encode) {return new HttpHelp (). postHtml (url, Referer, data, encode, false);} public string GetHtml (string url, string Referer, Encoding encode, bool SaveCookie) {HttpWebRequest req = WebRequest. create (url) as HttpWebRequest; req. method = "GET"; req. cookieContainer = this. cookieContainer; req. proxy = null; if (! String. isNullOrEmpty (Referer) req. referer = Referer; using (HttpWebResponse hwr = req. getResponse () as HttpWebResponse) {if (SaveCookie) {this. cookieCollection = hwr. cookies; this. cookieContainer. getCookies (req. requestUri);} using (StreamReader SR = new StreamReader (hwr. getResponseStream (), encode) {return SR. readToEnd () ;}} public string PostHtml (string url, string Referer, string data, E Ncoding encode, bool SaveCookie) {HttpWebRequest req = WebRequest. create (url) as HttpWebRequest; req. cookieContainer = this. cookieContainer; req. contentType = "application/x-www-form-urlencoded"; req. method = "POST"; req. userAgent = "Mozilla/5.0 (Windows NT 5.1; rv: 30.0) Gecko/20100101 Firefox/30.0"; req. proxy = null; req. protocolVersion = HttpVersion. version10; if (! String. isNullOrEmpty (Referer) req. referer = Referer; byte [] mybyte = Encoding. default. getBytes (data); req. contentLength = mybyte. length; using (Stream stream = req. getRequestStream () {stream. write (mybyte, 0, mybyte. length);} using (HttpWebResponse hwr = req. getResponse () as HttpWebResponse) {if (SaveCookie) {this. cookieCollection = hwr. cookies; this. cookieContainer. getCookies (req. requestUri );} Using (StreamReader SR = new StreamReader (hwr. getResponseStream (), encode) {return SR. readToEnd ();}}} ///// upload a file ///// upload address // file path // the name of the original webpage file control /// the contentType in the request stream // returned encoding // post parameter dictionary // public static string PostFile (string url, string filepath, string paramName, string contentType, Encoding encode, Dictionary <string, string> dict) {HttpWebRequest hrq = WebRequest. create (u Rl) as HttpWebRequest; string boundary = "---------------------------" + DateTime. now. ticks. toString ("x"); byte [] boundarybytes = System. text. encoding. default. getBytes ("\ r \ n --" + boundary + "\ r \ n"); hrq. contentType = "multipart/form-data; boundary =" + boundary; hrq. method = "POST"; using (Stream stream = hrq. getRequestStream () // request stream {// write the post parameter string formdataTemplete = "Content-Disposition: M-data; name = \ "{0} \" \ r \ n {1} "; if (dict! = Null & dict. count> 0) {foreach (KeyValuePair <string, string> pair in dict) {stream. write (boundarybytes, 0, boundarybytes. length); string formitem = string. format (formdataTemplete, pair. key, pair. value); byte [] formitemBytes = Encoding. default. getBytes (formitem); stream. write (formitemBytes, 0, formitemBytes. length) ;}} stream. write (boundarybytes, 0, boundarybytes. length); // write header information string headerTemplate = "Content-Disposition: form-data; name = \" {0 }\"; filename = \ "{1} \" \ r \ nContent-Type: {2} \ r \ n "; string header = string. format (headerTemplate, paramName, Path. getFileName (filepath), contentType); byte [] headerBytes = Encoding. UTF8.GetBytes (header); stream. write (headerBytes, 0, headerBytes. length); // write the File byte [] fileBytes = File. readAllBytes (filepath); stream. write (fileBytes, 0, fileBytes. length); // write the end byte [] footerBytes = Encoding. default. getBytes ("\ r \ n --" + boundary + "-- \ r \ n"); stream. write (footerBytes, 0, footerBytes. length); using (HttpWebResponse hrp = hrq. getResponse () as HttpWebResponse) // response stream {using (StreamReader SR = new StreamReader (hrp. getResponseStream (), encode) {return SR. readToEnd ();}}}}}

 

 

 

Baidu login example: Click to download

I logged on to a demo of Baidu. I wrote = in 13 years. =,Login encryption nowHowever, this method can still be used in the past.

 

I have never written a blog, either Sina, QQ or others. Today, I found that the layout is too tired.

 

 

Then I plan to write it and simulate webqq requests to implement the QQ robot. Write a series as much as possible *)

 

 

.