About the StreamReader. ReadToEnd method, reader. readtoend

About the StreamReader. ReadToEnd method, reader. readtoend

In the past, the code used to capture web pages liked to use ReadToEnd, because it was easy to use. Later I found that when crawling a Web page, if the network speed is very slow, the chance of ReadToEnd timeout is very high. After the Read is rewritten, the timeout rate is greatly reduced. The complete code is as follows:

/// <Summary> /// HttpPost /// </summary> public static string HttpPost (string url, string data) {byte [] bArr = ASCIIEncoding. UTF8.GetBytes (data); // set the parameter HttpWebRequest = WebRequest. create (url) as HttpWebRequest; request. cookieContainer = m_Cookie; request. method = "POST"; request. contentType = "application/x-www-form-urlencoded"; request. contentLength = bArr. length; request. userAgent = "Mozilla/4.0 (compatible; MSIE 8.0; Windows NT 6.0; Trident/4.0)"; Stream postStream = request. getRequestStream (); postStream. write (bArr, 0, bArr. length); postStream. close (); // send the request and obtain the response data HttpWebResponse response = request. getResponse () as HttpWebResponse; // wait until request. the GetResponse () program starts to send the Post request Stream responseStream = response to the target webpage. getResponseStream (); // response result webpage (html) code MemoryStream memoryStream = new MemoryStream (); bArr = new byte [1024]; int size = responseStream. read (bArr, 0, (int) bArr. length); while (size> 0) {memoryStream. write (bArr, 0, size); size = responseStream. read (bArr, 0, (int) bArr. length); Thread. sleep (1);} string content = Encoding. UTF8.GetString (memoryStream. toArray (); return content ;}View Code

In the code, Thread. Sleep (1); can also be removed.