HttpWebRequest fetching of streams and WebClient files

HttpWebRequest fetching of streams and WebClient files

Yesterday wrote a crawl, encountered a pit, is to get the network stream, the man-made use of the stream. Length to get the lengths of the stream, the fetch will throw an error, check the document, because some of the flow is unable to obtain the length of the data, so it cannot be directly obtained. You can avoid this problem if you are dealing with stream often. Actually directly using Do-while to get on the line, the code is as follows:

int i=0;do{    byte[] buffer = new byte[1024]; i = stream.Read(buffer, 0, 1024); fs.Write(buffer, 0, i);} while (i >0);

Where the while can only write i>0; and cannot be written i>=1024; reasons can be seen in MSDN an explanation: MSDN

Read returns 0 only if there is no more data in the stream and no more data is expected (such as if the socket is closed or at the end of the file). The implementation can return less than the requested byte at will, even if the end of the stream has not been reached.

Here is a short code for HttpWebRequest and WebClient fetching data:

HttpWebRequest

///<summary>//////</summary>///<param name= "url" > Crawl URL</param>///<param name= "FilePath" > Save file name</param>///<param name= "Oldurl" > Source path</param>///<returns></returns>PublicStaticboolHttpdown (String URL,String FilePath,String oldurl) {try {HttpWebRequest req = webrequest.create (URL)As HttpWebRequest; Req. Accept =@ "text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,*/*;q=0.8"; Req. Referer = Oldurl; Req. UserAgent =@ "mozilla/5.0 (Windows NT 6.3; WOW64) applewebkit/537.36 (khtml, like Gecko) chrome/33.0.1750.154 safari/537.36 "; Req. ContentType ="Application/octet-stream"; HttpWebResponse response = req. GetResponse ()As HttpWebResponse; Stream stream = Response. GetResponseStream ();//StreamReader readstream=new StreamReader FileStream fs = File.create (FilePath); Long length = response. ContentLength; int i=0; Do { byte[] buffer = new byte[1024x768]; i = stream. Read (buffer, 0, 1024x768); fs. Write (buffer, 0, i);} while (i >0); fs. Close (); return true;} catch (Exception ex) { return false;}}           

WebClient

PublicStaticboolDown (String URL,String desc,String oldurl) {try {WebClient WC =new WebClient (), WC. Headers.add (httprequestheader.accept, @ "text/html,application/xhtml+xml,application/ xml;q=0.9,image/webp,*/*;q=0.8 "); Wc. Headers.add (Httprequestheader.referer, Oldurl); Wc. Headers.add (httprequestheader.useragent, @ "mozilla/5.0 (Windows NT 6.3; WOW64) applewebkit/537.36 (khtml, like Gecko) chrome/33.0.1750.154 safari/537.36 "); Wc. Headers.add (Httprequestheader.contenttype,  "Application/octet-stream"), WC. DownloadFile (new Uri (URL), desc); Console.WriteLine (URL); Console.WriteLine ( "+desc + " yes! "); return true;} catch (Exception ex) {return  False }}

HttpWebRequest Get the file crawl of streams and WebClient