First, explain: Code snippets are retrieved from the network and then modified by themselves. I think the good things should be shared.
Implementation Principle: When we collect the page, if the collected site needs to log in to collect. Whether based on cookies or session, we will first send an HTTP request header, which contains the cookie information that the Web site needs. When a Web site receives a sent HTTP request header, it obtains the relevant cookie or session information from the HTTP request header, which is then processed by the program to determine whether you have permission to access the current page.
Well, the principle is clear, it is good to do. All we have to do is to put the cookie information in the HTTP request header when it is collected (or when HttpWebRequest submits the data).
Here I offer 2 ways.
First Kind, put the cookie information directly into the HttpWebRequest Cookiecontainer. Look at the code:
Copy Code code as follows:
protected void Page_Load (object sender, EventArgs e)
{
Set up cookies and deposit Hashtable
Hashtable ht = new Hashtable ();
Ht. ADD ("username", "youraccount");
Ht. ADD ("id", "Yourid");
This. Collect (HT);
}
public void Collect (Hashtable ht)
{
String content = String. Empty;
String url = "http://www.ibest100.com/need to log in to collect the page";
String host = "http://www.ibest100.com";
Try
{
Get the bytes submitted
byte[] bs = Encoding.UTF8.GetBytes (content);
Set up related parameters for submission
HttpWebRequest req = (HttpWebRequest) httpwebrequest.create (URL);
Req. method = "POST";
Req. ContentType = "Application/json;charset=utf-8";
Req. ContentLength = BS. Length;
Put the cookie in the Cookiecontainer and add Cookiecontainer to HttpWebRequest
Cookiecontainer cc = new Cookiecontainer ();
Cc. ADD (New Uri (host), New Cookie ("username", ht["username"). ToString ()));
Cc. ADD (New Uri (host), New Cookie ("id", ht["id"]. ToString ()));
Req. Cookiecontainer = CC;
Submit Request Data
Stream Reqstream = req. GetRequestStream ();
Reqstream.write (BS, 0, BS.) Length);
Reqstream.close ();
Receive the returned page, must not omit
WebResponse WR = req. GetResponse ();
System.IO.Stream Respstream = wr. GetResponseStream ();
System.IO.StreamReader reader = new System.IO.StreamReader (Respstream, System.Text.Encoding.GetEncoding ("Utf-8"));
String t = Reader. ReadToEnd ();
System.Web.HttpContext.Current.Response.Write (t);
Url Close ();
}
catch (Exception ex)
{
System.Web.HttpContext.Current.Response.Write ("Anomaly in Getpostrespone:" + ex.) Source + ":" + ex. message);
}
}
Second Kind, each time to open the acquisition program, you need to first to be collected site simulation login once, get Cookiecontainer, and then collect. Look at the code:
Copy Code code as follows:
protected void Page_Load (object sender, EventArgs e)
{
Try
{
Cookiecontainer Cookiecontainer = new Cookiecontainer ();
String formatstring = "Username={0}&password={1}";//***************
String poststring = String. Format (formatstring, "Youradminaccount", "YourPassword");
Converts the submitted string data into a byte array
byte[] PostData = Encoding.UTF8.GetBytes (poststring);
Set up related parameters for submission
String URI = "http://www.ibest100.com/login page";//***************
HttpWebRequest request = WebRequest.Create (URI) as HttpWebRequest;
Request. method = "POST";
Request. KeepAlive = false;
Request. ContentType = "application/x-www-form-urlencoded";
Request. Cookiecontainer = Cookiecontainer;
Request. ContentLength = Postdata.length;
Submit Request Data
System.IO.Stream outputstream = Request. GetRequestStream ();
Outputstream.write (postdata, 0, postdata.length);
Outputstream.close ();
Receive the returned page, must not omit
HttpWebResponse response = Request. GetResponse () as HttpWebResponse;
System.IO.Stream Responsestream = Response. GetResponseStream ();
System.IO.StreamReader reader = new System.IO.StreamReader (Responsestream, encoding.getencoding ("gb2312"));
String srcstring = reader. ReadToEnd ();
Open the page you want to visit
URI = "http://www.ibest100.com/need to log in to collect the page";//***************
Request = WebRequest.Create (URI) as HttpWebRequest;
Request. method = ' Get ';
Request. KeepAlive = false;
Request. Cookiecontainer = Cookiecontainer;
Receive the returned page
Response = Request. GetResponse () as HttpWebResponse;
Responsestream = Response. GetResponseStream ();
reader = new System.IO.StreamReader (Responsestream, encoding.getencoding ("gb2312"));
srcstring = reader. ReadToEnd ();
The output gets the page or the processing
Response.Write (srcstring);
}
catch (WebException We)
{
String msg = We. message;
Response.Write (msg);
}
}
Some people may ask, if the other side login to verify the code how to do? Then you should use the first method, but you need to analyze each other's cookies.
Application Range: Data collection, forum posts, blog posts.
