Simple crawler, crawling to the blog garden article title, crawler blog article title

Simple crawler, crawling to the blog garden article title, crawler blog article title

It took several hours to write a small program with no technical skills. I wrote a blog for the first time. I am a newbie and want to record my learning.

Running Effect

The Code is as follows:

String url =" https://www.cnblogs.com/ "; Int pagNum = 10; string html = ""; // string pater = "(<a class = \" titlelnk \ "href = \" [\ s \ S] * \ "target = \" _ blank \ ">) ([\ s \ S] *) (</a>) "; string pater =" <a class = \ "titlelnk \" href = \"(. *?) \ "Target = \" _ blank \ "> (.*?) </A> "; Regex regex = new Regex (pater); int num = 0; for (int I = 1; I <pagNum; I ++) {string url2 = url + "# p" + I; try {WebRequest request = WebRequest. create (url2); Stream stream = request. getResponse (). getResponseStream (); using (StreamReader reader = new StreamReader (stream, Encoding. UTF8) {html = reader. readToEnd () ;}} catch (Exception ex) {Console. writeLine (ex. toString (); Console. readK Ey ();} if (! String. isNullOrEmpty (html) {// Match match = Regex. match (html, pater); foreach (Match ma in regex. matches (html) {Match math = Regex. match (ma. value, pater); string title = math. groups [2]. value; num ++; Console. writeLine ("no." + num + "title: \ B" + title); File. appendAllText (@ "d: \ 1.txt", title +" \ r \ n ") ;}} Console. readKey ();