Analysis of core technology of network acquisition Software series (6)---Download all blog posts from any blogger to the SQLite database and display them through Webbrower (combining the previous content)

A series of essays catalogue and this section code download

Development environment: VS2008

This section source location: Https://github.com/songboriceboy/GatherAllStoreInDB

Source code Download method: Install the SVN client (provided at the end of this article), and then checkout the following address: Https://github.com/songboriceboy/GatherAllStoreInDB

The outline of the article series is as follows:

1. How to use the C # language to obtain the blog of a blogger's full essay link and title;
2. How to use the C # language to get the content of the blog post;
3. How to convert HTML pages to PDF (html2pdf) using the C # language
4. How to use the C # language to download all the pictures in the blog post to local and can browse offline
5. How to use the C # language to synthesize multiple individual PDF files into a PDF and generate a table of contents
6. NetEase Blog Links How to use C # language to obtain, NetEase blog particularity;
7. How to use the C # language to download the public number articles;
8. How to obtain the full text of any one article
9. How to use the C # language to get rid of all the tags in HTML for plain text (html2txt)
10. How to compile multiple HTML files into CHM (Html2chm) using the C # language
11. How to use the C # language to publish articles remotely to Sina Blog
12. How to develop a static site builder using the C # language
13. How to build a program framework using the C # language (Classic WinForm interface, top menu bar, toolbars, left tree list, right multi-tab interface)
14. How to implement the Web page Editor (Winform) using the C # language ...

The sixth section of the main content of the introduction (download any blogger's blog post to the SQLite database and display through Webbrower)

Download all blog posts from any blogger to the SQLite database and display the solution via Webbrower, demo demo as shown: executable file download

Unlike the demo in the previous section, all of the blog posts we got in the previous section were saved in a DataTable (memory), and when the program was closed, all the downloaded posts were gone, and the next time we had to download them again, it was obviously not good.

This time we will download the blog in the SQLite database, each new blogger, the program will automatically in the executable file in the folder under the Websitedb subdirectory to create a. db file named after the Bo Master ID, the database is a SQLite database.

When the program loads, it automatically executes the Websitedb subdirectory scan under the folder where the file is located, lists the scan to the database name in the ComboBox dropdown, clicks on a drop-down item, and the program automatically loads all the data from the article table in the database to the DataGridView display. Click on an item in DataGridView to browse the Web in the lower Webbrower.

Three basic principles

We have defined a database table for all blogs of a blogger, with the following table structure:

   stringM_strcreattable =@"--1-2-Layer Node table (au_layernode), drop table if exists [Au_layernode]; CREATE TABLE au_layernode (au_layernodeid int not NULL PRIMARY KEY, Au_parentlayernodeid int Not NULL for DEFAULT 0,au_urladdressVARCHAR (+) not NULL DEFAULT ' ',Au_urltitleNVARCHAR (+) not NULL DEFAULT ' ',au_urlcontent              NTEXT NOT NULL default "', Au_urllayer INT NOT null default 0, Au_isvisit          int NOT NULL for default 0, Au_removesameoffset1 INT NOT null default 0, Au_removesameoffset2             INT NOT NULL default 0, au_lastupdatedate DATETIME NOT null default ' 2012-01-01 ', au_reserveint1 int NOT NULL default 0, Au_reserveint2 int not null default 0, Au_reserveint3 int  Not NULL for default 0, Au_reserveint4 int not NULL for default 0, AU_RESERVEINT5 int NOT NULL default 0, Au_reserveint6 int not NULL for default 0, au_reserveint7 int not null default 0, Au_reserv EInt8 INT NOT NULL default 0, AU_RESERVESTR1 VARCHAR (+) NOT null default ', Au_reservest    R2 VARCHAR (+) NOT null default ' ', Au_reservenstr1 NVARCHAR (+) NOT null default ',         Au_reservenstr2     NVARCHAR (+) NOT null default "', AU_RESERVETEXT1 TEXT NOT null default ', AU_RESERVETEXT2 Text NOT NULL default "', AU_RESERVETEXT3 TEXT NOT null default ', Au_reserventext1 NT EXT NOT NULL default ", AU_RESERVENTEXT2 NTEXT NOT null default", AU_RESERVENTEXT3 NTEXT NOT null D Efault ', au_reservedatetime1 DATETIME not NULL DEFAULT ' 2012-01-01 ', au_reservedatetime2 DATE Time NOT NULL for default ' 2012-01-01 ', au_reservedatetime3 DATETIME not null default ' 2012-01-01 ', Au_reser    VeDateTime4 DATETIME NOT null default ' 2012-01-01 ', au_reservedecmial1 DECIMAL NOT null default 0, Au_reservedecmial2 DECIMAL not NULL DEFAULT 0);";

One of the most important is the Au_urladdress,au_urltitle,au_urlcontent 3 fields, respectively, the blog link address, blog post title, the text content of the blog post.

Next, comparing the previous section, we added the following lines of code to the new blogger download function:

Private CLS_SQLITEMNG m_sqlitemng = new Cls_sqlitemng ();
String@"Data source="@ "\websitedb\"; string @" ; Initial catalog=sqlite;integrated security=true; Max Pool size=10";

private string m_strinserttaskinitdata = @ "INSERT into [Au_layernode] values (0, 0, ' #^ $BlogID $^# ', '", ", 0, 0, 0, 0
, ' 2012-01-01 ', 0, 0, 0, 0, 0, 1, 1, 0, ', ', ', ', ', ', ', ', ', ', ', ', ' 2012-01-01 ', ' 2012-01-01 ', ' 2012-01-01 ', ' 2012 -01-01 ', 1, 0) ";

m_sqlitemng. Createdb (M_strdbfolder+ This. Toolstriptextbox1.text +". DB"); M_sqlitemng. ExecuteSQL (M_strcreattable, M_connstr1+ This. Toolstriptextbox1.text +". DB"+m_connstr2); stringStrinserttaskinitdata = M_strinserttaskinitdata.replace ("#^ $BlogID $^#", This. Toolstriptextbox1.text); M_sqlitemng. ExecuteSQL (Strinserttaskinitdata, M_connstr1+ This. Toolstriptextbox1.text +". DB"+ M_CONNSTR2);

M_sqlitemng.createdb (M_strdbfolder + This.toolStripTextBox1.Text + ". db");
The above sentence is to create a database;
M_sqlitemng.executesql (m_strcreattable, m_connstr1 + This.toolStripTextBox1.Text + ". db" + M_CONNSTR2);
The above sentence is to create a database table in the database;
M_sqlitemng.executesql (Strinserttaskinitdata, m_connstr1 + This.toolStripTextBox1.Text + ". db" + M_CONNSTR2);
The above sentence inserts a default data in the database table;
Where CLS_SQLITEMNG is the package of a SQLite operation class, the code is as follows:

 classcls_sqlitemng {//string m_dbname = ""; //string connstr = ""; //Create a database file that is saved in the current directory under the Hydata folder//         Public voidCreatedb (stringdbName) {           //string databasefilename = System.Environment.CurrentDirectory + @ "/hydata/" + dbName;Sqliteconnection.createfile (dbName); }        //Execute SQL statement//Create a table: ExecuteSQL ("CREATE Table Hytest (TestID TEXT)"); //Insert some data: ExecuteSQL ("INSERT into Hytest (TestID) VALUES (' 1001 ')");         Public voidExecuteSQL (stringSQLSTR,stringstrconstr) {            //connstr = connStr1 + m_dbname + connstr;            using(DbConnection conn =Newsqliteconnection (STRCONSTR)) {Conn.                Open (); DbCommand Comm=Conn.                CreateCommand (); Comm.commandtext=sqlstr; Comm.commandtype=CommandType.Text; Comm.            ExecuteNonQuery (); }        }    }

In contrast to the previous section, another modification is to get a addblog (Bloggather.delegatepara dp) function on the back of a post callback interface in the underlying collector:

  Private voidAddblog (Bloggather.delegatepara dp) {if( This. invokerequired) { This. Invoke (NewBloggathercnblogs.greetingdelegate ( This.                Addblog), DP); return; }            Try            {                stringStrwholedbname = M_strdbconstringpath + This. Toolstriptextbox1.text +". DB"; dyh_db. Model.au_layernode Modelau_layernode=Newdyh_db.                Model.au_layernode (); Modelau_layernode.au_parentlayernodeid= -1; Modelau_layernode.au_layernodeid=M_bllau_layernode.getmaxid (strwholedbname); Modelau_layernode.au_urllayer=0; Modelau_layernode.au_urladdress=""; stringStrtitle = Regex.Replace (Dp.strtitle,@"[|/\;. ':*? <>-]","").                ToString (); Strtitle= Regex.Replace (Strtitle,"[\"]","").                ToString (); Strtitle= Regex.Replace (Strtitle,@"\s",""); Modelau_layernode.au_urltitle=strtitle; Modelau_layernode.au_urlcontent=dp.strcontent;; Modelau_layernode.au_isvisit=0; Modelau_layernode.au_removesameoffset1=0; Modelau_layernode.au_removesameoffset2=0; Modelau_layernode.au_lastupdatedate=System.DateTime.Now.Date;                M_bllau_layernode.add (Strwholedbname, Modelau_layernode); DataSet Dstemps= M_bllau_layernode.getlist (Strwholedbname,"");  This. Datagridview1.datasource = dstemps.tables[0];  This. datagridview1.columns[1]. Visible =false;  This. datagridview1.columns[0]. Width = This.            Width; }            Catch(Exception ex) {}}

Here, we will save the collected blog post to the database, which used the dynamic code generator three layer structure, the specific code please download the study yourself.

Song Bo
Source: http://www.cnblogs.com/ice-river/
The copyright of this article is owned by the author and the blog Park, welcome reprint, but without the consent of the author must retain this paragraph, and in the article page obvious location to give the original link.
is looking at my blog This children's shoes, I see you imposing, there is a faint of the king's Breath, there will be a future! Next to the word "recommended", you can conveniently point it, action quasi, I do not accept a penny, you also good to come back to me!

Analysis of core technology of network acquisition Software series (6)---Download all blog posts from any blogger to the SQLite database and display them through Webbrower (combining the previous content)