PHP crawler (2) DOM processing

Summary: In PHP Crawler ( 1 ) describes in detail the CURL Crawl HTML data Technology. The acquisition of data processing is also a very important part of crawler technology. Processing HTML data can be found in strings or regular expressions, but using Dom processing is the most advanced method.

Now we have to crawl the Chinese Military Network homepage "Military Media News to discuss" the first content,

View page elements through the browser tool and get data through the console . We can see that in the browser we can easily call Jquery 's dom lookup function to find the data,PHP Is there a tool that can handle the DOMeasily?

looking for PHP Dom parseon GitHub , we found sunra/php-simple-html-dom-parser, There are quite a few users in use. Here we describe how to use the Phpdomparse component, which describes the direct reference,composer,thinkphp In how to use it.

1. Direct reference

Preferred Download Components folder, executed in the current project directory

git clone https://github.com/sunra/php-simple-html-dom-parser.git

At this time, the project directory will be more than the php-simpple-html-dom-parser folder, folder directory such as

Reference htmldomparaer.php

<?PHPinclude"./php-simple-html-dom-parser/src/sunra/phpsimple/htmldomparser.php";  UseSunra\phpsimple\htmldomparser; $url= "http://www.81.cn"; $dom= Htmldomparser::file_get_html ($url ); $u=$dom->find ("#JMYWYL DT", 0); $title=$u->find (' A ', 0); Echo $title-innertext;?>

2.Composer Calling Methods

Install Composer.pharin the project directory, create the composer.json file,

{    "require": {        "Sunra/php-simple-html-dom-parser": "v1.5.0"    }}

Run, php Compose.phar Install

Composer will Download the required files according to Composer.json content, after successful installation, the project folder is as follows,

code as follows, the system will automatically load \sunra\phpsimple\ Htmldomparse

 <? php  require  __dir__.    '/vendor/autoload.php ' ;      $url  = "http://www.81.cn" < span>  $dom  = \sunra\phpsimple\htmldomparser::file_get_html ( $url  );   $u  =  $dom ->find ("#JMYWYL DT",    0);   $title  =  $u ->find (' A ', 0);  echo   $title ->innertext; ? 

3, thinkphp Frame

I also use the TP Framework Daily, perhaps some readers did not introduce other frameworks in TP. TP provides a mechanism for entering the framework, and we place the sunra folder in the "Installation path -->thinkphp->library- >vendor"directory, as shown in the structure,

The project code is as follows

classIndexcontrollerextendsController { Public functionindex () {} Public functionDom () {Vendor (' Sunra.PhpSimple.HtmlDomParser '); $url= "http://www.81.cn"; //$dom = \sunra\phpsimple\htmldomparser::file_get_html ($url);        $dom=file_get_html ($url ); $u=$dom->find ("#JMYWYL DT", 0); $title=$u->find (' A ', 0); Echo $title-innertext; }}

Appendix

PHP Composer.phar Install, the following error occurred,

Ssl3_get_server_certificate:certificate Verify failed

Solution,

(1) wget Http://curl.haxx.se/ca/cacert.pem

(2) Modify php.ini, add

Openssl.cafile= "/opt/lampp/cacert.pem"

PHP crawler (2) DOM processing