Learn more about how Php+javascript crawls Web content

This article to share the content is detailed introduction php+javascript How to crawl Web content, has a certain reference value, the need for friends can refer to

Php+js Crawl Page Content--first look at the effect

How do we do that?

We always thought Python was the only way to crawl Web content, which is because Python itself sets up many class libraries to crawl pages, but we use the Php+js method as easily as we can get the content of the Web page we want, and it doesn't have to be tedious.

First we need PHP to simulate the request to get the HTML for the entire site.

  Allow all domains to access  the header ("Access-control-allow-origin: *");  /receive a parameter named Parm  $parm =$_get[' mod ');  if (empty ($parm)) {      $url = ' http://m.80s.tw/';//target site    $html = file_get_contents ($url);} else{    $url = ' http://m.80s.tw/'. $parm;      $html = file_get_contents ($url);}     Preg_match ("/<body[^>]*?> (. *\s*?) <\/body>/is ", $html, $match 1);//Regular matches the contents of the Body    echo $match 1[0];//output Web page

Note: If you encounter file_get_contents error, please try to find the Extension=php_openssl.dll in the php.ini to open it OK

And then it's the front end to get the data processed.

Write an asynchronous request first

$.ajax ({         type: ' Get ',        URL: '). /./admin/test.php ',        success:function (data) {        console.log (data)//can see the HTML obtained, very simple, very excited        }    );

After we get the HTML, we can do whatever we want.

How to use these HTML? Is that a problem? No

        First create a container        var p = document.createelement (' P ');                The entire HTML string is stored in the p node        p.innerhtml = data;                Then you can check for P        //For example get class List_mov_title under all the A-label        var list = P.queryselectorall ('. List_mov_title a ');                Print it out and look at the        console.log (list)                //What you want is in the bar        //Then put what you want into your page.

A crawl of the content of the Web page to the end of the tutorial, if you suddenly see the right to forward it, do not understand the message it