Use the Symfony Crawler component in laravel to analyze HTML and laravelsymfony.

Use the Symfony Crawler component in laravel to analyze HTML and laravelsymfony.

The full name of the Crawler is DomCrawler, which is a component of the Symfony framework. What's so frightening is that DomCrawler does not have a Chinese document and Symfony does not translate this part. Therefore, we can only find out how to use DomCrawler for development. Now we will summarize the experience in the process.

First, install

composer require symfony/dom-crawlercomposer require symfony/css-selector

Css-seelctor is a css selector, which is used by some functions when css is used to select nodes.

The example used in the manual is

use Symfony\Component\DomCrawler\Crawler;$html = <<<‘HTML‘Hello World!Hello Crawler!HTML;$crawler = new Crawler($html);foreach ($crawler as $domElement){var_dump($domElement->nodeName);}

The printed result is

string ‘html‘ (length=4)

Because the nodeName of this html code is html, and the English language is poor, I thought the program was wrong when I started using it...

In actual use, if the new Crawler ($ html) is garbled, it should be related to page encoding. Therefore, you can initialize the crawler in the following way, and then add the node

$crawler = new Crawler();$crawler->addHtmlContent($html);

The second parameter of addHtmlContent is charset. The default parameter is UTF-8.

For other examples, refer to the official documentation, http://symfony.com/doc/current/components/dom_crawler.html

Record the usage that I tried at work

FilterXPath (string $ xpath) method. According to the manual, the parameter of this method is $ xpath, and blocks such as p and div are often used.

echo $crawler->filterXPath(‘//body/p‘)->text();echo $crawler->filterXPath(‘//body/p‘)->last()->text();

The output is the text of the first and next p tag blocks.

var_dump($crawler->filterXPath(‘//body‘)->html());

Html in the output body

foreach ($crawler->filterXPath(‘//body/p‘) as $i => $node) {$c = new Crawler($node);echo $c->filter(‘p‘)->text();}

FilterXPath obtains the DOMElement block array. Each DOMElement block can be parsed using a new crawler object.

$nodeValues =$crawler->filterXPath(‘//body/p‘)->each(function (Crawler $node, $i) {return $node->text();});

Crawler provides an each loop and uses the closure function to simplify the Code. However, note that $ nodeValues gets an array and needs further processing.

Other usage

echo $crawler->filterXPath(‘//body/p‘)->attr(‘class‘);

You can obtain the value "message" of the class Attribute corresponding to the first p tag"

$ Crawler-> filterXPath ('// div [@ class = "style"]')-> filter ('A')-> attr ('href '); $ crawler-> filterXPath ('// div [@ class = "style"]')-> filter ('a> img ')-> extract (array ('alt ', 'href '))

The preceding methods are used to obtain tag attributes.

Filter is different from filterXPath. the css selector is written in the manual. I don't quite understand it. I think it is the elements contained in an XPath node like div. The specific situation still needs to be tried in actual development.

In general, DomCrawler is easier to use than the simple html dom, probably because it is relatively simple to use.

The above is only the basic functions of the Crawler. For more usage, refer to the functions of the Crawler section in the symfony manual.

Http://api.symfony.com/3.2/Symfony/Component/DomCrawler/Crawler.html

The main problem with Crawler is that there are too few examples. There are no examples in the function manual. You can only find them in actual use ....

Symfony's documentation on DomCrawler contains a few examples

Http://symfony.com/doc/current/components/dom_crawler.html

The above section describes how to use the Symfony Crawler component in laravel to analyze HTML. I hope it will help you. If you have any questions, please leave a message, the editor will reply to you in a timely manner. Thank you very much for your support for the help House website!