Crawler, web analysis and Analytic Assistant tool Xpath-helper

Moving from my blog: http://www.xgezhang.com/xpath_helper.html

Every person who writes a crawler, or does a Web page analysis, believes that it will take a lot of time to locate, get the XPath path, and even sometimes when the crawler framework matures, basically the main time is spent on page parsing. In the absence of these aids, we can only search the HTML source code, locate some ID to find the corresponding location, very troublesome, and often error. Here is an XPath helper plugin for Chrome for everyone.

Download link First: xpath-helper

Once installed, we reopen the browser and press Ctrl+shift+x to bring up the Xpath-helper box. The interface is as follows:

If we are looking for an XPath path for one or a piece of element, you can hold down SHIFT and move to the block, and the box above will show the XPath path to the element, and the parsed text will be displayed on the right, and we can change the XPath path ourselves. The program will also automatically display the corresponding location, it is convenient to help us to determine whether our XPath statement is written correctly.

Although this small plug-in is very convenient to use, but it is not omnipotent, there are two problems:

1.XPath Helper automatically extracts the XPath from the root path, which almost inevitably lead to XPath too long, not conducive to maintenance;

2. When extracting the list data for a loop, the XPath Helper is using the subscript to extract each piece of data separately from the list, which is not suitable for batch processing of the program, or the need to manually modify some similar to the * tag.

However, a reasonable use of XPath, or can save us a lot of time!

Copyright NOTICE: This article for Bo Master original article, without Bo Master permission not reproduced.

Crawler, web analysis and Analytic Assistant tool Xpath-helper