Regular expressions are usually used to capture the content of a single website. However, the structures of different websites are strange, and it is difficult to use a unified regular expression for matching. The author of the generic web page text extraction algorithm based on the Row block Distribution function summarizes the methods for extracting the text from the web page, and proposes a text extraction algorithm based on the Row block distribution, and provides PHP, Java, and other implementations. The main principle of this algorithm is based on two points: regular expression matching is often used to capture the content of a single website webpage. However, the structures of different websites are strange, and it is difficult to use a unified regular expression for matching. The author of the generic web page text extraction algorithm based on the Row block Distribution function summarizes the methods for extracting the text from the web page, and proposes a text extraction algorithm based on the Row block distribution, and provides PHP, Java, and other implementations. The main principle of this algorithm is based on two points: 1. body area density: after all tags in HTML are removed, the character density in the body area is higher, with fewer blank lines; 2. Row block length: the content of a non-body area is generally shorter than that of a separate label (Row block. The algorithm steps are as follows:
1. remove all tags, including style and Js script content, but retain the original line break \ n
The above is the content of the image extraction algorithm for the webpage body and content in the [python Tutorial]. For more information, see The PHP Chinese website (www.php1.cn )!
