Using Python to extract the abstract,

Using Python to extract the abstract,

This example describes how to extract the abstract in Python. Share it with you for your reference. The details are as follows:

I. Overview

In the blog system's article list, in order to more effectively present the article content, so that readers can select to read more specifically, the article title and abstract are usually provided at the same time.

The content of an article can be in plain text format, but in today's popular network, it is mostly in HTML format. Regardless of the format, the abstract is generally the content at the beginning of the article, which can be extracted according to the specified number of words.

Ii. plain text summarization

A plain text document is a long string that can be easily extracted from its Abstract:

#! /Usr/bin/env python #-*-coding: UTF-8-*-"Get a summary of the TEXT-format document" "def get_summary (text, count ): u "Get the first 'Count' characters from 'text' >>> text = u'welcome this is an article about Python '>>> get_summary (text, 12) = u'welcome this is an article 'true "assert (isinstance (text, unicode) return text [0: count] if _ name _ = '_ main _': import doctest. testmod ()

Iii. HTML Summary

HTML documents contain a large number of tags (such as

While following the HTML document structure, you must parse the HTML document and intercept the content. In Python, you can use the standard library HTMLParser.

The simplest abstract extraction function is to extract only native text inside the tag while ignoring the HTML Tag. Python implementation similar to this function is as follows:

#!/usr/bin/env python# -*- coding: utf-8 -*-"""Get a raw summary of the HTML-format document"""from HTMLParser import HTMLParserclass SummaryHTMLParser(HTMLParser):  """Parse HTML text to get a summary    >>> text = u'<p>Hi guys:</p><p>This is a example using SummaryHTMLParser.</p>'    >>> parser = SummaryHTMLParser(10)    >>> parser.feed(text)    >>> parser.get_summary(u'...')    u'<p>Higuys:Thi...</p>'  """  def __init__(self, count):    HTMLParser.__init__(self)    self.count = count    self.summary = u''  def feed(self, data):    """Only accept unicode `data`"""    assert(isinstance(data, unicode))    HTMLParser.feed(self, data)  def handle_data(self, data):    more = self.count - len(self.summary)    if more > 0:      # Remove possible whitespaces in `data`      data_without_whitespace = u''.join(data.split())      self.summary += data_without_whitespace[0:more]  def get_summary(self, suffix=u'', wrapper=u'p'):    return u'<{0}>{1}{2}</{0}>'.format(wrapper, self.summary, suffix)if __name__ == '__main__':  import doctest  doctest.testmod()

HTMLParser (or BeautifulSoup) is more suitable for complex HTML abstract extraction functions. For the simple HTML abstract extraction functions described above, there are actually more concise implementation solutions (compared with SummaryHTMLParser ):

#!/usr/bin/env python# -*- coding: utf-8 -*-"""Get a raw summary of the HTML-format document"""import redef get_summary(text, count, suffix=u'', wrapper=u'p'):  """A simpler implementation (vs `SummaryHTMLParser`).    >>> text = u'<p>Hi guys:</p><p>This is a example using SummaryHTMLParser.</p>'    >>> get_summary(text, 10, u'...')    u'<p>Higuys:Thi...</p>'  """  assert(isinstance(text, unicode))  summary = re.sub(r'<.*?>', u'', text) # key difference: use regex  summary = u''.join(summary.split())[0:count]  return u'<{0}>{1}{2}</{0}>'.format(wrapper, summary, suffix)if __name__ == '__main__':  import doctest  doctest.testmod()

I hope this article will help you with Python programming.