Well, meego's Chinese core site, miqu.com, has published a new blog.
I have introduced youPyqueryNext we will introduceBeautifulsoupBeautiful soup is a built-in Python web analysis tool named beautiful butterfly. Oh, sometimes it's as beautiful as a butterfly.
The first section introduces:
Beautiful soupIs a python html/XML processor designed to quickly convert web page capture. The following features supportBeautiful soup:
- Beautiful soupNo, even if you give him a broken tag. He generates a conversion DOM tree and tries to make it consistent with the meaning of your original document. This measure can usually meet your data collection needs.
- Beautiful soupProvides some simple methods and Python-like syntax to find, search, and modify a conversion tree: A toolset helps you parse a tree and release what you need. You do not need to create your own parsing tools for each application.
- Beautiful soup AutomaticDocuments to be sent inConvert to unicode encodingAnd in the output to the UTF-8 ,. Unless this document does not specify the encoding method or the beautiful Soup does not automatically detect the encoding method, you need to manually specify the encoding method, otherwise you do not need to consider the encoding problem.
Beautiful soupConvert anything you give him, and then do those conversion tasks for you. You can run the command to "find all links", "Find all links whose classes are externallink", or "Find all links whose URLs match" foo.com ", even "find the headers in bold and return them to me".
Valuable data in poorly designed websites can be locked once by you. It would have taken several hours to work, and you can use beautiful Soup within minutes.
Let's get started quickly:
First, reference the package:
- From beautifulsoup import beautifulsoup # for processing html
- From beautifulsoup import beautifulstonesoup # for processing XML
- Import beautifulsoup # To get everything [/font] [/color]
CopyCode
The following code demonstrates the basic usage of beautiful Soup. You can copy and paste the code and run it yourself.
- From beautifulsoup import beautifulsoup
-
- Import re
-
-
- Doc = ['<HTML>
-
- '<Body> <p id = "firstpara" align = "center"> This is paragraph <B> one </B> .',
-
- '<P id = "secondpara" align = "blah"> This is paragraph <B> two </B> .',
-
- '</Html>']
-
- Soup = beautifulsoup (''. Join (DOC ))
-
-
- Print soup. pretloads ()
-
- # <HTML>
-
- # <Head>
-
- # <Title>
-
- # Page title
-
- # </Title>
-
- # </Head>
- # <Body>
-
- # <P id = "firstpara" align = "center">
-
- # This is paragraph
-
- # <B>
-
- # One
-
- # </B>
-
- #.
-
- # </P>
-
- # <P id = "secondpara" align = "blah">
-
- # This is paragraph
-
- # <B>
-
- # Two
-
- # </B>
-
- #.
-
- # </P>
-
- # </Body>
-
- # </Html>
Copy code
The following is a method to parse the document:
- Soup. Contents [0]. Name
- # U'html'
- Soup. Contents [0]. Contents [0]. Name
- # U'head'
- Head = soup. Contents [0]. Contents [0]
- Head. Parent. Name
- # U'html'
- Head. Next
- # <Title> page title </title>
- Head. nextsibling. Name
- # U'body'
- Head. nextsibling. Contents [0]
- # <P id = "firstpara" align = "center"> This is paragraph <B> one </B>. </P>
- Head. nextsibling. Contents [0]. nextsibling
- # <P id = "secondpara" align = "blah"> This is paragraph <B> two </B>. </P>
Copy code
The next step is to search for tags contained in a document in a dozen ways or tags containing specified attributes.
- Titletag = soup.html. Head. Title
- Titletag
- # <Title> page title </title>
- Titletag. String
- # U'page title'
- Len (soup ('P '))
- #2
- Soup. findall ('P', align = "center ")
- # [<P id = "firstpara" align = "center"> This is paragraph <B> one </B>. </P>]
- Soup. Find ('P', align = "center ")
- # <P id = "firstpara" align = "center"> This is paragraph <B> one </B>. </P>
- Soup ('P', align = "center") [0] ['id']
- # U'firstpara'
- Soup. Find ('P', align = Re. Compile ('^ B. *') ['id']
- # U'secondpara'
- Soup. Find ('P'). B. String
- # U'one'
- Soup ('P') [1]. B. String
- # U'two'
Copy code
You can also modify the document
- Titletag ['id'] = 'thetitle'
-
- Titletag. Contents [0]. replacewith ("New title ")
-
- Soup.html. Head
-
- # <Head> <title ID = "thetitle"> New title </title>
-
-
- Soup. P. Extract ()
-
- Soup. prettasks ()
-
- # <HTML>
-
- # <Head>
-
- # <Title ID = "thetitle">
-
- # New title
-
- # </Title>
-
- # </Head>
-
- # <Body>
-
- # <P id = "secondpara" align = "blah">
-
- # This is paragraph
-
- # <B>
-
- # Two
-
- # </B>
- #.
-
- # </P>
-
- # </Body>
-
- # </Html>
-
-
- Soup. P. replacewith (soup. B)
-
- # <HTML>
-
- # <Head>
-
- # <Title ID = "thetitle">
-
- # New title
-
- # </Title>
-
- # </Head>
-
- # <Body>
-
- # <B>
-
- # Two
-
- # </B>
-
- # </Body>
-
- # </Html>
-
-
- Soup. Body. insert (0, "This page used to have ")
-
- Soup. Body. insert (2, "& lt; P & gt; tags! ")
- Soup. Body
-
- # <Body> This page used to have <B> two </B> & lt; P & gt; tags! </Body>
Copy code
Finally, we will provide you with the beautiful Soup documentation. Hope to help you.
ReprintedArticle, Please indicate fromMiquyi
