Python crawler crawl page source code is shown on this page

When crawling Web content, the Python crawler needs to crawl the content together with the content format, and then display it in its own web page, defining a variable HTML for the Django framework, with a variable value of HTML code.

Print (HTML)<div id=1>  &nbsp;&nbsp;my <br> is   &nbsp; JAY <br>  </div>

, we now want to take the contents of the Div, display in our own web page, spaces and line breaks, etc. are crawled in the form of HTML code. The final desired data is &nbsp;&nbsp;my <br>&nbsp; Name <br>is &nbsp; JAY <br> (1) First soup.string is not possible, because there is more than one sub-label under the DIV

 from Import BeautifulSoup ' Html.parser ' )>>> soup.string

You can see that the return value is null (2) using Get_text () is also not possible, because Get_text () gets the string is escaped, we want the native HTML code

>>> soup.get_text ()'  \ \xa0\xa0my  \ \xa0 name \ n is \xa0 JAY \ n'

But Get_text () is useful on many other occasions, it can get the text content of all descendants ' tags under the tag and can specify parameters

>>> soup.get_text ('| ')  # delimiter for all tag text contents ' \ \xa0\xa0my | \ \xa0 name |\n is \xa0 JAY |\n '>>> soup.get_text ('| ', strip=true)   # Remove the whitespace before and after the text content 'my|name|is \xa0 JAY' 

or use the soup.stripped_strings generator to get text content manually (3) can be used. Contents

>>> Content_soup =soup.div.contents>>>content_soup['\ \xa0\xa0my', <br/>,'\ \xa0 Name', <br/>,'\ n is \xa0 JAY', <br/>,'\ n']>>> content_soup = [Str (i) forIinchContent_soup]#Change all values in the list to string types>>> Content_text ="'. Join (Content_soup)#merge the list into a string>>>Content_text'\ \xa0\xa0my <br/> \ \xa0 name <br/>\n is \xa0 JAY <br/>\n'>>>Print(Content_text) My<br/>name<br/> isJAY <br/>

At this point, you can put the variables directly into the Web page

Python crawler crawl page source code is shown on this page