Python Crawler (2)--Crawling microblogging content __python

Source: Internet
Author: User

This paper is a simple way to capture the content information of Huawei's official microblogging blog. First grab the cookie that logs on to the microblog, and then use a cookie to log on to the microblog.
The specific code looks like this:

#-*-Coding:utf-8-*-
"" "
Created on Sun Apr 14:16:32 2017
@author: Zch" ""

import requests
F Rom BS4 import beautifulsoup
import time
import pandas as PD

#放入cookie信息
cook = {"Cookie": "_t_wm= ..."}< c10/> #爬取华为终端官方微博的内容
url = "Https://weibo.cn/huaweidevice"

html = requests.get (url,cookies=cook). Content

#使用Beautiful来解析网页内容.
soup =beautifulsoup (HTML, "Html.parser")

r = Soup.findall (' span ', attrs={' class ': ' CTT '}) for e-in
r:< C17/>print (E.text)

The results of the run are as follows:

Of course, the code above can only crawl a small amount of content information at a time, but also very simple. Then we should find a way to solve the problem of multiple pages continuous crawling and automatic classification of storage .

Contact Us

The content source of this page is from Internet, which doesn't represent Alibaba Cloud's opinion; products and services mentioned on that page don't have any relationship with Alibaba Cloud. If the content of the page makes you feel confusing, please write us an email, we will handle the problem within 5 days after receiving your email.

If you find any instances of plagiarism from the community, please send an email to: info-contact@alibabacloud.com and provide relevant evidence. A staff member will contact you within 5 working days.

A Free Trial That Lets You Build Big!

Start building with 50+ products and up to 12 months usage for Elastic Compute Service

  • Sales Support

    1 on 1 presale consultation

  • After-Sales Support

    24/7 Technical Support 6 Free Tickets per Quarter Faster Response

  • Alibaba Cloud offers highly flexible support services tailored to meet your exact needs.