Python Crawler Primer (3)--build API with requests

This article mainly introduces you to the use of Python crawler requests building knowledge of the API, the text through the sample code introduced in very detailed, for everyone has a certain reference learning value, the need for friends below to see together.

Objective

In the Reptile Series article elegant HTTP library requests introduced the use of requests, this time we use the requests to build a knowledge API, features include: Send messages, articles like, user attention, and so on, because any user operations involved in the function need to log in before the operation, So before reading this article, it is recommended to understand the Python simulation before logging in. Now suppose you already know how to log in using the requests simulation.

Thinking analysis

The process of sending a private message is that the browser sends an HTTP request to the server, including the request URL, the request header, and the body of the request, as long as the information is clear, it is easy to use requests to simulate the browser to send private messages.

Open a Chrome browser, find a user, click Send a private message, and trace the network request process of private messages.

First look at the request header information

There is a cookie login in the header header, there is also a authorization field, which is used for user authentication, and this field is also present in the cookie (in order to prevent the disclosure of cookie information, I played Mosaic), requests request this The information must be carried.

Take a look at the URL of the request and the request body

The request URL is www.zhihu.com/api/v4/messages, the request method is POST, the request body

{"Type": "Common", "content": "Hello, I am Pythoner", "Receiver_hash": "1da75b85900e00adb072e91c56fd9149"}

The request body is a JSON string, type and content are well understood, but Receiver_hash is not known and needs to be further determined, but you should guess that this is a field similar to the user ID.

So now the question, how to find the user's id through the URL of the user's homepage? In order to complete the entire process of simulating private messages, I specifically registered a small size.

If you do not have the spare cell phone number, you can use Google search "receive SMS online", online many provide free online to receive SMS mobile phone number, I registered the small home: Https://www.zhihu.com/people/xiaoxiaodouzi

Try to focus on the trumpet first, and then find the trumpet in the list of my concerns, and when I move the mouse over the small head, I find an HTTP network request.

The request URL is Www.zhihu.com/api/v4/members/xiaoxiaodouzi, and the latter part of the URL "xiaoxiaodouzi" corresponds to the back part of the small home page URL, which we call Url_token.

The return data of the interface is the personal public information of the user.

{  ... "id": "1da75b85900e00adb072e91c56fd9149", "Favorite_count": 0, "Voteup_count": 0, "Commercial_question_count": 0, "url _token ":" Xiaoxiaodouzi "," type ":" People "," Avatar_url ":" https://pic1.zhimg.com/ V2-ca13758626bd7367febde704c66249ec_is.jpg "," is_active ": 1492224390," name ":" \u6211\u662f\u5c0f\u53f7 "," url ":" http://www.zhihu.com/api/v4/people/1da75b85900e00adb072e91c56fd9149 "," Gender ":-1 ...}

We can clearly see that there is a field of ID, as we have guessed before, the Receiver_hash field inside the private messages is the user's ID.

Code implementation

To this we put the idea of the function of private messages clear, code implementation is the matter of the inevitable.

User Information

In order to get the Receiver_hash dictionary required for the private message interface, we first need to obtain the user information, which contains the ID value used.

@need_logindef User (self, url_token): "" "gets the information:p Aram Url_token:url_token is the later part of the user's home page URL, for example: https://www.zhihu.com/ People/xiaoxiaodouzi Url_token is Xiaoxiaodouzi:return:dict "" "Response = Self._session.get (Url.profile (Url_token)) Return Response.json ()

Send private messages

@need_logindef send_message (self, user_id, content): "" "sends private messages to the specified user:p Aram user_id: User ID:p Aram content: Private Messages" "" Data = {" Type ":" Common "," content ": Content," Receiver_hash ": user_id} response = Self._session.post (Url.message (), Json=data) data = Response.json () If Data.get ("error"): Self.logger.info ("Private message send failed,%s"% Data.get ("error"). Get ("message")) Else: Self.logger.info ("Send Success") return data

The above two methods put in a class called Zhihu, I only listed the key code, involving the @need_login is a user-certified adorner, indicating that the method needs to log in before the operation. Careful you may find that I did not specify the Header field in each request, at that time because I initialized it in the init.py method.

def init (self): Self._session = requests.session () self._session.verify = False Self._session.headers = {"Host": "Www.zhi Hu.com ",    " Referer ":" https://www.zhihu.com/",    ' user-agent ': ' mozilla/5.0 (Macintosh;  Intel Mac OS X 10_10_5) applewebkit/537.36 "      (khtml, like Gecko) chrome/56.0.2924.87 ',    } self._session.cookies = Cookiejar. Lwpcookiejar (Filename=cookie_filename) try:self._session.cookies.load (ignore_discard=true) Except:pass

Call execution

From Zhihu import zhihuif name = = ' main ': Zhihu = Zhihu () profile = Zhihu.user ("Xiaoxiaodouzi") _id = Profile.get ("id") en Ihu.send_message (_id, "Hello, this is the greeting from the Zen of Python")

After execution, the trumpet successfully received the private message I sent.

Finally, we can follow similar ideas to focus on users, like and other functions to achieve.

"Recommended"

1. Python Crawler Primer (5)--Regular Expression Example tutorial

2. Python crawler Primer (4)--Detailed HTML text parsing library BeautifulSoup

3. Python crawler Primer (2)--http Library requests

4. Python crawler Primer (1)--Quick understanding of HTTP protocol

5. Share an instance of login using Python crawler simulation