Parse the request object usage in the Python Flask framework from the source code, pythonflask

Parse the request object usage in the Python Flask framework from the source code, pythonflask

From flask import request
Flask is a very popular Python Web framework. I have also written some projects, large and small. Flask has a feature that I like very much, no matter where it is, if you want to obtain the current request object, you can simply:
Get content from current request:

• Method: Start row, metadata
• Host: Start row, metadata
• Path: Start row, metadata
• Environ: SERVER_PROTOCOL is the starting row and metadata.
• Headers: Header, metadata
• Data: body, metadata
• Remote_addr: client address
• Args: The GET parameter in the request link.
• Form: parameters in form submission, after Parsing
• Values: a set of args and forms
• Json: body Data in json format.
• Cookies: links to cookies

The Request object is very detailed in parameter classification. Pay attention to the differences between args, form, valeus, and json. Of course, the most secure and original method is to parse data by yourself.

Another thing to note is that some attribute types are not Python standard dict, but MultiDict or CombinedMultiDict. This is to cope with the repeated parameters in the HTTP protocol. Therefore, pay attention to the features of these objects when taking values. For example, the items returned by the. get () and. get_list () methods are different.
It is very easy to remember and friendly to use. However, the implementation behind the simple code is a little more complicated. Follow my articles to see the mysteries of this article!

Two questions?
Before proceeding, let's raise two questions:

Question 1: The request looks like a static class instance. Why can we directly use an expression like request. args to obtain the args attribute of the current request instead of the following:

From flask import get_request # obtain the current requestrequest = get_request (). args

In this way? How does flask map a request to the current request object?

Question 2: In a real production environment, there may be many threads (or coroutines) under the same worker process, as I just mentioned, how does a request instance work normally in such an environment?

To know the secrets, we can only read the flask source code.

Source code, source code, or source code
First, let's open the flask source code and start with _ init _. py to see how the request came out:

# File: flask/__init__.pyfrom .globals import current_app, g, request, session, _request_ctx_stack# File: flask/globals.pyfrom functools import partialfrom werkzeug.local import LocalStack, LocalProxydef _lookup_req_object(name):  top = _request_ctx_stack.top  if top is None:    raise RuntimeError('working outside of request context')  return getattr(top, name)# context locals_request_ctx_stack = LocalStack()request = LocalProxy(partial(_lookup_req_object, 'request'))

We can see that the flask request is from globals. introduced by py, and the Code for the request defined here is request = LocalProxy (partial (_ lookup_req_object, 'request ')), if you do not know what partial is, you need to know about partial first.

However, we can simply understand that partial (func, 'request') uses 'request' as the first default parameter of func to generate another function.

Therefore, partial (_ lookup_req_object, 'request') can be understood:

Generate a callable function. This function obtains the first RequestContext object at the top of the stack from the LocalStack object _ request_ctx_stack, and then returns the request attribute of this object.

The LocalProxy under werkzeug caught our attention. Let's see what it is:

@implements_boolclass LocalProxy(object):  """Acts as a proxy for a werkzeug local. Forwards all operations to  a proxied object. The only operations not supported for forwarding  are right handed operands and any kind of assignment.  ... ...

After reading the previous several introductions, you can see what it mainly does. As the name suggests, LocalProxy is mainly a Proxy, a Proxy for the werkzeug Local object service. He "Forwards" all operations to the objects it acts as proxies.

So how is this Proxy implemented through Python? The answer is in the source code:

# For convenience, I have deleted and modified the code. @ implements_boolclass LocalProxy (object): _ slots _ = ('_ local ', '_ dict _', '_ name _') def _ init _ (self, local, name = None ): # note that the # "_ LocalProxy _ local" attribute of self is set to local through the _ setattr _ method, you may wonder why # This attribute name is so strange, in fact this is because Python does not support true # Private member, For details, refer to the official documentation: # http://docs.python.org/2/tutorial/classes.html#private-variables-and-class-local-references # Here you just need to put it when Do self. _ local = local: the object. _ setattr _ (self, '_ LocalProxy _ local', local) object. _ setattr _ (self, '_ name _', name) def _ get_current_object (self): "Get the real object currently being proxy, generally, this method is not called automatically, unless you need to obtain the real object to be proxy for some performance reasons, or you need to use it elsewhere. "# This mainly determines whether the proxy object is a werkzeug Local object. This logic is not used in our request # analysis. If not hasattr (self. _ local, '_ release_local _'): # From LocalProxy (partial (_ lookup_req_object, 'request') # by calling self. _ local () method, we get partial (_ lookup_req_object, 'request') () # That is, ''_ request_ctx_stack.top.request'' return self. _ local () try: return getattr (self. _ local, self. _ name _) doesn t AttributeError: raise RuntimeError ('no object bound to % s' % self. _ name _) # The next step is a piece of Python magic method. Lo Cal Proxy is overloaded (almost )? All Python # built-in magic methods allow all operations related to him to point to the objects returned by _ get_current_object () #, that is, the real proxy object. ...... _ Setattr _ = lambda x, n, v: setattr (x. _ get_current_object (), n, v) _ delattr _ = lambda x, n: delattr (x. _ get_current_object (), n) _ str _ = lambda x: str (x. _ get_current_object () _ lt _ = lambda x, o: x. _ get_current_object () <o _ le _ = lambda x, o: x. _ get_current_object () <= o _ eq _ = lambda x, o: x. _ get_current_object () = o _ ne _ = lambda x, o: x. _ get_current_object ()! = O _ gt _ = lambda x, o: x. _ get_current_object ()> o _ ge _ = lambda x, o: x. _ get_current_object ()> = o ......

Here, we can answer the second question at the beginning of the article. We do not need to call methods like get_request () to obtain the current request object, it is the credit of LocalProxy.

LocalProxy acts as a proxy and uses custom magic methods. Proxy for all our request operations to point to the real request object.

Now we know that request. args is not as simple as it looks.

Now, let's take a look at the second question. In a multi-threaded environment, how does the request work normally? Let's go back to globals. py:

from functools import partialfrom werkzeug.local import LocalStack, LocalProxydef _lookup_req_object(name):  top = _request_ctx_stack.top  if top is None:    raise RuntimeError('working outside of request context')  return getattr(top, name)# context locals_request_ctx_stack = LocalStack()request = LocalProxy(partial(_lookup_req_object, 'request'))

The key to the problem lies in the _ request_ctx_stack object. Let's find the source code of LocalStack:

Class LocalStack (object): def _ init _ (self ): # In fact, LocalStack mainly uses another Local class # some of its key methods are also proxies to this Local class # compared with the Local class, it implements some methods related to the Stack "Stack", such as push, pop, and so on. so we only need to look at the Local Code directly to implement self. _ local = Local ()...... @ property def top (self): "return the object at the top of the stack" try: return self. _ local. stack [-1] T (AttributeError, IndexError): return None # So when we call _ request_ctx_stack.top, we actually call _ request_ctx_stack. _ local. st Ack [-1] # Let's see how the Local class is implemented, however, before that, let's take a look at the get_ident method that appears under the handler # first try to import the getcurrent method from greenlet, this is because if flask runs in a container such as gevent #, the request uses greenlet as the minimum unit, rather than the thread. Try: from greenlet import getcurrent as get_identexcept ImportError: try: from thread import get_ident failed t ImportError: from _ thread import get_ident # In short, the get_ident method will return the current coroutine/thread ID, this is the only class Local (object): _ slots _ = ('_ storage _', '_ ident_func _') for each request __') def _ init _ (self): object. _ setattr _ (self, '_ storage _', {}) object. _ setattr _ (self, '_ ident_func _', get_ident )...... # The Key to the problem is that the Local class has overloaded the _ getattr _ and _ setattr _ magic methods def _ getattr _ (self, name): try: # Here we call self. _ ident_func _ (), which is the current unique ID # used as the key return self of _ storage. _ storage _ [self. _ ident_func _ ()] [name] doesn t KeyError: raise AttributeError (name) def _ setattr _ (self, name, value): ident = self. _ ident_func _ () storage = self. _ storage _ try: storage [ident] [name] = value into T KeyError: storage [ident] = {name: value }...... # After the two magic methods are reloaded # Local (). some_value is no longer as simple as it looks: # first, call the get_ident method to obtain the currently running thread/coroutine ID # And then obtain the some_value attribute in the ID space, as shown in the following figure: # Local (). some_value-> Local () [current_thread_id ()]. some_value # This is also true when setting attributes

Through these analyses, I believe that question 2 has also been resolved. By using the current thread/coroutine ID and reloading some magic methods, flask enables different worker threads to use their own stack objects. This ensures the normal operation of the request.

Speaking of this, this article is almost done. We can see that for the convenience of users, developers who use frameworks and tools need to pay a lot of extra work. Sometimes, magic in some languages is unavoidable, python also has quite good support in this regard.

All we need to do is to learn and master those magic parts in Python, and use magic to make your code simpler and easier to use.

But remember, although magic is dazzling, never abuse it.