A deep understanding of magic methods in Python and a deep understanding of python

A deep understanding of magic methods in Python and a deep understanding of python

I have been in touch with Python for a while, and I have been in touch with many Python-related frameworks and modules. I hope to share with you the Design and Implementation I have come into contact, so I took a small "Charming Python" label and gave myself a start. I hope you will criticize and correct me a lot. :)

From flask import request

Flask is a very popular Python Web framework. I have also written some projects, large and small. Flask has a feature that I like very much, no matter where it is, if you want to obtain the current request object, you can simply:
Copy codeThe Code is as follows:
From flask import request

# Retrieving content from the current request
Request. args
Request. forms
Request. cookies
......
It is very easy to remember and friendly to use. However, the implementation behind the simple code is a little more complicated. Follow my articles to see the mysteries of this article!

Two questions?

Before proceeding, let's raise two questions:

Question 1: The request looks like a static class instance. Why can we directly use an expression like request. args to obtain the args attribute of the current request instead of the following:
Copy codeThe Code is as follows:
From flask import get_request

# Retrieving the current request
Request = get_request ()
Get_request (). args
In this way? How does flask map a request to the current request object?

Question 2: In a real production environment, there may be many threads (or coroutines) under the same worker process, as I just mentioned, how does a request instance work normally in such an environment?

To know the secrets, we can only read the flask source code.

Source code, source code, or source code

First, let's open the flask source code and start with _ init _. py to see how the request came out:
Copy codeThe Code is as follows:
# File: flask/_ init _. py
From. globals import current_app, g, request, session, _ request_ctx_stack

# File: flask/globals. py
From functools import partial
From werkzeug. local import LocalStack, LocalProxy

Def _ lookup_req_object (name ):
Top = _ request_ctx_stack.top
If top is None:
Raise RuntimeError ('Working outside of request context ')
Return getattr (top, name)

# Context locals
_ Request_ctx_stack = LocalStack ()
Request = LocalProxy (partial (_ lookup_req_object, 'request '))

We can see that the flask request is from globals. introduced by py, and the Code for the request defined here is request = LocalProxy (partial (_ lookup_req_object, 'request ')), if you do not know what partial is, you need to know about partial first.

However, we can simply understand that partial (func, 'request') uses 'request' as the first default parameter of func to generate another function.

Therefore, partial (_ lookup_req_object, 'request') can be understood:

Generate a callable function. This function obtains the first RequestContext object at the top of the stack from the LocalStack object _ request_ctx_stack, and then returns the request attribute of this object.

The LocalProxy under werkzeug caught our attention. Let's see what it is:

Copy codeThe Code is as follows:
@ Implements_bool
Class LocalProxy (object ):
"Acts as a proxy for a werkzeug local. Forwards all operations
A proxied object. The only operations not supported for forwarding
Are right handed operands and any kind of assignment.
......

After reading the previous several introductions, you can see what it mainly does. As the name suggests, LocalProxy is mainly a Proxy, a Proxy for the werkzeug Local object service. He "Forwards" all operations to the objects it acts as proxies.

So how is this Proxy implemented through Python? The answer is in the source code:
Copy codeThe Code is as follows:
# For convenience, I have deleted and modified the code

@ Implements_bool
Class LocalProxy (object ):
_ Slots _ = ('_ local',' _ dict _ ',' _ name __')

Def _ init _ (self, local, name = None ):
# Note that the _ setattr _ method is used
# The "_ LocalProxy _ local" attribute is set to local. You may be curious
# Why is this attribute name so strange? In fact, this is because Python does not support real
# Private member. For details, refer to the official documentation:
# Http://docs.python.org/2/tutorial/classes.html#private-variables-and-class-local-references
# Here you just need to regard it as self. _ local = local :)
Object. _ setattr _ (self, '_ LocalProxy _ local', local)
Object. _ setattr _ (self, '_ name _', name)

Def _ get_current_object (self ):
"""
Obtain the real object of the current proxy. Generally, this method is not called automatically unless you
For some performance reasons, you need to obtain the real object to be proxy, or you need to use it for another
Location.
"""
# Here, we mainly determine whether the proxy object is a werkzeug Local object.
# This logic is not used in the process.
If not hasattr (self. _ local, '_ release_local __'):
# From LocalProxy (partial (_ lookup_req_object, 'request ')
# By calling the self. _ local () method, we obtain partial (_ lookup_req_object, 'request ')()
# That is, ''_ request_ctx_stack.top.request''
Return self. _ local ()
Try:
Return getattr (self. _ local, self. _ name __)
T AttributeError:
Raise RuntimeError ('no object bound to % s' % self. _ name __)

# The next step is a piece of Python magic method. Is the Local Proxy overloaded (almost )? All Python
# The built-in magic method directs all operations related to him to _ get_current_object ()
# The returned object is the real proxy object.

......
_ Setattr _ = lambda x, n, v: setattr (x. _ get_current_object (), n, v)
_ Delattr _ = lambda x, n: delattr (x. _ get_current_object (), n)
_ Str _ = lambda x: str (x. _ get_current_object ())
_ Lt _ = lambda x, o: x. _ get_current_object () <o
_ Le _ = lambda x, o: x. _ get_current_object () <= o
_ Eq _ = lambda x, o: x. _ get_current_object () = o
_ Ne _ = lambda x, o: x. _ get_current_object ()! = O
_ Gt _ = lambda x, o: x. _ get_current_object ()> o
_ Ge _ = lambda x, o: x. _ get_current_object ()> = o
......

Here, we can answer the second question at the beginning of the article. We do not need to call methods like get_request () to obtain the current request object, it is the credit of LocalProxy.

LocalProxy acts as a proxy and uses custom magic methods. Proxy for all our request operations to point to the real request object.

Now we know that request. args is not as simple as it looks.

Now, let's take a look at the second question. In a multi-threaded environment, how does the request work normally? Let's go back to globals. py:
Copy codeThe Code is as follows:
From functools import partial
From werkzeug. local import LocalStack, LocalProxy

Def _ lookup_req_object (name ):
Top = _ request_ctx_stack.top
If top is None:
Raise RuntimeError ('Working outside of request context ')
Return getattr (top, name)

# Context locals
_ Request_ctx_stack = LocalStack ()
Request = LocalProxy (partial (_ lookup_req_object, 'request '))

The key to the problem lies in the _ request_ctx_stack object. Let's find the source code of LocalStack:

Copy codeThe Code is as follows:
Class LocalStack (object ):

Def _ init _ (self ):
# Actually, LocalStack uses another Local class.
# Some of its key methods are also proxies to this Local class
# Compared with the Local class, it implements some methods related to the Stack "Stack", such as push and pop.
# So we only need to look at the Local Code directly.
Self. _ local = Local ()

......

@ Property
Def top (self ):
"""
Returns the object at the top of the stack.
"""
Try:
Return self. _ local. stack [-1]
T (AttributeError, IndexError ):
Return None

# So when we call _ request_ctx_stack.top, we actually call _ request_ctx_stack. _ local. stack [-1].
# Let's take a look at how the Local class is implemented, but before that, let's take a look at the get_ident method shown in the detail section.

# First try to import the getcurrent method from greenlet, because if flask runs in a container like gevent
# Therefore, all requests use greenlet as the minimum unit, rather than the thread.
Try:
From greenlet import getcurrent as get_ident
Failed t ImportError:
Try:
From thread import get_ident
Failed t ImportError:
From _ thread import get_ident

# In short, the get_ident method will return the current coroutine/thread ID, which is unique for each request

Class Local (object ):
_ Slots _ = ('_ storage _', '_ ident_func __')

Def _ init _ (self ):
Object. _ setattr _ (self, '_ storage __',{})
Object. _ setattr _ (self, '_ ident_func _', get_ident)

......

# The Key to the problem is that the Local class reloads the _ getattr _ and _ setattr _ magic methods.

Def _ getattr _ (self, name ):
Try:
# Here we call self. _ ident_func _ (), which is the current unique ID.
# As the key of _ storage _
Return self. _ storage _ [self. _ ident_func _ ()] [name]
Failed t KeyError:
Raise AttributeError (name)

Def _ setattr _ (self, name, value ):
Ident = self. _ ident_func __()
Storage = self. _ storage __
Try:
Storage [ident] [name] = value
Failed t KeyError:
Storage [ident] = {name: value}

......

# After the two magic methods are reloaded

# Local (). some_value is no longer as simple as it looks:
# First, call the get_ident method to obtain the ID of the currently running thread/coroutine.
# Then obtain the some_value attribute in the ID space, as shown in the following code:
#
# Local (). some_value-> Local () [current_thread_id ()]. some_value
#
# This is also true when setting attributes.

Through these analyses, I believe that question 2 has also been resolved. By using the current thread/coroutine ID and reloading some magic methods, flask enables different worker threads to use their own stack objects. This ensures the normal operation of the request.

Speaking of this, this article is almost done. We can see that for the convenience of users, developers who use frameworks and tools need to pay a lot of extra work. Sometimes, magic in some languages is unavoidable, python also has quite good support in this regard.

All we need to do is to learn and master those magic parts in Python, and use magic to make your code simpler and easier to use.

But remember, although magic is dazzling, never abuse it.

What is the significance of PYTHON magic?

The magic method is to let you change it. Of course it doesn't make sense.

How is the private method in Python expressed?

>>> Class Template ():
Def ___ haha ():
Pass

>>> T = Template ()
>>> Dir (t)
['_ Template ___ hahaha',' _ doc _ ',' _ mydule _ ']
When a method starts with _, it will be considered as a private method and cannot be called externally. In fact, it is because python has changed its name, And the ___ haha method in the above formula has become
_ + Class name + original method name form