Introduction to the use of Python black magic descriptors

Introduction

Descriptors (descriptor) is a deep but important black magic in the Python language, which is widely used in the kernel of the Python language, and mastering the descriptor will add an extra skill to the Python programmer 's toolbox. In this article I will describe the definition of descriptors and some common scenarios, and at the end of the article I will add that the getattr getattribute getitem three magic methods that also involve property access.

Definition of Descriptor

Descrget (self, obj, Objtype=none)--valuedescr.set (self, obj, value)--nonedescr.delete (self, obj)--None

object attributeThis class can be called a descriptor class as long as one (object property) defines any of the three methods above.

Descriptor Basics

In the following example, we create a RevealAcess class and implement the get method, and now the class can be called a descriptor class.

Class Revealaccess (object):    def get (self, obj, objtype):        print ("Self" revealaccess: {} '. Format (self))        Print (' self: {}\nobj: {}\nobjtype: {} '. Format (self, obj, ObjType)) class MyClass (object):    x = revealaccess ()    def test (self):        print (' MyClass: {} '. Format (self))

EX1 Instance Properties

Next we take a look at the meaning of the get various parameters of the method, in the following example, the self instance of Revealaccess class X, that is, obj the MyClass class instance M, as the objtype name implies is the MyClass class itself. As you can see from the output statement, the m.x Access descriptor x invokes the get method.

>>> m = MyClass () >>> m.test () Self in MyClass: <main. MyClass object at 0x7f19d4e42160>>>> m.xself in revealaccess: <main. Revealaccess object at 0x7f19d4e420f0>self: <main. Revealaccess object at 0x7f19d4e420f0>obj: <main. MyClass object at 0x7f19d4e42160>objtype: <class ' main. MyClass ' >

EX2 class Properties

If the property is accessed directly through the class x , then the obj Direct is none, which is better understood, because there is no instance of MyClass.

>>> myclass.xself in revealaccess: <main. Revealaccess object at 0x7f53651070f0>self: <main. Revealaccess object at 0x7f53651070f0>obj:noneobjtype: <class ' main. MyClass ' >

The principle of descriptors

Descriptor triggering

In the above example, we have enumerated the use of descriptors from the perspective of instance properties and class attributes, so let's examine the internal principle in detail:

• In the case of 实例属性 access, the GetAttribute method of the base class object is actually called, in this method the OBJ.D is translated type(obj).dict['d'].get(obj, type(obj)) .

• In the case of 类属性 access, it is equivalent to calling the GetAttribute method of the meta-class type, which translates cls.d into cls.dict['d'].get(None, cls) , where the obj of Get () is none, because there is no instance.

Briefly speaking the getattribute Magic method, this method will be called unconditionally when we access the properties of an object, with detailed details such as getattr getitem the difference between and, I will make an additional supplement at the end of the article, we do not delve into the moment.

Descriptor Precedence

First, the descriptors are divided into two types:

• This descriptor is called if an object defines both the get () and the Set () methods data descriptor .

• If an object defines only the Get () method, this descriptor is called non-data descriptor .

There are four things we do when we access properties:

• Data descriptor

• Instance Dict

• Non-data Descriptor

• GetAttr ()

Their priority size is:

Data descriptor > Instance dict > Non-data descriptor > GetAttr ()

What does that mean? That is, if the instance object obj appears with the same name data descriptor->d and instance attribute->d , obj.d d when accessing the property, because data descriptor has a higher priority, Python will call instead of type(obj).dict['d'].get(obj, type(obj)) calling obj.dict[' d '. However, if the descriptor is a non-data Descriptor,python it will be called obj.dict['d'] .

Property

A descriptor class is defined every time a descriptor is used, which can seem tedious. Python provides a concise way to add a data descriptor to a property.

Property (Fget=none, Fset=none, Fdel=none, Doc=none) and property attribute

Fget, Fset, and Fdel are class getter, setter, and Deleter methods respectively. We use the following example to illustrate how the property is used:

Class Account (Object):    def init (self):        self._acct_num = None    def get_acct_num (self):        return self._ Acct_num    def set_acct_num (self, value):        self._acct_num = value    def del_acct_num (self):        del self._ Acct_num    Acct_num = Property (Get_acct_num, Set_acct_num, Del_acct_num, ' _acct_num property. ')

If Acct is an instance of account, Acct.acct_num will call Getter,acct.acct_num = value will call Setter,del Acct_num.acct_num will call deleter.

>>> acct = account () >>> acct.acct_num = 1000>>> acct.acct_num1000

Python also provides @property adorners, which can be used to create attributes for simple scenarios. A Property object has the Getter,setter and deleter adorner methods that can be used to create a copy of the property through the corresponding accessor function of the decorated function.

Class Account (Object):    def init:        self._acct_num = None    @property     # The _acct_num property. The Decorator creates a Read-only property    def acct_num (self):        return self._acct_num    @acct_num. Setter    # The _acct_num property setters makes the property writeable    def set_acct_num (self, value):        self._acct_num = value< c10/> @acct_num. Deleter    def del_acct_num (self):        del self._acct_num

If you want to make the property read-only, simply remove the setter method.

Creating descriptors at run time

We can add property properties at run time:

class Person (object): Def addproperty (self, attribute): # Create local setter and getter with a particular attribute name getter = Lambda Self:self._getproperty (attribut e) setter = Lambda self, value:self._setproperty (attribute, value) # Construct property attribute and add I                                                    T to the class SetAttr (Self.class, Attribute, property (Fget=getter, \ Fset=setter, \ doc= "Auto-generated method")) def _setproperty ( Self, attribute, value): Print ("Setting: {} = {}". Format (attribute, value)) SetAttr (self, ' _ ' + attribute, V Alue.title ()) def _getproperty (self, attribute): Print ("Getting: {}". Format (attribute)) return getattr (SE LF, ' _ ' + attribute) 

>>> user = person () >>> user.addproperty (' name ') >>> user.addproperty (' phone ') >>> User.Name = ' John smith ' Setting:name = John smith>>> user.phone = ' 12345 ' setting:phone = 12345>>> user . Namegetting:name ' John Smith ' >>> user.dict{' _phone ': ' 12345 ', ' _name ': ' John Smith '}

Static Methods and Class methods

We can use descriptors to simulate the implementation of @staticmethod and in Python @classmethod . Let's start by looking at the following table:

from an Object

Transformation	called	called from a Class
function	F (obj, *args)	F (*args)
Staticmethod	F (*args)	F (*args)
Classmethod	F (Type (obj), *args)	F (Klass, *args)

Static methods

For static methods f . c.fand C.f is equivalent, are directly query object.getattribute(c, ‘f’) or object.getattribute(C, ’f‘) . One obvious feature of static methods is that there are no self variables.

What is the use of static methods? Suppose there is a container class that handles specialized data, which provides methods to calculate the average, median, and so on, which are dependent on the corresponding data. However, there may be some methods in the class that do not depend on the data, and we can declare these methods as static methods at this time, which can also improve the readability of the code.

Use non-data descriptors to simulate the implementation of a static method:

Class Staticmethod (object):    def init (self, f):        self.f = f    def get (self, obj, Objtype=none):        return Self.f

Let's apply it:

Class MyClass (object):    @StaticMethod    def get_x (x):        return XPrint (myclass.get_x)  # output: 100

Class method

Python's @classmethod @staticmethod usage is somewhat similar, but it's a bit different, and when some methods just need to get the 类的引用 appropriate data in the class, they have to use Classmethod.

Use non-data descriptors to simulate the implementation of a class method:

Class Classmethod (object):    def init (self, f):        self.f = f    def get (self, obj, Klass=none):        If Klass is None :            klass = Type (obj)        def newfunc (*args):            return Self.f (Klass, *args)        return Newfunc

Other methods of magic

The first time I touched the Python magic method, I was also,,, the get getattribute getattr getitem difference between them, and they were all related to property access to the Magic method, which overrides getattr , getitem to construct a collection of their own class is very common, Let's look at some examples of their applications.

GetAttr

Python's default access to a property of a class/instance is invoked by the call, which is called getattribute getattribute unconditionally, and is called if it is not found getattr . If we want to customize a class, usually we should not rewrite getattribute , but should rewrite getattr , rarely see getattribute the situation of rewriting.

As you can see from the output below, a property getattribute is called when it cannot be found getattr .

In [1]: Class Test (object):    ...:     def getattribute (self, item):    ...:         print (' Call getattribute ')    ...:         return super (Test, self). getattribute (item) ...    :     def getattr (self, item):    ...:         return ' Call GetAttr ' ...    : in [2]: Test (). Acall getattributeout[2]: ' Call getattr '

Application

For the default dictionary, Python is only supported in the form of obj['foo'] access, unsupported obj.foo form, and we can rewrite it to getattr let the dictionary also support obj['foo'] the Access form, which is a very classic common usage:

Class Storage (Dict): "" "    a Storage object is like a dictionary except ' Obj.foo ' can being used in    addition to ' OB j[' foo '] '.    ' "" def getattr (self, key):        try:            return Self[key]        except Keyerror as K:            raise Attributeerror (k)    def SetAttr (self, Key, value):        Self[key] = value    def delattr (self, key):        try:            del Self[key]        Except Keyerror as K:            raise Attributeerror (k)    def repr (self):        return ' <storage ' + dict.repr (self) + ' & gt; '

Let's use our custom version of the Enhanced Dictionary:

>>> s = Storage (a=1) >>> s[' a ']1>>> s.a1>>> S.A. = 2>>> s[' a ']2>>> Del S.a>>> S.A. Attributeerror: ' A '

GetItem

The getitem is used to [] get the elements in the object in the form of subscripts, and we getitem implement a list of our own by rewriting.

Class MyList (object):    def init (self, *args):        self.numbers = args    def getitem (self, item):        return Self.numbers[item]my_list = MyList (1, 2, 3, 4, 6, 5, 3) print my_list[2]

This implementation is very humble, does not support slice and step functions, please the reader to improve themselves, here I will not repeat.

Application

Below is getitem a reference to the use of the requests library, we have customized a dictionary class that ignores attribute capitalization.

The program is a little complicated, and I'll explain a little bit: because it's simple here, there's no need to use descriptors, so @property instead of using adorners, lower_keys the function is to convert 实例字典 all the keys in lowercase and store them in the dictionary self._lower_keys . The method is overridden getitem , and later we access a property that first converts the key to lowercase, and then does not directly access the instance dictionary, but instead accesses the dictionary self._lower_keys to find it. Assignment/delete operation because the instance dictionary will be changed, in order to maintain self._lower_keys and the instance dictionary synchronization, first clear self._lower_keys the content, we re-look for the key again when the call getitem will be re-create a new one self._lower_keys .

Class Caseinsensitivedict (dict):    @property    def lower_keys (self):        if isn't hasattr (self, ' _lower_keys ') or Not self._lower_keys:            Self._lower_keys = Dict ((K.lower (), K) to K in Self.keys ())        return Self._lower_keys    def _clear_lower_keys (self):        if hasattr (self, ' _lower_keys '):            self._lower_keys.clear ()    def contains ( Self, key):        return Key.lower () in Self.lower_keys    def getitem (self, key):        if key in self:            return Dict.getitem (self, Self.lower_keys[key.lower ()])    def setitem (self, Key, value):        Dict.setitem (self, key, Value)        Self._clear_lower_keys ()    def delitem (self, key):        Dict.delitem (self, key)        Self._lower_ Keys.clear ()    def get (self, Key, Default=none):        if key in self:            return Self[key]        else:            return Default

Let's call this class:

>>> d = caseinsensitivedict () >>> d[' Ziwenxie '] = ' Ziwenxie ' >>> d[' ziwenxie '] = ' Ziwenxie ' >>> print (d) {' Ziwenxie ': ' Ziwenxie ', ' Ziwenxie ' : ' Ziwenxie '}>>> print (d[' Ziwenxie ') ziwenxie# d[' ziwenxie '] = d[' Ziwenxie ']>>> print (d[' Ziwenxie ']) Ziwenxi