Magical descriptors in Python

Introduction

Descriptors (descriptor) is a deep but important black magic in the Python language, which is widely used in the kernel of the Python language, and mastering the descriptor will add an extra skill to the Python programmer 's toolbox. In this article I will describe the definition of descriptors and some common scenarios, and at the end of the article I will add that the __getattr __getattribute__ __getitem__ three magic methods that also involve property access.

Definition of Descriptor

Descr__get__ (self, obj, Objtype=none)--valuedescr.__set__ (self, obj, value)--nonedescr.__delete__ (self, obj )--None

object attributeThis class can be called a descriptor class as long as one (object property) defines any of the three methods above.

Descriptor Basics

In the following example, we create a RevealAcess class and implement the __get__ method, and now the class can be called a descriptor class.

Class Revealaccess (object):    def __get__ (self, obj, objtype):        print ("Self" revealaccess: {} '. Format (self))        print (' self: {}\nobj: {}\nobjtype: {} '. Format (self, obj, ObjType)) class MyClass (object):    x = revealaccess ()    def test (self):        print (' MyClass: {} '. Format (self))

EX1 Instance Properties

Next we take a look at the meaning of the __get__ various parameters of the method, in the following example, the self instance of Revealaccess class X, that is, obj the MyClass class instance M, as the objtype name implies is the MyClass class itself. As you can see from the output statement, the m.x Access descriptor x invokes the __get__ method.

>>> m = MyClass () >>> m.test () Self in MyClass: <__main__. MyClass object at 0x7f19d4e42160>>>> m.xself in revealaccess: <__main__. Revealaccess object at 0x7f19d4e420f0>self: <__main__. Revealaccess object at 0x7f19d4e420f0>obj: <__main__. MyClass object at 0x7f19d4e42160>objtype: <class ' __main__. MyClass ' >

EX2 class Properties

If the property is accessed directly through the class x , then the obj Direct is none, which is better understood, because there is no instance of MyClass.

>>> myclass.xself in revealaccess: <__main__. Revealaccess object at 0x7f53651070f0>self: <__main__. Revealaccess object at 0x7f53651070f0>obj:noneobjtype: <class ' __main__. MyClass ' >

The principle of descriptors

Descriptor triggering

In the above example, we have enumerated the use of descriptors from the perspective of instance properties and class attributes, so let's examine the internal principle in detail:

• In the case of 实例属性 access, the __getattribute__ method of the base class object is actually called, in this method the OBJ.D is translated type(obj).__dict__['d'].__get__(obj, type(obj)) .

• In the case of 类属性 access, it is equivalent to calling the __getattribute__ method of the meta-class type, which translates cls.d into cls.__dict__['d'].__get__(None, cls) , where the obj of __get__ () is none, because there is no instance.

Briefly speaking the __getattribute__ Magic method, this method will be called unconditionally when we access the properties of an object, with detailed details such as __getattr __getitem__ the difference between and, I will make an additional supplement at the end of the article, we do not delve into the moment.

Descriptor Precedence

First, the descriptors are divided into two types:

• This descriptor is called if an object defines both the __get__ () and the __set__ () methods data descriptor .

• If an object defines only the __get__ () method, this descriptor is called non-data descriptor .

There are four things we do when we access properties:

• Data descriptor

• Instance Dict

• Non-data Descriptor

• __GETATTR__ ()

Their priority size is:

Data descriptor > Instance dict > Non-data descriptor > __getattr__ ()

What does that mean? That is, if the instance object obj appears with the same name data descriptor->d and instance attribute->d , obj.d d when accessing the property, because the data descriptor has a higher priority, Python will call type(obj).__dict__['d'].__get__(obj, type(obj)) instead of calling obj.__dict__[ ' d ']. However, if the descriptor is a non-data Descriptor,python it will be called obj.__dict__['d'] .

Property

A descriptor class is defined every time a descriptor is used, which can seem tedious. Python provides a concise way to add a data descriptor to a property.

Property (Fget=none, Fset=none, Fdel=none, Doc=none) and property attribute

Fget, Fset, and Fdel are class getter, setter, and Deleter methods respectively. We use the following example to illustrate how the property is used:

Class Account (Object):    def __init__ (self):        self._acct_num = None    def get_acct_num (self):        return self . _acct_num    def set_acct_num (self, value):        self._acct_num = value    def del_acct_num (self):        del self._ Acct_num    Acct_num = Property (Get_acct_num, Set_acct_num, Del_acct_num, ' _acct_num property. ')

If Acct is an instance of account, Acct.acct_num will call Getter,acct.acct_num = value will call Setter,del Acct_num.acct_num will call deleter.

>>> acct = account () >>> acct.acct_num = 1000>>> acct.acct_num1000

Python also provides @property adorners, which can be used to create attributes for simple scenarios. A Property object has the Getter,setter and deleter adorner methods that can be used to create a copy of the property through the corresponding accessor function of the decorated function.

Class Account (Object):    def __init__ (self):        self._acct_num = None    @property     # the _acct_num property. The decorator creates a Read-only property    def acct_num (self):        return self._acct_num    @acct_num. Setter    # The _acct_num Property setters makes the property writeable    def set_acct_num (self, value):        self._acct_num = Value    @acct_num. Deleter    def del_acct_num (self):        del self._acct_num

If you want to make the property read-only, simply remove the setter method.

Creating descriptors at run time

We can add property properties at run time:

class Person (object): Def addproperty (self, attribute): # Create local setter and getter with a particular attribute name getter = Lambda Self:self._getproperty (attribut e) setter = Lambda self, value:self._setproperty (attribute, value) # Construct property attribute and add I                                                    T to the class SetAttr (self.__class__, Attribute, property (Fget=getter, \ Fset=setter, \ doc= "Auto-generated method")) def _setprope Rty (self, attribute, value): Print ("Setting: {} = {}". Format (attribute, value)) SetAttr (self, ' _ ' + attribut E, Value.title ()) def _getproperty (self, attribute): Print ("Getting: {}". Format (attribute)) return Getatt R (Self, ' _ ' + attribute) 

>>> user = person () >>> user.addproperty (' name ') >>> user.addproperty (' phone ') >>> User.Name = ' John smith ' Setting:name = John smith>>> user.phone = ' 12345 ' setting:phone = 12345>>> user . Namegetting:name ' John Smith ' >>> user.__dict__{' _phone ': ' 12345 ', ' _name ': ' John Smith '}

Static Methods and Class methods

We can use descriptors to simulate the implementation of @staticmethod and in Python @classmethod . Let's start by looking at the following table:

from an Object

Transformation	called	called from a Class
function	F (obj, *args)	F (*args)
Staticmethod	F (*args)	F (*args)
Classmethod	F (Type (obj), *args)	F (Klass, *args)

Static methods

For static methods f . c.fand C.f is equivalent, are directly query object.__getattribute__(c, ‘f’) or object.__getattribute__(C, ’f‘) . One obvious feature of static methods is that there are no self variables.

What is the use of static methods? Suppose there is a container class that handles specialized data, which provides methods to calculate the average, median, and so on, which are dependent on the corresponding data. However, there may be some methods in the class that do not depend on the data, and we can declare these methods as static methods at this time, which can also improve the readability of the code.

Use non-data descriptors to simulate the implementation of a static method:

Class Staticmethod (object):    def __init__ (self, f):        self.f = f    def __get__ (self, obj, Objtype=none):        Return SELF.F

Let's apply it:

Class MyClass (object):    @StaticMethod    def get_x (x):        return XPrint (myclass.get_x)  # output: 100

Class method

Python's @classmethod @staticmethod usage is somewhat similar, but it's a bit different, and when some methods just need to get the 类的引用 appropriate data in the class, they have to use Classmethod.

Use non-data descriptors to simulate the implementation of a class method:

Class Classmethod (object):    def __init__ (self, f):        self.f = f    def __get__ (self, obj, Klass=none):        if Klass is None:            klass = Type (obj)        def newfunc (*args):            return Self.f (Klass, *args)        return Newfunc

Other methods of magic

The first time I touched the Python magic method, I was also,,, the __get__ __getattribute__ __getattr__ __getitem__ difference between them, and they were all related to property access to the Magic method, which overrides __getattr__ , __getitem__ to construct a collection of their own class is very common, Let's look at some examples of their applications.

__getattr__

Python's default access to a property of a class/instance is invoked by the call, which is called __getattribute__ __getattribute__ unconditionally, and is called if it is not found __getattr__ . If we want to customize a class, usually we should not rewrite __getattribute__ , but should rewrite __getattr__ , rarely see __getattribute__ the situation of rewriting.

As you can see from the output below, a property __getattribute__ is called when it cannot be found __getattr__ .

In [1]: Class Test (object):    ...:     def __getattribute__ (self, item):    ...:         print (' Call __getattribute __ ') ...    :         return Super (Test, self). __GETATTRIBUTE__ (item) ...    :     def __getattr__ (self, item):    ...:         return ' call __getattr__ ' ...    : in [2]: Test (). Acall __getattribute__out[2]: ' Call __getattr__ '

Application

For the default dictionary, Python is only supported in the form of obj['foo'] access, unsupported obj.foo form, and we can rewrite it to __getattr__ let the dictionary also support obj['foo'] the Access form, which is a very classic common usage:

Class Storage (Dict): "" "    a Storage object is like a dictionary except ' Obj.foo ' can being used in    addition to ' OB j[' foo '] '.    ' "" def __getattr__ (self, key):        try:            return Self[key]        except Keyerror as K:            raise Attributeerror (k)    def __setattr__ (self, Key, value):        Self[key] = value    def __delattr__ (self, key):        try:            del Self[key ]        except Keyerror as K:            raise Attributeerror (k)    def __repr__ (self):        return ' <storage ' + dict.__ Repr__ (self) + ' > '

Let's use our custom version of the Enhanced Dictionary:

>>> s = Storage (a=1) >>> s[' a ']1>>> s.a1>>> S.A. = 2>>> s[' a ']2>>> Del S.a>>> S.A. Attributeerror: ' A '

__getitem__

The getitem is used to [] get the elements in the object in the form of subscripts, and we __getitem__ implement a list of our own by rewriting.

Class MyList (object):    def __init__ (self, *args):        self.numbers = args    def __getitem__ (self, item):        return self.numbers[item]my_list = MyList (1, 2, 3, 4, 6, 5, 3) print my_list[2]

This implementation is very humble, does not support slice and step functions, please the reader to improve themselves, here I will not repeat.

Application

Below is __getitem__ a reference to the use of the requests library, we have customized a dictionary class that ignores attribute capitalization.

The program is a little complicated, I'll explain it a little bit: because it's simple here, there's no need to use descriptors, so @property adorner instead, Lower_keys is the instance dictionary The keys in the are all converted to lowercase and stored in the dictionary self._lower_keys . The __getitem__ method is overridden, and later we access a property that first converts the key to lowercase, and then does not directly access the instance dictionary, but instead accesses the dictionary Self._lower_keys to find it. Assignment/delete operation because the instance dictionary changes, in order to keep the Self._lower_keys and the instance dictionary synchronized, first clear the contents of Self._lower_keys . We'll re-create a Self._lower_keys when we re-locate the key and then call __getitem__ again.

Class Caseinsensitivedict (dict):    @property    def lower_keys (self):        if isn't hasattr (self, ' _lower_keys ') or Not self._lower_keys:            Self._lower_keys = Dict ((K.lower (), K) to K in Self.keys ())        return Self._lower_keys    def _clear_lower_keys (self):        if hasattr (self, ' _lower_keys '):            self._lower_keys.clear ()    def __contains_ _ (Self, key):        return Key.lower () in Self.lower_keys    def __getitem__ (self, key):        if key in self:            Return dict.__getitem__ (Self, Self.lower_keys[key.lower ()])    def __setitem__ (self, Key, value):        dict.__ Setitem__ (self, key, value)        Self._clear_lower_keys ()    def __delitem__ (self, key):        dict.__delitem__ ( Self, key)        Self._lower_keys.clear ()    def get (self, Key, Default=none):        if key in self:            return self[ Key]        Else:            return default

Let's call this class:

>>> d = caseinsensitivedict () >>> d[' Ziwenxie '] = ' Ziwenxie ' >>> d[' ziwenxie '] = ' Ziwenxie ' >>> print (d) {' Ziwenxie ': ' Ziwenxie ', ' Ziwenxie ' : ' Ziwenxie '}>>> print (d[' Ziwenxie ') ziwenxie# d[' ziwenxie '] = d[' Ziwenxie ']>>> print (d[' Ziwenxie ']) Ziwenxie