Magic descriptor in Python

Introduction Descriptors (descriptor) is an esoteric but important dark magic in Python. it is widely used in the Python kernel, mastering descriptors will add an additional skill to the toolbox of Python programmers. This article describes the descriptor definition and some common scenarios. at the end of the article, we will add _ getattr ,__ getattribute __, _ getitem _ these three magic methods also involve attribute access. Descriptor definition descr _ get _ (self, obj, obj... introduction

Descriptors (descriptor) is an esoteric but important dark magic in the Python language. it is widely used in the Python kernel, mastering descriptors will add an additional skill to the toolbox of Python programmers. This article describes the definition of descriptors and some common scenarios.__getattr,__getattribute__,__getitem__These three magic methods also involve attribute access.

Descriptor definition

descr__get__(self, obj, objtype=None) --> valuedescr.__set__(self, obj, value) --> Nonedescr.__delete__(self, obj) --> None

Only oneobject attribute(Object attribute) defines any of the above three methods, so this class can be called a descriptor class.

Descriptor basics

In the following example, we createRevealAcessClass, and implements__get__METHOD. now this class can be called a descriptor class.

class RevealAccess(object):    def __get__(self, obj, objtype):        print('self in RevealAccess: {}'.format(self))        print('self: {}\nobj: {}\nobjtype: {}'.format(self, obj, objtype))class MyClass(object):    x = RevealAccess()    def test(self):        print('self in MyClass: {}'.format(self))

EX1 instance attributes

Let's take a look.__get__The meaning of each parameter of the method. in the following example,selfThat is, instance x of the RevealAccess class,objThat is, the instance m of the MyClass class,objtypeAs the name implies, it is the MyClass class itself. From the output statement, we can see that,m.xAccess descriptorxYes__get__Method.

>>> m = MyClass()>>> m.test()self in MyClass: <__main__.MyClass object at 0x7f19d4e42160>>>> m.xself in RevealAccess: <__main__.RevealAccess object at 0x7f19d4e420f0>self: <__main__.RevealAccess object at 0x7f19d4e420f0>obj: <__main__.MyClass object at 0x7f19d4e42160>objtype: 
 

EX2 attributes

If you directly access properties through a classx, ThenobjThe connection is directly set to None, which is better understood because there is no MyClass instance.

>>> MyClass.xself in RevealAccess: <__main__.RevealAccess object at 0x7f53651070f0>self: <__main__.RevealAccess object at 0x7f53651070f0>obj: Noneobjtype: 
 

Descriptor trigger

In the above example, we list the descriptor usage from the perspective of instance attributes and class attributes. next we will analyze the internal principles carefully:

• ForInstance attributesActually, the _ getattribute _ method of the base class object is called. In this method, obj. d is translatedtype(obj).__dict__['d'].__get__(obj, type(obj)).

• ForClass attributesIs equivalent to calling the _ getattribute _ method of the metadata type. it translates cls. dcls.__dict__['d'].__get__(None, cls)Here, the _ get _ () obj is None because the instance does not exist.

Briefly__getattribute__Magic method. this method is called unconditionally when we access the attributes of an object. details such__getattr,__getitem__I will make an additional supplement at the end of the article. we will not go into it for the moment.

Descriptor priority

First, there are two types of descriptors:

• If an object defines both the _ get _ () and _ set _ () methods, this descriptor is calleddata descriptor.

• If an object only defines the _ get _ () method, this descriptor is callednon-data descriptor.

When we access attributes, there are four situations:

• Data descriptor

• Instance dict

• Non-data descriptor

• _ Getattr __()

Their priority values are:

data descriptor > instance dict > non-data descriptor > __getattr__()

What does this mean? That is to say, if the Object obj contains an object with the same namedata descriptor->dAndinstance attribute->d,obj.dAttributedPython callstype(obj).__dict__['d'].__get__(obj, type(obj))Instead of calling obj. _ dict _ ['D']. However, if the descriptor is a non-data descriptor, Python will callobj.__dict__['d'].

Property

Every time a descriptor is used, a descriptor class is defined, which looks very complicated. Python provides a simple way to add data descriptors to attributes.

property(fget=None, fset=None, fdel=None, doc=None) -> property attribute

Fget, fset, and fdel are respectively the getter, setter, and deleter methods of the class. The following example shows how to use Property:

class Account(object):    def __init__(self):        self._acct_num = None    def get_acct_num(self):        return self._acct_num    def set_acct_num(self, value):        self._acct_num = value    def del_acct_num(self):        del self._acct_num    acct_num = property(get_acct_num, set_acct_num, del_acct_num, '_acct_num property.')

If acct is an instance of an Account, acct. acct_num will call getter. acct. acct_num = value will call setter, del acct_num.acct_num will call deleter.

>>> acct = Account()>>> acct.acct_num = 1000>>> acct.acct_num1000

Python also provides@propertyDecorator, which can be used to create attributes for simple application scenarios. An attribute object has the getter, setter, and deleter decorator methods. you can use them to create copies of attributes through the corresponding accessor function of the decoration function.

class Account(object):    def __init__(self):        self._acct_num = None    @property     # the _acct_num property. the decorator creates a read-only property    def acct_num(self):        return self._acct_num    @acct_num.setter    # the _acct_num property setter makes the property writeable    def set_acct_num(self, value):        self._acct_num = value    @acct_num.deleter    def del_acct_num(self):        del self._acct_num

To make the attribute read-only, you only need to remove the setter method.

Create a descriptor at runtime

We can add the property at runtime:

class Person(object):    def addProperty(self, attribute):        # create local setter and getter with a particular attribute name        getter = lambda self: self._getProperty(attribute)        setter = lambda self, value: self._setProperty(attribute, value)        # construct property attribute and add it to the class        setattr(self.__class__, attribute, property(fget=getter, \                                                    fset=setter, \                                                    doc="Auto-generated method"))    def _setProperty(self, attribute, value):        print("Setting: {} = {}".format(attribute, value))        setattr(self, '_' + attribute, value.title())    def _getProperty(self, attribute):        print("Getting: {}".format(attribute))        return getattr(self, '_' + attribute)

>>> user = Person()>>> user.addProperty('name')>>> user.addProperty('phone')>>> user.name = 'john smith'Setting: name = john smith>>> user.phone = '12345'Setting: phone = 12345>>> user.nameGetting: name'John Smith'>>> user.__dict__{'_phone': '12345', '_name': 'John Smith'}

Static methods and class methods

We can use descriptors to simulate@staticmethodAnd@classmethod. First, let's look at the table below the category:

Transformation	Called from an Object	Called from a Class
Function	F (obj, * args)	F (* args)
Staticmethod	F (* args)	F (* args)
Classmethod	F (type (obj), * args)	F (klass, * args)

Static method

For static methodsf.c.fAndC.fIt is equivalent to direct query.object.__getattribute__(c, ‘f’)Orobject.__getattribute__(C, ’f‘). One obvious feature of static methods is that they do notselfVariable.

What is the use of static methods? Assume that there is a container class for processing specialized data, which provides methods to calculate the mean, median, and other statistical data methods. these methods depend on the corresponding data. However, some methods in the class may not depend on the data. in this case, we can declare these methods as static methods, which can also improve the readability of the code.

Use non-data descriptors to simulate the implementation of static methods:

class StaticMethod(object):    def __init__(self, f):        self.f = f    def __get__(self, obj, objtype=None):        return self.f

Let's apply the following:

class MyClass(object):    @StaticMethod    def get_x(x):        return xprint(MyClass.get_x(100))  # output: 100

Class method

Python@classmethodAnd@staticmethodThe usage is somewhat similar, but there are still some differences, when some methods only need to getClass referenceClassmethod is required when you do not care about the corresponding data in the class.

Use non-data descriptors to simulate the implementation of class methods:

class ClassMethod(object):    def __init__(self, f):        self.f = f    def __get__(self, obj, klass=None):        if klass is None:            klass = type(obj)        def newfunc(*args):            return self.f(klass, *args)        return newfunc

Other magic methods

When I first came into contact with the Python magic method__get__,__getattribute__,__getattr__,__getitem__The difference between them is Plagued. they are all magic methods related to attribute access, where rewriting__getattr__,__getitem__It is very common to construct a collection class of your own. here we will look at their applications through some examples.

_ Getattr __

By default, a property of the runtime class/instance in Python is__getattribute__To call,__getattribute__It will be called unconditionally. if it is not found, it will be called.__getattr__. If we want to customize a class, we should not overwrite it normally.__getattribute__But should be rewritten.__getattr__, Rarely seen rewriting__getattribute__.

The output below shows that when an attribute passes through__getattribute__It is called when it cannot be found.__getattr__.

In [1]: class Test(object):    ...:     def __getattribute__(self, item):    ...:         print('call __getattribute__')    ...:         return super(Test, self).__getattribute__(item)    ...:     def __getattr__(self, item):    ...:         return 'call __getattr__'    ...:In [2]: Test().acall __getattribute__Out[2]: 'call __getattr__'

Application

For the default dictionary, Python only supportsobj['foo']Format to access, not supportedobj.fooIn the format__getattr__Enable dictionary supportobj['foo']Is a classic and common usage:

class Storage(dict):    """    A Storage object is like a dictionary except `obj.foo` can be used    in addition to `obj['foo']`.    """    def __getattr__(self, key):        try:            return self[key]        except KeyError as k:            raise AttributeError(k)    def __setattr__(self, key, value):        self[key] = value    def __delattr__(self, key):        try:            del self[key]        except KeyError as k:            raise AttributeError(k)    def __repr__(self):        return '
 
  '
 

Let's use our custom enhanced dictionary:

>>> s = Storage(a=1)>>> s['a']1>>> s.a1>>> s.a = 2>>> s['a']2>>> del s.a>>> s.a...AttributeError: 'a'

_ Getitem __

Getitem is used by subscript[]To obtain the elements in the object.__getitem__To implement a list.

class MyList(object):    def __init__(self, *args):        self.numbers = args    def __getitem__(self, item):        return self.numbers[item]my_list = MyList(1, 2, 3, 4, 6, 5, 3)print my_list[2]

This implementation is very simple and does not support slice, step, and other functions. please improve it on your own. I will not repeat it here.

Application

The following is a reference__getitem__We have customized a dictionary class that ignores the case sensitivity of attributes.

The program is a bit complicated. I will explain it a little: because it is relatively simple and there is no need to use descriptors, I used@propertyDecorator instead,lower_keysThe function isInstance DictionaryAll the keys in are converted to lowercase and stored in the dictionary.self._lower_keys. Rewritten__getitem__In the future, when we access a property, the key is first converted to lowercase, and then the instance dictionary is not directly accessed, but the dictionary is accessed.self._lower_keysTo search. When assigning values or deleting values, the instance dictionary is changed.self._lower_keysSynchronize with the instance Dictionary, first clearself._lower_keysAnd then call__getitem__Will create a newself._lower_keys.

class CaseInsensitiveDict(dict):    @property    def lower_keys(self):        if not hasattr(self, '_lower_keys') or not self._lower_keys:            self._lower_keys = dict((k.lower(), k) for k in self.keys())        return self._lower_keys    def _clear_lower_keys(self):        if hasattr(self, '_lower_keys'):            self._lower_keys.clear()    def __contains__(self, key):        return key.lower() in self.lower_keys    def __getitem__(self, key):        if key in self:            return dict.__getitem__(self, self.lower_keys[key.lower()])    def __setitem__(self, key, value):        dict.__setitem__(self, key, value)        self._clear_lower_keys()    def __delitem__(self, key):        dict.__delitem__(self, key)        self._lower_keys.clear()    def get(self, key, default=None):        if key in self:            return self[key]        else:            return default

Let's call this class:

>>> d = CaseInsensitiveDict()>>> d['ziwenxie'] = 'ziwenxie'>>> d['ZiWenXie'] = 'ZiWenXie'>>> print(d){'ZiWenXie': 'ziwenxie', 'ziwenxie': 'ziwenxie'}>>> print(d['ziwenxie'])ziwenxie# d['ZiWenXie'] => d['ziwenxie']>>> print(d['ZiWenXie'])ziwenxie

The above is the details of the magic descriptor in Python. For more information, see other related articles in the first PHP community!