python2.7 Advanced Programming Note II (descriptor in Python)

Python contains a number of built-in language features that make the code simple and easy to understand. These features include List/set/dictionary derivation, property, and Adorner (decorator). For most features, these "intermediate" language features are well documented and easy to Learn.

But there is an exception to this, which is the Descriptor. For me at least, the descriptor is one of the most time-plagued features of the Python language core. Here are a few reasons for this:

1. The official documentation for the descriptor is rather difficult to understand, and does not contain good examples of why you need to write a descriptor (i have to defend Raymond hettinger, the other topics he wrote on the Python articles and videos that helped me a lot)
2. The syntax for writing descriptors seems a little Weird.
3. Custom descriptors may be the least of the features in python, so it's hard to find good examples in open source projects

But once you understand it, the descriptor does have its application Value. This article tells you what descriptors can be used to do, and why it should draw your Attention.

A nutshell: descriptors are reusable properties

Here's What I'm going to tell You: essentially, descriptors are properties that can be Reused. In other words, the descriptor allows you to write code like This:

f = Foo () b = F.barf.bar = CDel F.bar

　　

When the interpreter executes the above code, the custom method is called when you try to access the property (b = f.bar), Assign a value to the property (f.bar = c), or delete the property of an instance variable (del f.bar).

Let's first explain why it's good to pretend that a call to a function is an access to a Property.

property--a function call disguised as access to a property

Imagine that you are writing code to manage your movie Information. The last movie class you wrote might look something like this:

Class Movie (object):    def __init__ (self, title, rating, runtime, budget, gross):        self.title = title        self.rating = rating        self.runtime = runtime        self.budget = Budget        Self.gross = Gross     def profit (self): C7/>return Self.gross-self.budget

　　

You start using this class elsewhere in the project, but then you realize: what if you accidentally hit a negative score on the movie? You think this is the wrong behavior and hope that the movie class can block this Error. The first thing you want to do is change the movie class to This:

Class Movie (object):    def __init__ (self, title, rating, runtime, budget, gross):        self.title = title        self.rating = rating        self.runtime = runtime        Self.gross = Gross        if budget < 0:            raise ValueError (" Negative value not allowed:%s "% budget)        self.budget = Budget     def profit (self):        return SELF.GROSS-SELF.BUDG Et

　　

But it won't Work. Because the other parts of the code are directly assigned by Movie.budget-the newly modified class will only catch the wrong data in the __init__ method, but there is nothing to do with an existing class Instance. If someone tries to run M.budget =-100, then no one can stop it. As a Python programmer is also a movie fan, what should you do?

fortunately, Python's Property solves this Problem. If you have never seen the use of a property, here is an example:

Class Movie (object):    def __init__ (self, title, rating, runtime, budget, gross):        self._budget = None         Self.title = title        self.rating = rating        self.runtime = runtime        Self.gross = Gross        self.budget = budget< c7/> @property    def budget:        return self._budget     @budget. setter    def budget (self, value):        if value < 0:            raise ValueError ("negative value not allowed:%s"% value)        self._budget = value     def  Profit (self):        return self.gross-self.budget m = Movie (' Casablanca ', $, 102, 964000, 1300000) print M.budget       # Calls M.budget (), returns resulttry:    m.budget = -100  # calls Budget.setter ( -100), and raises valueerrorexcept V Alueerror:    Print "woops. Not allowed "964000Woops. Not allowed

　　

we specified a getter method with the @property adorner, and a setter method was specified with the @budget.setter adorner. When we do this, python automatically calls the corresponding Getter/setter method whenever someone tries to access the budget Property. For example, Budget.setter is called automatically when a code such as M.budget = value is Encountered.

Take a moment to admire how graceful python is: if there is no property, we will have to hide all the instance properties and provide a lot of explicit methods like Get_budget and Set_budget. Writing a class like this will continue to call these getter/setter methods, which looks like bloated Java code. What's worse, If we don't use this coding style, we access the instance properties directly. Then there is no way to increase the conditional check for non-negative numbers in a clear manner-we have to recreate the Set_budget method and then search the source code in the entire project and replace the code M.budget = value with M.set_budget (value). The Egg hurts!!

therefore, The property lets us associate the custom code with the access/setting of the variable while maintaining a simple interface for your class to access the Properties. Nice work!

The inadequacy of the property

The biggest drawback to property is that they cannot be reused. For example, Suppose you want to add a non-negative check for rating,runtime and gross fields as Well. The following is a new class that has been modified:

1234567891011121314151617181920212223242526272829303132333435363738394041424344454647484950515253545556575859class Movie (object): def __init__ (self, title, rating, runtime, budget, gross): self._rating = None Self._runt        IME = None Self._budget = None Self._gross = None Self.title = Title self.rating = Rating Self.runtime = Runtime Self.gross = Gross Self.budget = Budget #nice @property def budget (self ): return self._budget @budget. setter def budget (self, value): if value < 0:raise Valu         Eerror ("negative value not allowed:%s"% value) self._budget = value #ok @property def-rating (self): Return self._rating @rating. setter def rating (self, value): if value < 0:raise Valuee        Rror ("negative value not allowed:%s"% value) self._rating = value #uhh ... @property def runtime (self):     Return Self._runtime@runtime. setter def runtime (self, value): if value < 0:raise valueerror (' negative value not ' allow    ed:%s "% value" self._runtime = value #is this forever? @property def Gross (self): return Self._gross @gross. setter def gross (self, value): if value < 0:raise valueerror ("negative value not allowed:%s"% value) Self._gross = value def profit (self): return Self.gross-self.budget

　　

You can see that the code has increased a lot, but the logic of repetition also appears quite a bit. Although the property can make the class from the outside look like the interface neat and beautiful, but do not make the interior equally neat and Beautiful.

Descriptor (the final Big kill Device)

This is the problem addressed by the Descriptor. The descriptor is an upgraded version of the property that allows you to write a separate class for the duplicated property logic to Handle. The following example shows how a descriptor works (you don't have to worry about the implementation of the Nonnegative class now):

From Weakref import weakkeydictionary class nonnegative (object): "", "A descriptor that forbids negative values" "def __init__ (self, default): self.default = Default Self.data = Weakkeydictionary () def __get__ (self, Insta nce, owner): # We get here if someone calls x.d, and D is a nonnegative instance # instance = x # o  Wner = Type (x) return self.data.get (instance, self.default) def __set__ (self, instance, value): # we get Here's when someone calls X.D = val, and D is a nonnegative instance # instance = x # value = val If v Alue < 0:raise valueerror ("negative value not allowed:%s"% value) self.data[instance] = value Clas    s Movie (object): #always put descriptors at the class-level rating = nonnegative (0) runtime = nonnegative (0) Budget = nonnegative (0) gross = nonnegative (0) def __init__ (self, title, rating, runtime, budget, gross): SE        Lf.title = TitleSelf.rating = Rating Self.runtime = Runtime Self.budget = Budget Self.gross = Gross def profit (se lf): return self.gross-self.budget m = movie (' Casablanca ', $, 102, 964000, 1300000) print M.budget # calls movie . budget.__get__ (m, Movie) m.rating = # calls movie.budget.__set__ (m, +) try:m.rating =-1 # calls Movie.budget. __set__ (m, -100) except valueerror:print "woops, negative value" 964000Woops, negative value

　　

Here we introduce some new syntax, we look at the Following:

Nonnegative is a descriptor object because it defines the __get__,__set__ or __delete__ method.

The movie class now looks very clear. We created 4 descriptors at the class level as normal instance Properties. obviously, The descriptor is doing a non-negative check for us here.

Access Descriptor

When the interpreter encounters print m.buget, It takes budget as a descriptor with a __get__ method, invokes the Movie.budget.__get__ method and prints the return value of the method instead of passing M.budget directly. This is similar to the way you access a property, python automatically calls a method and returns the Result.

__GET__ receives 2 Parameters: One is the instance object to the left of the dot number (in this case, m in M.budget) and the other is the type of the instance (Movie). In some Python documents, movie is referred to as the owner of the descriptor (owner). If we need access to Movie.budget,python will call movie.budget.__get__ (None, Movie). As you can see, the first parameter is either an instance of the owner or NONE. These input parameters may seem strange, but here they tell you what part of the object the descriptor belongs to. It all makes sense when we see the implementation of the nonnegative class.

to describe the assigned value

When the interpreter sees m.rating = 100, Python recognizes that rating is a descriptor with a __set__ method and calls movie.rating.__set__ (m, 100). As with __get__, the first argument to __set__ is the class instance to the left of the dot number (m.rating = m in 100). The second parameter is the assigned value (100).

Delete Descriptor

To illustrate the completeness, mention here the Deletion. If you call del M.budget,python it will call movie.budget.__delete__ (m).

How does the nonnegative class work?

With the confusion ahead, we are finally going to reveal how the Nonnegative class Works. Each instance of nonnegative maintains a dictionary that holds the mappings between the owner instance and the corresponding Data. When we call m.budget, the __get__ method looks for the data associated with m and returns the result (if the value does not exist, it returns a default value). The __set__ takes the same approach, but this includes additional non-negative checks. We use weakkeydictionary instead of a common dictionary to prevent memory leaks--we don't want to just have a useless instance of it in the Descriptor's dictionary.

Using descriptors can be a bit awkward. Because they work at the class level, each class instance shares the same descriptor. This means that the descriptors have to be managed manually for different instance objects, and that you need to explicitly pass the class instance as the first parameter exactly to the __get__, __set__, and __delete__ methods.

I hope this example explains what descriptors can be used to do-they provide a way to isolate the logic of the property into a separate class to Handle. If you find yourself repeating the same logic between different property, then this article might be a clue for you to think about why refactoring code with descriptors is worth a try.

Tips and Pitfalls Place the descriptor at the class level

In order for descriptors to work correctly, they must be defined at the class Level. If you do not, Python cannot automatically call the __get__ and __set__ methods for YOU.

Class Broken (object):    y = nonnegative (5)    def __init__ (self):        self.x = nonnegative (0)  # a good Descriptor B = Broken () print "x is%s, Y was%s"% (b.x, b.y) X is <__main__. Nonnegative object at 0x10432c250>, Y is 5

　　

As you can see, accessing the descriptor Y on the class hierarchy automatically calls __get__. But the descriptor x at the access instance level only returns the descriptor itself, which is a magical existence.

ensure that the Instance's data belongs only to the instance itself

You might write nonnegative descriptors like This:

Class Brokennonnegative (object):    def __init__ (self, default):        self.value = default     def __get__ (self, instance, owner):        return self.value     def __set__ (self, instance, value):        if value < 0:            Raise ValueError ("negative value not allowed:%s"% value)        self.value = value class Foo (object):    bar = brokennonnegative (5) f = Foo () try:    f.bar = -1except valueerror:    print "caught The invalid assignment" caught the invalid Assignmen T

　　

It seems to work as Expected. But the problem here is that all instances of Foo share the same bar, which produces some painful results:

Class Foo (object):    bar = brokennonnegative (5) f = foo () g = foo () print "f.bar is%s\ng.bar is%s"% (f.bar, g.bar) pri NT "Setting f.bar to" f.bar = 10print "f.bar are%s\ng.bar is%s"% (f.bar, g.bar)  #ouchf. Bar was 5g.bar is 5Setting F.bar to 10f.bar are 10g.bar is 10

　　

That's Why we want to use a data dictionary in Nonnegative. The first parameter of __get__ and __set__ tells us which instance to care About. Nonnegative uses this parameter as the dictionary key to save a single copy of the data for each Foo INSTANCE.

Class Foo (object):    bar = nonnegative (5) f = foo () g = foo () print "f.bar is%s\ng.bar is%s"% (f.bar, g.bar) Print "sett ing F.bar to ten "f.bar = 10print" f.bar is%s\ng.bar was%s "% (f.bar, g.bar)  #betterf. Bar is 5g.bar is 5Setting F.bar t O 10f.bar is 10g.bar is 5

　　

This is where the descriptor is most uncomfortable (frankly, I don't understand why Python doesn't allow you to define descriptors at the instance level, and you always need to distribute the actual processing to __get__ and __set__.) There must be a reason for this to Work.

Note the non-hashed descriptor owner

The nonnegative class uses a dictionary to save data that is unique to the Instance. This is generally not a problem, unless you use a non-hash (unhashable) object:

Class Moproblems (list):  #you can ' t use lists as dictionary keys    x = nonnegative (5) m = moproblems () print m.x  # Womp womp---------------------------------------------------------------------------TypeError                                 Traceback ( Most recent call Last) <ipython-input-8-dd73b177bd8d> in <module> ()      3      4 m = moproblems ()----> 5 Print m.x  # womp Womp <ipython-input-3-6671804ce5d5> in __get__ (self, instance, owner)      9         # instance = X     Ten         # owner = Type (x)--->         return self.data.get (instance, self.default)     def _ _set__ (self, instance, value): typeerror:unhashable type: ' Moproblems '

　　

Because instances of Moproblems (the subclasses of the List) are not hashed, they cannot be used as key to the data dictionary for MOPROBLEMS.X. There are ways to circumvent this problem, but none of them are perfect. The best way to do this might be to give your description a multibyte tag.

Class Descriptor (object):     def __init__ (self, label):        self.label = label     def __get__ (self, instance, owner) :        print ' __get__ ', instance, owner        return instance.__dict__.get (self.label)     def __set__ (self, instance, value):        print ' __set__ '        instance.__dict__[self.label] = value class Foo (list):    x = descriptor (' x ')    y = Descriptor (' y ') f = Foo () f.x = 5print f.x __set____get__ [] <class ' __main__. Foo ' >5

　　

This approach relies on Python's method parsing order (that is, MRO). We add a tag name to each descriptor in foo, with the same name as the variable name we assign to the descriptor, such as x = Descriptor (' x '). After that, the descriptor saves the instance-specific data in f.__dict__[' x ']. This dictionary entry is usually the return value that Python gives when we request F.X. however, because foo.x is a descriptor, python does not normally use f.__dict__[' x ', but the descriptor can safely store the data here. Just remember, don't tag this descriptor anywhere Else.

Class Foo (object):    x = descriptor (' y ') f = Foo () f.x = 5print f.x f.y = 4    #oh no!print f.x__set____get__ <__main __. Foo object at 0x10432c810> <class ' __main__. Foo ' >5__get__ <__main__. Foo object at 0x10432c810> <class ' __main__. Foo ' >4

　　

I don't like this way, because the code is fragile and there are a lot of subtleties. But this method is quite common and can be used on non-hashed owner Classes. David Beazley used this method in his Book.

using tagged descriptors in a meta-class

Because the label name of the descriptor is the same as the variable name assigned to it, someone uses the Meta-class to automatically handle this bookkeeping (bookkeeping) Task.

Class Descriptor (object):    def __init__ (self):        #notice we aren ' t setting the label here        Self.label = None     def __get__ (self, instance, owner):        print ' __get__. Label =%s '% Self.label        return instance.__dict__.get (self.label, None)     def __set__ (self, instance, value):        print ' __set__ '        instance.__dict__[self.label] = value class Descriptorowner (type):    def __new__ (cls, name , bases, attrs):        # Find all descriptors, auto-set their labels for        n, v in attrs.items ():            if Isinstance (v, descriptor):                V.label = n        return super (descriptorowner, cls). __new__ (cls, name, bases, attrs) class Foo (object ):    __metaclass__ = descriptorowner    x = descriptor () f = Foo () f.x = 10print f.x __set____get__. Label = x10

　　

I'm not going to explain the details of the Meta-class--david Beazley has already explained it clearly in his article in the reference Literature. It should be noted that the meta-class automatically adds tags to the descriptor and matches the name of the variable assigned to the Descriptor.

Although this solves the problem of inconsistent label and variable name of the descriptor, it introduces a complex meta-class. Although I doubt it, you can judge for yourself whether it is worth it.

ways to access descriptors

Descriptors are just classes, and maybe you want to add some methods to Them. For example, A descriptor is a good way to callback a Property. Let us know as soon as we want the state of a part of a class to Change. Most of the code below is used to do this:

Class Callbackproperty (object): "" "A property that would alert observers when upon updates" "def __init__ (self, DEFA ult=none): self.data = weakkeydictionary () Self.default = Default Self.callbacks = Weakkeydictionary ( ) def __get__ (self, instance, owner): return self.data.get (instance, self.default) def __set__ (self, Instan ce, value): for callback in Self.callbacks.get (instance, []): # Alert callback function of new V        Alue callback (value) self.data[instance] = value def add_callback (self, instance, callback):        "" "Add A new function to call everytime the descriptor updates '" "#but How do we get here?!?! If instance not in self.callbacks:self.callbacks[instance] = [] self.callbacks[instance].append (callbac        K) class BankAccount (object): balance = Callbackproperty (0) def low_balance_warning (value): if value < 100: Print "you are poor" ba = BankaccounT () # won't work--try it#ba.balance.add_callback (ba, Low_balance_warning) 

　　

This is an attractive pattern-we can customize the callback function to respond to state changes in a class, without having to modify the code for that class at All. This is really a burden on people. Now all we have to do is call Ba.balance.add_callback (ba, Low_balance_warning) so that low_balance_warning will be called every time the balance Changes.

But how do we do that? Descriptors always call __get__ when we try to access Them. It's like the Add_callback method is untouchable! The key is to take advantage of a special case where the first parameter of the __get__ method is none when accessed from the class hierarchy

Class Callbackproperty (object): "" "A property that would alert observers when upon updates" "def __init__ (self, DEFA ult=none): self.data = weakkeydictionary () Self.default = Default Self.callbacks = Weakkeydictionary ( ) def __get__ (self, instance, owner): If instance was None:return self return self.da  Ta.get (instance, Self.default) def __set__ (self, instance, value): for callback in Self.callbacks.get (instance,     []): # Alert callback function of new value callback (value) self.data[instance] = value def add_callback (self, instance, callback): "" "add A new function to call everytime the descriptor within instance Updates "" "if instance not in self.callbacks:self.callbacks[instance] = [] Self.callbacks[instan Ce].append (callback) class BankAccount (object): balance = Callbackproperty (0) def low_balance_warning (value): if Val UE < 100:prinT "you is now poor" ba = BankAccount () BankAccount.balance.add_callback (ba, Low_balance_warning) ba.balance = 5000print "B Alance is%s "% ba.balanceba.balance = 99print" balance was%s "% ba.balancebalance is 5000You was now poorbalance is 99

Original Link: http://www.geekfan.net/7862/

python2.7 Advanced Programming Note II (descriptor in Python)