Classes and metaclasses in Python

Classes and Objects

Classes and functions are all objects in Python. When a class definition is complete, Python creates a "class object" and assigns it to a variable of the same name. Class is a type of object (is it a bit of a mouthful?) )。

The class object is callable (callable, implements the __call__ method), and invokes it to create an object of the class. You can treat a class as if it were another object. For example, you can assign values to their properties, and you will be able to assign them to a variable, and you could use them where any callable object can be used, such as in a map. In fact, when you are using map (str, [+]), you are converting an integer type list to a list of string types, because STR is a class. You can look at the following code:

>>> class C (object): ...   def __init__ (self, s): ...       Print s...>>> MyClass = c>>> type (C)
 
  
   
  >>> type (MyClass)
  
   
    
   >> > MyClass (2) 2<__main__. C object at 0x10e2bea50>>>> map (MyClass, []) 123[<__main__. C object at 0x10e2be9d0> <__main__. C object at 0x10e2bead0> <__main__. C Object at 0x10e2beb10>]>>> Map (c, [123[<__main__]). C object at 0x10e2be950> <__main__. C object at 0x10e2beb50> <__main__. C object at 0x10e2beb90>]>>> c.test_attribute = true>>> myclass.test_attributetrue
  
   
 
  

Because of this, the "class" keyword in Python does not have to appear in the code main scope, as in other languages, such as C + +. In Python, it can be nested in a function, for example, we can create the class dynamically in the process of running the function. Look at the code:

>>> def make_class (class_name): ...   Class C (object): ...       def print_class_name (self): ...           Print Class_name   ... c.__name__ = Class_name   ... Return c...>>> C1, C2 = Map (Make_class, ["C1", "C2"]) >>> c1, c2 = C1 (), C2 () >>> C1.print_class_ Name () c1>>> c2.print_class_name () c2>>> type (c1)
 
  
   
  >>> type (C2)
  
   
    
   >>> c1.print_class_name.__closure__ (
   
    
     
    ,)
   
    
  
   
 
  

Note that the two classes created here through Make_class are different objects, so objects created by them are not of the same type. As we did in the adorner, we set the class name manually after the class was created. Also note that the Print_class_name method of the created class captures the closure and class_name of the class in a closure cell. If you are not very clear about the concept of closure, then it is better to go to the previous article, review the closures and decorators related content.
Metaclasses

If a class is an object that can make objects, what is the object of the manufacturing class called (believe me, this is not a question of chicken or egg first)? The answer is a meta-class (metaclasses). Most common base meta-classes are type. When you enter a parameter, the type will simply return the input object types, which does not involve a meta-class. However, when entering three parameters, type will play the role of a meta-class, creating a class based on the input parameters and returning. The input parameters are fairly simple: a Dictionary of class names, parent classes, and their arguments. The latter two can be empty, to see an example:

>>> MyClass = Type ("MyClass", (object,), {"My_attribute": 0}) >>> type (MyClass)
 
  
   
  > >> o = MyClass () >>> o.my_attribute0
 
  

Pay special attention to the second parameter is a tuple (syntax looks strange, ending with a comma). If you need to schedule a method in a class, create a function and pass it as a property as a third argument, like this:

>>> def myclass_init (self, my_attr): ...   Self.my_attribute = my_attr...>>> MyClass = Type ("MyClass", (object,), {"My_attribute": 0, "__init__": Myclass_ init}) >>> o = MyClass ("test") >>> o.my_attribute ' test ' >>> o.__init__
 
   <__main__.myclass object="" at="" 0x10ab72150="">
  >
 
  

We can customize the meta-class through a callable object (function or class) that requires three input parameters and returns an object. Such a meta-class is implemented on a class as long as its __metaclass__ property is defined. For the first example, let's do something interesting to see what we can do with the meta-class:

>>> def mymetaclass (name, parents, attributes): ...   Return "Hello" ...>>> class C (object): ...   __metaclass__ = mymetaclass...>>> print chello>>> type (C)
 
  

Note that the code above, c simply points a variable reference to the string "Hello". Of course, no one will write this code in practice, just a simple example to illustrate the use of the meta-class. Next, let's do some more useful things. In the second part of this series we saw how to use the adorner class to record the output of each method of the target class, and now we do the same thing, but this time we use the Meta class. We borrowed the previous adorner definition:

def log_everything_metaclass (class_name, parents, attributes):  print "Creating class", Class_name  Myattributes = {}  for name, attr in Attributes.items ():    myattributes[name] = attr    if hasattr (attr, ' __call__ '):      myattributes[name] = logged ("%b%d%Y-%h:%m:%s",                    class_name + ".") (attr)  return type (class_name, parents, Myattributes) class C (object):  __metaclass__ = Log_everything_metaclass   def _ _init__ (self, x):    self.x = x   def print_x (self):    print self.x # usage:print "Starting object creation" C = C ( "Test") c.print_x () # output:creating Class Cstarting object creation-running ' c.__init__ ' on 2013-13:50:58-fini Shed ' c.__init__ ', Execution time = 0.000s-running ' c.print_x ' on 2013-13:50:58test-finished ' c.print_x ', Execu tion time = 0.000s

As you can see, class decorators have a lot in common with meta classes. In fact, any functionality that can be done with a class decorator can be implemented using a meta-class. The class decorator has a very simple grammatical structure which is easy to read, so it is advocated. But in the case of a meta-class, it can do more because it runs before the class is created, and the class decorator runs after the class is created. Remember this, let's run both at the same time, notice the sequencing of the run:

def my_metaclass (class_name, parents, attributes):  print "In Metaclass, creating the class."  return type (class_name, parents, attributes) def my_class_decorator (class_):  print "in decorator, chance to modify th E class. "  return Class_ @my_class_decoratorclass C (object):  __metaclass__ = My_metaclass   def __init__ (self):    print "Creating object." C = C () # Output:in Metaclass, Creating the class. In decorator, chance to modify the class. Creating object.

An actual use case of a meta-class

Let's consider a more useful example. Suppose we are conceiving a collection of classes to handle the ID3v2 tags used in MP3 music files Wikipedia. In short, labels are made up of frames (frames), and each frame is tagged with a four-character identification number (identifier). For example, Tope identifies the original author frame, Toal identifies the original album name, and so on. If we want to write a separate class for each frame type, and allow ID3v2 tag library users to customize their own frame classes. Then we can use the Meta class to implement a class factory pattern, which can be implemented in such a way:

Frametype_class_dict = {} class Id3v2frameclassfactory (object): Def __new__ (CLS, class_name, parents, attributes): PRI NT "Creating class", Class_name # Here we could add some helper methods or attributes to c c = Type (class_name, pare NTS, attributes) if attributes[' frame_identifier ': frametype_class_dict[attributes[' frame_identifier ']] = c RE Turn C @staticmethod def get_class_from_frame_identifier (frame_identifier): Return Frametype_class_dict.get (frame_id Entifier) class Id3v2frame (object): Frame_identifier = None __metaclass__ = id3v2frameclassfactory Pass Class Id3v2titl Eframe (id3v2frame): __metaclass__ = id3v2frameclassfactory frame_identifier = "TIT2" Class Id3v2commentframe ( Id3v2frame): __metaclass__ = id3v2frameclassfactory frame_identifier = "COMM" Title_class = id3v2frameclassfactory.get_ Class_from_frame_identifier (' TIT2 ') Comment_class = Id3v2frameclassfactory.get_class_from_frame_identifier (' COMM ') ) Print Title_classprint Comment_class # Output: Creating Class Id3v2framecreating class Id3v2titleframecreating class Id3v2commentframe 
 
  
 
   

  
 

Of course, the above code can also be used to complete the class decorator, the following is the corresponding code:

Frametype_class_dict = {} class Id3v2frameclass (object): Def __init__ (self, frame_id): self.frame_id = frame_id def  __call__ (self, CLS): print ' Decorating class ', cls.__name__ # Here we could add some helper methods or attributes to c if self.frame_id:frametype_class_dict[self.frame_id] = CLS return CLS @staticmethod def Get_class_from_f Rame_identifier (frame_identifier): Return Frametype_class_dict.get (Frame_identifier) @ID3v2FrameClass (None) class Id3v2frame (object): Pass @ID3v2FrameClass ("TIT2") class Id3v2titleframe (id3v2frame): Pass @ID3v2FrameClass ("COMM") Class Id3v2commentframe (id3v2frame): Pass Title_class = Id3v2frameclass.get_class_from_frame_identifier (' TIT2 ') Comment_class = Id3v2frameclass.get_class_from_frame_identifier (' COMM ') print Title_classprint Comment_class Decorating class Id3v2framedecorating class id3v2titleframedecorating class Id3v2commentframe
 
  
 
   

  
 

As you can see, we are able to pass parameters directly to the adorner, but the meta-class is not. Passing a parameter to a meta class must pass a property. Because of this, the decorator solution here is clearer and easier to maintain. However, it is also important to note that when the adorner is called, the class has been created, which means that it is not possible to modify its properties at this time. For example, once the class is established, you cannot modify the __doc__. Take a look at practical examples:

>>> def mydecorator (CLS): ...   cls.__doc__ = "test!" ...   Return cls...>>> @mydecorator ... class C (object): ... "" "   DocString to being replaced with test! "" " ...   Pass ... Traceback (most recent): File "
 
  
   
  ", line 2, in file " 
  
   
   
    
     
    ", Line 2, in MyD Ecoratorattributeerror:attribute ' __doc__ ' of ' type ' objects is not writable>>> def mymetaclass (CLS, parents, at TRS): ...   attrs[' __doc__ '] = ' test! ' ...   return type (CLS, parents, Attrs) ...>>> class D (object): ... "" "   DocString to being replaced with test! "" " ...   __metaclass__ = mymetaclass...>>> d.__doc__ ' test! '
   
    
  
   
 
  

Generating a meta class from type

As we said, the most basic meta-class is type and the class is usually type. So the question comes naturally, what type is the type itself? The answer is also type. This means that the type is its own meta-class. Although it may sound strange, it is feasible at the level of the Python interpreter.

The type itself is a class, and we can inherit the new class from it. These generated classes can also be used as meta-classes, and the classes that use them can have types that are the same as using type. Take a look at the following example:

>>> class meta (type): ...   Def __new__ (CLS, class_name, parents, attributes): ...       Print "meta.__new__"       ... Return super (Meta, CLS). __new__ (CLS, class_name, parents, attributes)   ... def __call__ (self, *args, **kwargs): ...       Print "meta.__call__"       ... Return super (meta, self). __call__ (*args, **kwargs) ...>>> class C (object): ...   __metaclass__ = meta...meta.__new__>>> C = C () meta.__call__>>> type (c)
 
  

Note that when a class creates an object, the __call__ function of the meta-class is called, and the type.__call__ is called to create the object. In the next section, we'll merge the above content together.
Essentials Collection

Suppose a Class C's own meta class is my_metaclass and adorned by the adorner my_class_decorator. Also, assume that My_metaclass itself is a class, generated from type. Let's combine the above mentioned to make a summary of how the Class C and its objects are created. First, let's take a look at the code:

Class My_metaclass (type): Def __new__ (CLS, class_name, parents, attributes): print "-my_metaclass.__new__-Creating                        class instance of type ", CLS return Super (My_metaclass, CLS). __new__ (CLS, Class_name, Parents, attributes) def __init__ (self, class_name, parents, attributes): print " -My_metaclass.__init__-Initializing The class instance ", Self super (My_metaclass, self). __init__ (self) def __call_ _ (Self, *args, **kwargs): print "-my_metaclass.__call__-Creating object of type", Self return Super (My_metaclass  , self). __call__ (*args, **kwargs) def my_class_decorator (CLS): print "-My_class_decorator-chance to modify the class", CLS return CLS @my_class_decoratorclass C (object): __metaclass__ = My_metaclass def __new__ (CLS): print "-c.__new    __-Creating object. " Return Super (C, CLS). __new__ (CLS) def __init__ (self): print "-c.__init__-Initializing object." C = C () print "OBJECT C = ", C 

Now, you can take a few minutes to test your understanding and guess the order of the printouts.

First, let's take a look at how the Python interpreter reads this part of the code, and then we'll match the output to deepen our understanding.

1. Python first looks at the class declaration and prepares three arguments passed to the Meta class. These three parameters are the class name (Class_name), the parent class (parented), and the attribute list (attributs), respectively.

2. Python checks the __metaclass__ property, and if this property is set, it calls Metaclass, passes three arguments, and returns a class.

3. In this example, Metaclass is itself a class, so invoking it is like creating a new class. This means that my_metaclass.__new__ will be called first, entering four parameters, which will create a new instance of the Metaclass class. Then the my_metaclass.__init__ of this instance will be called the result is returned as a new class object. So at this point C will be set to this class object.

4. Next python will look at all adorners decorated with this class. In this example, there is only one adorner. Python will call this adorner and pass the class from the Meta class to it as a parameter. The class will then be replaced by the object returned by the adorner.

5. The adorner returns the same class type as the meta-class setting.

6. When a class is called to create a new object instance, because the class type is Metaclass, Python will call the __call__ method of the meta-class. In this example, my_metaclass.__call__ simply invokes the type.__call__ to create an object instance of the class passed to it.

7. Next type.__call__ create an object from c.__new__.

8. Finally type.__call__ runs c.__init__ with the results returned by c.__new__.

9. The returned object is ready for completion.

So based on the above analysis, we can see that the order of the calls is as follows: My_metaclass.__new__ is called first, then my_metaclass.__init__, then My_class_decorator. At this point the Class C is ready (The return result is C). When we call C to create an object, we first call my_metaclass.__call__ (when any object is created, Python will first call the __call__ method of its class), and c.__new__ will be type.__call__ Call (my_metaclass.__call__ simple call to type.__call__), and finally c.__init__ is called. Now let's take a look at the output:

-my_metaclass.__new__-Creating class instance of type 
 
  
   
  -my_metaclass.__init__-Initializing the class ins Tance 
  
   
    
   -My_class_decorator-chance to modify the class 
   
    
     
    -my_metaclass.__call__-Creating Obje CT of type 
    
     
      
     -c.__new__-Creating object.-c.__init__-Initializing object. Object C = <__main__. C object 
     at 
       0x1043feb90>
    
     
   
    
  
   
 
  

A few more words about the meta-class

Meta-class, a powerful and obscure technique. The results of searching for __metaclass__ on GitHub are mostly links to "cookbook" or other Python instructional materials. Some test cases (such as some of the test cases in Jython), or some other place where __metaclass__ = Type is written, are just to make sure that the new class is used properly. Frankly, these use cases do not really use meta-classes. Filtered down the results, I can only find two places that really use the Meta class: Abcmeta and Djangoplugins.

Abcmeta is a meta-class that allows you to register an abstract base class. If you want to learn more, check out its official documentation and this article will not discuss it.

For Djangoplugins, the basic idea is based on this article article on a-plugin framework for Python, which uses a meta-class to create a plug-in Mount system. I have not studied it in depth, but I feel that this function can be implemented using adorners. If you have a related idea, please leave a message after this article.
Summary Notes

By understanding the meta-classes can help us understand the behavior of classes and objects in Python in greater depth, the reality of using them may be much more complex than the examples in this article. Most meta-class functions can be accomplished using adorners. So when your first instinct is to use the meta-class to solve your problem, then stop and think about whether this is necessary. If you do not want to use a meta-class, then think twice. This will make your code easier to understand and easier to debug and maintain.

