For details about Classes and Metaclasses in Python, pythonmetaclasses

For details about Classes and Metaclasses in Python, pythonmetaclasses

Class and Object

Classes and functions are the same objects in Python. After a class is defined, Python creates a Class Object and assigns it to a variable with the same name. Class is a type object (is it a bit of a logging interface ?).

The class object is callable and can be called to create class objects. You can process a class as another object. For example, you can assign values to their attributes and assign them to a variable. You can use them wherever callable objects can be used, for example, in a map. In fact, when you are using map (str, [1, 2, 3]), you are converting a list of integer types to a list of string types, because str is a class. Let's look at the following code:

>>> class C(object):...   def __init__(self, s):...       print s...>>> myclass = C>>> type(C)<type 'type'>>>> type(myclass)<type 'type'>>>> myclass(2)2<__main__.C object at 0x10e2bea50>>>> map(myclass, [1,2,3])123[<__main__.C object at 0x10e2be9d0>, <__main__.C object at 0x10e2bead0>, <__main__.C object at 0x10e2beb10>]>>> map(C, [1,2,3])123[<__main__.C object at 0x10e2be950>, <__main__.C object at 0x10e2beb50>, <__main__.C object at 0x10e2beb90>]>>> C.test_attribute = True>>> myclass.test_attributeTrue

For this reason, the "class" keyword in Python does not have to appear in the Code main scope as it does in other languages (such as C ++. In Python, it can be nested in a function. For example, we can dynamically create classes during function running. Check the Code:

>>> def make_class(class_name):...   class C(object):...       def print_class_name(self):...           print class_name...   C.__name__ = class_name...   return C...>>> C1, C2 = map(make_class, ["C1", "C2"])>>> c1, c2 = C1(), C2()>>> c1.print_class_name()C1>>> c2.print_class_name()C2>>> type(c1)<class '__main__.C1'>>>> type(c2)<class '__main__.C2'>>>> c1.print_class_name.__closure__(<cell at 0x10ab6dbe8: str object at 0x10ab71530>,)

Note that the two classes created through make_class are different objects, so the objects created through make_class do not belong to the same type. As we did in the decorator, we manually set the class name after the class is created. Also note that the print_class_name method of the created class captures the closure and class_name of the class in a closure cell. If you are not clear about the concept of closure, you 'd better take a look at the previous article and review the content related to closures and decorators.
Metaclasses

If a class is an object that can make objects, what should it be called? (believe me, this is not a problem of having a chicken first or having an egg first )? The answer is Metaclasses ). Most common basic metadata classes are type. When a parameter is input, type simply returns the type of the input object, which does not involve metadata. However, when three parameters are input, type will assume the role of the meta-class, create a class based on the input parameters, and return. The input parameter is quite simple: the class name, the dictionary of the parent class and its parameters. The following two fields can be empty. Let's look at an example:
 

>>> MyClass = type("MyClass", (object,), {"my_attribute": 0})>>> type(MyClass)<type 'type'>>>> o = MyClass()>>> o.my_attribute0

Note that the second parameter is a tuple (the syntax looks strange and ends with a comma ). If you need to arrange a method in the class, create a function and pass it as an attribute as the third parameter, as shown in the following code:
 

>>> def myclass_init(self, my_attr):...   self.my_attribute = my_attr...>>> MyClass = type("MyClass", (object,), {"my_attribute": 0, "__init__": myclass_init})>>> o = MyClass("Test")>>> o.my_attribute'Test'>>> o.__init__<bound method MyClass.myclass_init of <__main__.MyClass object at 0x10ab72150>>

You can use a callable object (function or class) to customize the Meta class. This object requires three input parameters and returns an object. The _ metaclass _ attribute of a metaclass can be defined on a class. In the first example, let's do something interesting to see what we can do with the Meta class:
 

>>> def mymetaclass(name, parents, attributes):...   return "Hello"...>>> class C(object):...   __metaclass__ = mymetaclass...>>> print CHello>>> type(C)<type 'str'>

Note that the above Code, C simply points a variable reference to the string "Hello ". Of course, no one will write such code in practice. This is just a simple example to demonstrate the usage of the Meta class. Next we will do some more useful operations. In the second part of this series, we have seen how to use the decoration class to record the output of each method of the target class. Now we do the same thing, but this time we use the Meta class. We borrow the definition of the previously decorator:

def log_everything_metaclass(class_name, parents, attributes):  print "Creating class", class_name  myattributes = {}  for name, attr in attributes.items():    myattributes[name] = attr    if hasattr(attr, '__call__'):      myattributes[name] = logged("%b %d %Y - %H:%M:%S",                    class_name + ".")(attr)  return type(class_name, parents, myattributes) class C(object):  __metaclass__ = log_everything_metaclass   def __init__(self, x):    self.x = x   def print_x(self):    print self.x # Usage:print "Starting object creation"c = C("Test")c.print_x() # Output:Creating class CStarting object creation- Running 'C.__init__' on Aug 05 2013 - 13:50:58- Finished 'C.__init__', execution time = 0.000s- Running 'C.print_x' on Aug 05 2013 - 13:50:58Test- Finished 'C.print_x', execution time = 0.000s

As you can see, the class decorator has a lot in common with the Meta class. In fact, any function that can be completed with a class decorator can be implemented with metadata. The class decorator has a simple syntax structure and is easy to read. Therefore, it is recommended to use it. However, in terms of metadata, it can do more, because it runs before the class is created, and the class decorator runs only after the class is created. Remember this. Let's run both of them at the same time. Please note the sequence of running:
 

def my_metaclass(class_name, parents, attributes):  print "In metaclass, creating the class."  return type(class_name, parents, attributes) def my_class_decorator(class_):  print "In decorator, chance to modify the class."  return class_ @my_class_decoratorclass C(object):  __metaclass__ = my_metaclass   def __init__(self):    print "Creating object." c = C() # Output:In metaclass, creating the class.In decorator, chance to modify the class.Creating object.

An actual use case of the metadata

Let's consider a more useful instance. Suppose we are creating a collection of classes to process the ID3v2 tag Wikipedia used in MP3 music files. In short, a tag consists of frames, and each frame is identified by a four-character identifier (identifier. For example, TOPE identifies the original album, and TOAL identifies the original album name. If we want to write a separate class for each frame type and allow the ID3v2 tag library users to customize their own frame classes. Then, we can use the metacharacter to implement a class factory mode. The specific implementation method can be as follows:

frametype_class_dict = {} class ID3v2FrameClassFactory(object):  def __new__(cls, class_name, parents, attributes):    print "Creating class", class_name    # Here we could add some helper methods or attributes to c    c = type(class_name, parents, attributes)    if attributes['frame_identifier']:      frametype_class_dict[attributes['frame_identifier']] = c    return c   @staticmethod  def get_class_from_frame_identifier(frame_identifier):    return frametype_class_dict.get(frame_identifier) class ID3v2Frame(object):  frame_identifier = None  __metaclass__ = ID3v2FrameClassFactory  pass class ID3v2TitleFrame(ID3v2Frame):  __metaclass__ = ID3v2FrameClassFactory  frame_identifier = "TIT2" class ID3v2CommentFrame(ID3v2Frame):  __metaclass__ = ID3v2FrameClassFactory  frame_identifier = "COMM" title_class = ID3v2FrameClassFactory.get_class_from_frame_identifier('TIT2')comment_class = ID3v2FrameClassFactory.get_class_from_frame_identifier('COMM')print title_classprint comment_class # Output:Creating class ID3v2FrameCreating class ID3v2TitleFrameCreating class ID3v2CommentFrame<class '__main__.ID3v2TitleFrame'><class '__main__.ID3v2CommentFrame'>

Of course, the above Code can also be completed using the class decorator. The following is the corresponding code:

frametype_class_dict = {} class ID3v2FrameClass(object):  def __init__(self, frame_id):    self.frame_id = frame_id   def __call__(self, cls):    print "Decorating class", cls.__name__    # Here we could add some helper methods or attributes to c    if self.frame_id:      frametype_class_dict[self.frame_id] = cls    return cls   @staticmethod  def get_class_from_frame_identifier(frame_identifier):    return frametype_class_dict.get(frame_identifier) @ID3v2FrameClass(None)class ID3v2Frame(object):  pass @ID3v2FrameClass("TIT2")class ID3v2TitleFrame(ID3v2Frame):  pass @ID3v2FrameClass("COMM")class ID3v2CommentFrame(ID3v2Frame):  pass title_class = ID3v2FrameClass.get_class_from_frame_identifier('TIT2')comment_class = ID3v2FrameClass.get_class_from_frame_identifier('COMM')print title_classprint comment_class Decorating class ID3v2FrameDecorating class ID3v2TitleFrameDecorating class ID3v2CommentFrame<class '__main__.ID3v2TitleFrame'><class '__main__.ID3v2CommentFrame'>

As you can see, we can pass parameters directly to the decorator, but the Meta class cannot. Parameters must pass through attributes to the metadata class. For this reason, the solution of the decorator here is clearer and easier to maintain. However, you also need to note that when the modifier is called, the class has been created, which means that its attributes cannot be modified. For example, once the class is created, you cannot modify _ doc __. Let's look at the actual example:

>>> def mydecorator(cls):...   cls.__doc__ = "Test!"...   return cls...>>> @mydecorator... class C(object):...   """Docstring to be replaced with Test!"""...   pass...Traceback (most recent call last): File "<stdin>", line 2, in <module> File "<stdin>", line 2, in mydecoratorAttributeError: attribute '__doc__' of 'type' objects is not writable>>> def mymetaclass(cls, parents, attrs):...   attrs['__doc__'] = 'Test!'...   return type(cls, parents, attrs)...>>> class D(object):...   """Docstring to be replaced with Test!"""...   __metaclass__ = mymetaclass...>>> D.__doc__'Test!'

Generate metadata using type

As we said, the most basic metadata class is type and the class is usually of the type. The question naturally comes: What type is the type itself? The answer is type. That is to say, type is its own metadata. Although it sounds a little strange, it is feasible at the Python interpreter level.

Type itself is a class, and we can inherit the new class from it. These generated classes can also be used as meta classes, and their classes can be of the same type as the type. Let's look at the following example:
 

>>> class meta(type):...   def __new__(cls, class_name, parents, attributes):...       print "meta.__new__"...       return super(meta, cls).__new__(cls, class_name, parents, attributes)...   def __call__(self, *args, **kwargs):...       print "meta.__call__"...       return super(meta, self).__call__(*args, **kwargs)...>>> class C(object):...   __metaclass__ = meta...meta.__new__>>> c = C()meta.__call__>>> type(C)<class '__main__.meta'>

Note that when the class creates an object, the _ call _ function of the metaclass is called, and type. _ call _ is called to create an object. In the next section, we will integrate the above content.
Key points set

Assume that the Meta class of class C is my_metaclass and decorated by the decorator my_class_decorator. In addition, assume that my_metaclass is a class and is generated from type. Let's combine the content mentioned above and make a summary to show how class C and its objects are created. First, let's look at the Code:

class my_metaclass(type):  def __new__(cls, class_name, parents, attributes):    print "- my_metaclass.__new__ - Creating class instance of type", cls    return super(my_metaclass, cls).__new__(cls,                        class_name,                        parents,                        attributes)   def __init__(self, class_name, parents, attributes):    print "- my_metaclass.__init__ - Initializing the class instance", self    super(my_metaclass, self).__init__(self)   def __call__(self, *args, **kwargs):    print "- my_metaclass.__call__ - Creating object of type ", self    return super(my_metaclass, self).__call__(*args, **kwargs) def my_class_decorator(cls):  print "- my_class_decorator - Chance to modify the class", cls  return cls @my_class_decoratorclass C(object):  __metaclass__ = my_metaclass   def __new__(cls):    print "- C.__new__ - Creating object."    return super(C, cls).__new__(cls)   def __init__(self):    print "- C.__init__ - Initializing object." c = C()print "Object c =", c

Now, you can take a few minutes to test your understanding and guess the order of output.

First, let's take a look at how the Python interpreter reads this part of the code, and then we will get the corresponding output to deepen our understanding.

1. Python first looks at the class declaration and prepares three parameters passed to the Meta class. These three parameters are respectively the class name (class_name), the parent class (parent), and the attribute list (attributs ).

2. Python checks the _ metaclass _ attribute. If this attribute is set, it calls metaclass, passes three parameters, and returns a class.

3. In this example, metaclass itself is a class, so the process of calling it is similar to creating a new class. This means that my_metaclass. _ new _ will be called first, and four parameters will be input. This will create a new metaclass class instance. Then the my_metaclass. _ init _ of this instance will be called and the result will be returned as a new class object. Therefore, C will be set to this class object.

4. Next, Python will view all the decorators that have decorated this class. In this example, there is only one decorator. Python will call this decorator and pass the class obtained from the Meta class to it as a parameter. The class will be replaced by the objects returned by the decorator.

5. The class type returned by the decorator is the same as that set by the Meta class.

6. When a class is called to create a new object instance, Python will call the _ call _ method of the metaclass because the class type is metaclass. In this example, my_metaclass. _ call _ simply calls type. _ call __to create an object instance of the class passed to it.

7. Next, type. _ call _ creates an object through C. _ new.

8. Finally, type. _ call _ runs C. _ init _ through the result returned by C. _ new __.

9. The returned object is ready.

Based on the above analysis, we can see that the call sequence is as follows: my_metaclass. _ new _ is called first, then my_metaclass. _ init __, and then my_class_decorator. Now C class has been prepared (the returned result is C ). When we call C to create an object, we first call my_metaclass. _ call _ (when an object is created, Python first calls the _ call _ method of its class), and then C. _ new _ will be type. _ call (my_metaclass. _ call _ type is simply called. _ call _) and C. _ init _ is called. Now let's take a look at the output:
 

- my_metaclass.__new__ - Creating class instance of type <class '__main__.my_metaclass'>- my_metaclass.__init__ - Initializing the class instance <class '__main__.C'>- my_class_decorator - Chance to modify the class <class '__main__.C'>- my_metaclass.__call__ - Creating object of type <class '__main__.C'>- C.__new__ - Creating object.- C.__init__ - Initializing object.Object c = <__main__.C object at 0x1043feb90> <class '__main__.C'>

I would like to say a few more about the metadata

Meta, a powerful and obscure technique. Most of the results obtained by searching _ metaclass _ on GitHub are links to "cookbook" or other Python teaching materials. Some test cases (such as some test cases in Jython) or some other places with _ metaclass _ = type are only used to ensure that the new class is used normally. Frankly speaking, none of these use cases actually use the metadata. After filtering the results, I can only find two places where the Meta class is actually used: ABCMeta and djangoplugins.

ABCMeta is a metadata class that allows you to register abstract base classes. For more information, see its official documentation. This article will not discuss it.

For djangoplugins, the basic idea is based on this article on a simple plugin framework for Python. The Meta class is used to create a plug-in to mount the system. I did not have any in-depth research on it, but I feel that this function can be implemented using a decorator. If you have any related ideas, please leave a message after this article.
Summary Notes

Understanding meta-classes can help us better understand the behavior of classes and objects in Python. In reality, using them may be much more complex than the example in the document. Most of the functions completed by the Meta class can be implemented using the decorator. So when your first instinct is to use the Meta class to solve your problem, please stop and think about whether this is necessary. If you do not have to use a metadatabase, think twice. This makes your code easier to understand and debug and maintain.