Introduction to the meta-class in Python

This is a very hot post on stack overflow. The questioner claims to have mastered various concepts in Python OOP programming, but has always felt that the meta-class (Metaclass) is difficult to understand. He knows that this must be related to introspection, but still not very clear, and hopefully you can give some practical examples and snippets to help understand and under what circumstances you need to do meta-programming. So the E-satis classmate gave the God general reply, the reply received 985 points of approval points, more people commented that this reply should be added to the official Python document. And the E-satis classmate himself in the stack overflow prestige points also up to 64271 points. Here is the wonderful reply (hint: very long)

Class is also an object

Before you understand the meta-class, you need to master the classes in Python first. The concept of class in Python draws on Smalltalk, which is somewhat peculiar. In most programming languages, a class is a set of code snippets that describe how to generate an object. This is still true in Python:

>>> class Objectcreator (object): ...       pass...>>> my_object = Objectcreator () >>> print My_object

<__main__.objectcreator object= "" at= "" 0x8974f2c= "" >

However, the classes in Python are far more than that. Class is also an object. Yes, yes, that's the object. As long as you use the keyword Class,python The interpreter will create an object when it executes. The following code snippet:

>>> class Objectcreator (object): ...       Pass ...

An object is created in memory and the name is Objectcreator. This object (class) itself has the ability to create an object (class instance), and that is why it is a class. However, its nature is still an object, so you can do the following with it:

1) You can assign it to a variable

2) You can copy it

3) You can add properties to it

4) You can pass it as a function parameter

Here's an example:

>>> Print Objectcreator     # You can print a class because it's actually an object >>> def Echo (o): ...       Print o...>>> echo (objectcreator)            # You can pass the class as parameter to function >>> print hasattr (objectcreator, ' New_ Attribute ') fasle>>> objectcreator.new_attribute = ' foo ' #  You can add properties to the class >>> print hasattr ( Objectcreator, ' New_attribute ') true>>> print objectcreator.new_attributefoo>>> Objectcreatormirror = objectcreator # You can assign a class value to a variable >>> print objectcreatormirror () <__main__.objectcreator object= "at=" "0x8997b4c=" ">

Dynamically creating classes

Because classes are also objects, you can create them dynamically at run time, just like any other object. First, you can create a class in a function, using the class keyword.

>>> def choose_class (name): ...       If name = = ' Foo ': ...           Class Foo (object): ...               Pass ...           Return Foo     # Returns a class, not an instance       of the class ... else:           ... Class Bar (object): ...               Pass ...           return bar...>>> MyClass = Choose_class (' foo ') >>> print MyClass              # function return class, not instance of class >>> print MyClass ()            # You can create class instances through this class, which is the object <__main__.foo object= "at=" "0x89c6d4c=" >

But this is not dynamic enough, because you still need to write your own code for the entire class. Because classes are also objects, they must be generated by something. When you use the class keyword, the Python interpreter automatically creates the object. But like most things in Python, Python still provides you with a way to handle it manually. Do you remember the built-in function type? This ancient but powerful function lets you know what the type of an object is, just like this:

>>> print type (1) >>> Print type ("1") >>> print type (objectcreator) >>> print type ( Objectcreator ())

Here, type has a completely different ability, and it can also dynamically create classes. The type can accept a description of a class as a parameter and then return a class. (I know that, depending on the parameters passed in, it's a silly thing to have two different uses of the same function, but that's in python to keep backwards compatibility)

Type can work like this:

Type (the class name, the tuple of the parent class (which can be empty for inheritance), the dictionary that contains the property (name and value))

For example, the following code:

>>> class Myshinyclass (object): ...       Pass

You can create it manually like this:

>>> Myshinyclass = Type (' Myshinyclass ', (), {})  # Returns a Class object >>> print myshinyclass>>> print Myshinyclass ()  #  Create an instance of this class <__main__.myshinyclass object= "" at= "0x8997cec=" ">

You will find that we use "Myshinyclass" as the class name, and we can also treat it as a variable as a reference to the class. Classes and variables are different, and there's no reason to complicate things.

The type accepts a dictionary to define properties for the class, so

>>> class Foo (object): ...       Bar = True

Can be translated as:

>>> foo = type (' Foo ', (), {' Bar ': True})

And you can use Foo as a normal class:

>>> Print foo>>> print foo.bartrue>>> f = Foo () >>> print F<__main__.foo object= "" at= "0x8a9b84c=" ">>>> print F.bartrue

Of course, you can inherit from this class, so the following code:

>>> class Foochild (Foo): ...       Pass

Can be written as:

>>> foochild = Type (' Foochild ', (Foo,), {}) >>> print foochild>>> print Foochild.bar   # The bar attribute is inherited by Foo and true

Eventually you will want to add a method to your class. You just need to define a function with the proper signature and assign it as a property.

>>> def echo_bar (self): ...       Print self.bar...>>> foochild = Type (' Foochild ', (foo,), {' Echo_bar ': Echo_bar}) >>> hasattr (foo, ' Echo_ Bar ') false>>> hasattr (foochild, ' Echo_bar ') true>>> My_foo = Foochild () >>> My_foo.echo_bar () True

As you can see, in Python, classes are also objects, and you can create classes dynamically. This is what Python does behind the scenes when you use the keyword class, and this is done through the Meta class.

What the hell is a meta-class (finally to the subject)

A meta-class is a "thing" used to create a class. You create a class just to create an instance object of the class, don't you? But we've learned that the classes in Python are also objects. Well, the meta-class is used to create these classes (objects), the meta-class is the class class, you can understand that:

MyClass = Metaclass () MyObject = MyClass ()

You've seen the type that allows you to do something like this:

MyClass = Type (' MyClass ', (), {})

This is because the function type is actually a meta class. The type is the meta-class that Python uses to create all the classes behind. Now you want to know why is the type all in lowercase instead of type? Well, I guess that's for consistency with STR, STR is the class used to create the string object, and int is the class used to create the integer object. Type is the class that creates the class object. You can see this by checking the __class__ property. All the things in Python, note that I mean all things--are objects. This includes integers, strings, functions, and classes. They are all objects, and they are all created from a class.

>>> age = 35>>> age.__class__>>> name = ' Bob ' >>> name.__class__>>> def foo ( ): Pass>>>foo.__class__>>> class Bar (object): pass>>> B = Bar () >>> b.__class__

Now, what is the __class__ attribute for any one __class__?

>>> a.__class__.__class__>>> age.__class__.__class__>>> foo.__class__.__class__> >> b.__class__.__class__

Therefore, a meta-class is something that creates objects of this class. If you like, you can call the Meta class "class factory" (Don't confuse the factory class: D) type is the inner Jianyuan class of Python, and of course, you can also create your own meta-class.

__metaclass__ Property

You can add the __metaclass__ property to a class when you write it.

Class Foo (object):    __metaclass__ = something ... [...]

If you do this, Python will use the Meta class to create the class Foo. Be careful, there's some tricks in there. You first write class Foo (object), but the Class object Foo is not yet created in memory. Python will look for the __metaclass__ attribute in the class definition, and if found, Python will use it to create the class Foo, and if not found, it will use the built-in type to create the class. Read this passage a few times. When you write the following code:

Class Foo (Bar):

Pass

Python does the following:

Is there a __metaclass__ attribute in Foo? If so, Python will create a class object named Foo in memory via __metaclass__ (I'm talking about class objects, please follow my line of thought). If Python does not find __metaclass__, it will continue to look for the __metaclass__ attribute in bar (parent class) and try to do the same thing as before. If Python cannot find __metaclass__ in any parent class, it will look for __metaclass__ in the module hierarchy and try to do the same. If you still can't find __metaclass__,python, you'll use the built-in type to create this class object.

Now the question is, what code can you put in the __metaclass__? The answer is: You can create something of a class. So what can be used to create a class? Type, or anything that uses the type or subclass type.

Custom meta-classes

The primary purpose of the meta-class is to automatically change the class when the class is created. Typically, you do something like this for the API, and you want to create a class that matches the current context. Imagine a silly example where you decide that the properties of all classes in your module should be in uppercase form. There are several ways to do this, but one is by setting __metaclass__ at the module level. In this way, all classes in this module are created by this meta-class, and we just need to tell the Meta class to change all the attributes to uppercase.

Fortunately, __metaclass__ can actually be called arbitrarily, and it doesn't need to be a formal class (I know that something with a ' class ' in some names doesn't need to be a class, and it's helpful to draw a drawing comprehension). So, let's start with a simple function as an example.

# The meta-class automatically passes the arguments you normally pass to ' type ' as your own arguments

def upper_attr (Future_class_name, Future_class_parents, future_class_attr):    "Returns a class object, converts the attribute to uppercase"    #  Select all properties that do not start with ' __ '    attrs = ((name, value) for name, value in Future_class_attr.items () if not name.startswith (' __ ')    # convert them to uppercase    uppercase_attr = Dict ((Name.upper (), value) for name, value in Attrs)     # Make class object creation    by ' type ' return type (Future_class_name, future_class_parents, uppercase_attr) __metaclass__ = upper_attr  #  This will work for all classes in this module, Class Foo (object):    # We can also define __metaclass__ here only, which will only work in this class    bar = ' Bip ' Print hasattr (Foo, ' Bar ') # Output: Falseprint hasattr (foo, ' BAR ') # output: True f = Foo () print f.bar# output: ' Bip '

Now let's do it again, this time using a real class as a meta class.

# Remember, ' type ' is actually a class, just like ' str ' and ' int ' # so you can inherit class Upperattrmetaclass from type: # __new__ is a special method called before __init__ # _ _new__ is the method used to create the object and return it # and __init__ is just used to initialize the passed parameters to the object # you rarely use __new__ unless you want to be able to control the creation of objects # here, the object is the class, we want to be able to customize it, so we rewrite __ here NEW__ # If you want, you can also do something in __init__ # There are some advanced uses that involve rewriting __call__ special methods, but we don't use def __new__ here (Upperattr_metaclass, future _class_name, Future_class_parents, future_class_attr): Attrs = ((name, value) for name, value in Future_class_attr.        Items () If not name.startswith (' __ ')) Uppercase_attr = Dict ((Name.upper (), value) as name, value in Attrs) return type (Future_class_name, future_class_parents, uppercase_attr) However, this approach is not oop. We called the type directly, and we didn't rewrite the __new__ method of the parent class. Now let's deal with this: Class Upperattrmetaclass (type): Def __new__ (Upperattr_metaclass, Future_class_name, future_class_parents , future_class_attr): Attrs = ((name, value) for name, value in Future_class_attr.items () if not Name.startswith (' _ _ ')) Uppercase_attr = Dict ((Name.upper (), value) for name, value in Attrs) # multiplexing type.__new__ Method # This is basic OOP programming, no magic return type._ _new__ (Upperattr_metaclass, Future_class_name, future_class_parents, uppercase_attr)

You may have noticed that there is an extra parameter upperattr_metaclass, which is nothing special. The first argument of a class method always represents the current instance, just like the self parameter in a normal class method. Of course, for the sake of clarity, the name here is very long. But like self, all parameters have their traditional names. Therefore, in the real product code a meta-class should be like this:

Class Upperattrmetaclass (Type):    def __new__ (CLS, name, bases, DCT):        attrs = ((name, value) for name, value in DCT. Items () If not name.startswith (' __ ')        uppercase_attr  = Dict ((Name.upper (), value) as name, value in Attrs)        Return type.__new__ (CLS, name, bases, uppercase_attr) If you use the Super method, we can also make it clearer, which alleviates inheritance (yes, you can have a meta class, inherit from a meta class, Inherit from type class Upperattrmetaclass (type):    def __new__ (CLS, name, bases, DCT):        attrs = ((name, value) for name, Valu E in Dct.items () if not name.startswith (' __ '))        uppercase_attr = Dict ((Name.upper (), value) as name, value in Attrs) 
  return Super (Upperattrmetaclass, CLS). __new__ (CLS, name, bases, uppercase_attr)

That's it, besides, there's really nothing else to say about the meta-class. The code used to use the meta-class is more complex, not because of the meta-class itself, but because you typically use meta-classes to do obscure things, rely on introspection, control inheritance, and so on. Indeed, it is particularly useful to use meta-classes to make some "dark magic", so that something complicated will come up. But in terms of the meta-class itself, they are really simple:

1) Creation of interception classes

2) Modify the class

3) return the modified class

Why use the Metaclass class instead of the function?

Since __metaclass__ can accept any callable object, why use a class, because it is obviously more complex to use the class? There are several reasons for this:

1) intentions will be clearer. When you read Upperattrmetaclass (type), you know what's going to happen next.

2) You can use OOP programming. A meta-class can inherit from a meta-class, overwriting the method of the parent class. A meta class can even use a meta class.

3) You can organize the code better. When you use the meta-class, it's certainly not a simple scenario like the one I've mentioned above, and it's usually about more complex issues. It can be helpful to which comes multiple methods into a class and make the code easier to read.

4) You can use special methods such as __new__, __init__ and __call__. They can help you with different tasks. Even if you can usually dispose of everything in __new__, some people still feel more comfortable with __init__.

5) Wow, the name of this thing is metaclass, certainly non-angel, I have to be careful!

Why should we use meta-classes?

Now back to our big theme, why are you going to use such an error-prone and obscure feature? Well, generally speaking, you don't use it at all:

"Meta-class is the magic of depth, 99% of users should not worry about it at all." If you want to figure out whether you need to use a meta-class, then you don't need it. Those who actually use the meta-class know very well what they need to do and do not need to explain why they use the meta-class at all. "The leader of the--python world Tim Peters

The primary purpose of the meta-class is to create an API. A typical example is the Django ORM. It allows you to define like this:

Class person (models. Model):    name = models. Charfield (max_length=30) Age    = models. Integerfield ()

But if you do it like this:

Guy  = person (name= ' Bob ', age= ') print Guy.age

This does not return an Integerfield object, but it returns an int that can even fetch data directly from the database. This is possible because of the models. The model defines the __metaclass__ and uses some magic to transform the simple person class you just defined into a complex hook to the database. The Django framework will simplify these seemingly complex things by exposing a simple API that uses meta-classes, recreating the code through this API, and doing real work behind the scenes.

Conclusion

First, you know that a class is actually an object that can create an instance of a class. Well, actually, the classes themselves are instances, of course, they are instances of the Meta class.

>>>class Foo (object): Pass>>> ID (foo)

Everything in Python is an object, either an instance of a class or an instance of a meta class, except for type. Type is actually its own meta-class, which is not something you can do in a pure Python environment, by playing some small tricks on the implementation level. Second, the meta-class is very complex. For very simple classes, you might not want to modify the class by using a meta-class. There are two other techniques you can use to modify a class:

1) Monkey Patching

2) Class Decorators

When you need to dynamically modify classes, you should use both of these techniques in 99% of the time. Of course, in 99% of the time you do not need to dynamically modify the class:D