Introduction to Meta-class programming in Python _python

Review object-oriented programming

Let's start with 30 seconds to review what OOP is all about. In object-oriented programming languages, you can define classes that are used to bundle related data and behavior together. These classes can inherit some or all of the properties of their parent classes, but they can also define their own properties (data) or methods (behavior). When the process of defining a class ends, the class usually acts as a template for creating instances, sometimes simply called objects. Different instances of the same class usually have different data, but "appearances" are the same-for example, the Employee object Bob and Jane have. Salary and. Room_number, but both have different rooms and salaries.

Some OOP languages (including Python) allow objects to be introspective (also known as reflection). That is, an introspective object can describe itself: which class does an instance belong to? What ancestors did the class have? What methods and properties can an object use? Introspection lets the function or method of the processing object be determined according to the type of object passed to the function or method. Even without introspection, functions are often divided according to the instance data, for example, the route to the Jane.room_number is different from the route to the Bob.room_number, because they are in different rooms. With introspection, you can also skip the calculation of Bob while safely calculating Jane's winnings, for example, because Jane has the. Profit_share attribute, or because Bob is an instance of subclass hourly (Employee).

The answer of Meta class programming (metaprogramming)

The basic OOP system features outlined above are quite powerful. But one element of the above description is not taken seriously: in Python (and other languages), the class itself is an object that can be passed and introspective. As mentioned earlier, since you can use a class as a template to generate an object, what is the template used to generate the class? The answer, of course, is the Meta Class (Metaclass).

Python has a meta class all the time. But the methods involved in the Meta class are better exposed to people in Python 2.2. The Python V2.2 explicitly no longer uses only a special (usually hidden) meta class to create each class object. Programmers can now create subclasses of primitive class type, and can even dynamically generate classes with various classes of primitives. Of course, just because you can manipulate the Meta class in Python 2.2 doesn't explain why you might want to do that.

Also, you do not need to use custom meta classes to manipulate the generation of classes. A less-than-brain concept is a class factory: an ordinary function that can return a class that is created dynamically in the function body. With the traditional Python syntax, you can write:
Listing 1. Vintage Python 1.5.2 class factory

Python 1.5.2 (#0, June 1999, 11:23:01) [...]
Copyright 1991-1995 stichting Mathematisch Centrum, Amsterdam
>>> def class_with_method (func):
...   class Klass:pass ...   SetAttr (Klass, func.__name__, func) ...   Return Klass ...
>>> def say_foo (self): print ' foo '
...
>>> foo = class_with_method (say_foo)
>>> foo = foo ()
>>> foo.say_foo ()
foo

The Factory function Class_with_method () creates a class dynamically and returns the class that contains the method/function passed to the factory. Before returning the class, the class itself is manipulated in the function body. The new module provides a simpler way to encode, but the options differ from the custom code options in the class factory, for example:
Listing 2. Class factory in the new module

>>> from new import classobj
>>> Foo2 = classobj (' Foo2 ', (Foo,), {' Bar ': Lambda self: ' Bar '})
>>> Foo2 (). Bar ()
' bar '
>>> Foo2 (). Say_foo ()
foo

In all of these cases, the behavior of the class (Foo and Foo2) is not written directly as code, but rather by invoking the function at run time with dynamic parameters to create the behavior of the class. The point to emphasize here is that not only can instances be created dynamically, but also the class itself can be created dynamically.

Meta class: Finding a solution to the problem?

The magic of the Meta class is so great that 99% of users have had concerns that are unnecessary. If you want to know if you need them, you can do without them (people who actually need the meta class do know they need them and don't need to explain why). -python expert Tim Peters

Method (Class) can return an object like a normal function. So in this sense, class factories can be classes, just as easily as they can be functions, which is obvious. In particular, the Python 2.2+ provides a special class called type, which is just such a factory. Of course, readers will recognize that type () is not as "ambitious" as the built-in functions of Python's old version-fortunately, the behavior of the old version of the type () function is maintained by the type class (in other words, type (obj) returns the types/classes of object obj). As a new type class for a class factory, it works the same way as the function New.classobj has always been:
Listing 3. Type as the class factory meta class

>>> x = Type (' x ', (), {' Foo ': Lambda self: ' foo '})
>>> x, X (). Foo ()
(<class ' __main__. X ', ' foo ')

But because type is now a (meta) class, you can use it freely to create subclasses:
Listing 4. As a type descendant of a class factory

>>> class Chattytype (type):
...   Def __new__ (CLS, name, bases, DCT):
...     Print "Allocating memory for Class", name
...     Return type.__new__ (CLS, name, bases, DCT) ...   Def __init__ (CLS, name, bases, DCT):
...     print "Init ' ing (configuring) class", Name
...     Super (Chattytype, CLS). __INIT__ (name, bases, DCT) ...
>>> X = Chattytype (' x ', (), {' Foo ': Lambda self: ' foo '})
allocating memory for class X
Init ' ing ( Configuring) class X
>>> x, X (). Foo ()
(<class ' __main__. X ', ' foo ')

Rich in "Magic". The __new__ () and. __init__ () methods are special, but conceptually, they work the same way for any other class. The. __init__ () method enables you to configure the objects you create;. __new__ () method enables you to customize its allocation. Of course, the latter is not widely used, but this method exists for each Python 2.2 new style class (usually through inheritance rather than overwrite).

Attention should be paid to an attribute of the type descendant, which often makes it a "trap" for people who use the Meta class for the first time. By convention, the first parameter of these methods is named CLS, not self, because these methods operate on the generated class, not on the Meta class. In fact, there is nothing special about this; All methods are attached to their instances, and instances of the class are classes. Non-special names make this more obvious:
Listing 5. Attaching a class method to the generated class

>>> class Printable (type):
...   def whoami (CLS): print "I am a", cls.__name__ ...
>>> Foo = printable (' foo ', (), {})
>>> Foo.whoami ()
I am a Foo
>>> Printable.whoami ()
Traceback (most recent call last):
Typeerror:unbound method WhoAmI () [...]

All of these surprising but common practices and easy to master syntax make the use of the Meta class easier, but also confuse new users. There are several elements for the other syntax. But the analytic order of these new variants requires some skill. A class can inherit a class from its ancestors-note that this is not the same as having a meta class as an ancestor (this is another place that is often confusing). For older classes, defining a global _metaclass_ variable can force custom meta classes to be used. But most of the time, the safest approach is to set the class's _metaclass_ class properties when you want to create a class by customizing the Meta class. You must set the variable in the class definition itself, because if you set the property later (after you have created the class object), the meta class is not used. For example:
Listing 6. To set the Meta class with class attributes

>>> class Bar:
...   __metaclass__ = Printable
...   def foomethod (self): print ' foo ' ...
>>> Bar.whoami ()
I am a Bar
>>> bar (). Foomethod ()
foo

Use this "magic" to solve the problem.

At this point, we have learned some basic knowledge about the Meta class. However, it is more complex to use the Meta class. The difficulty with using a meta class is that in OOP design, classes do not actually do much. The inheritance structure of a class is useful for encapsulating and packaging data and methods, but in specific cases, it is common to use instances.

We think that the Meta class is really useful in two broad class programming tasks.

The first (and perhaps more common) type is the inability to know exactly what a class needs to do at design time. Obviously, you know something about it, but a particular detail may depend on the information that you will get later. "Later" itself has two classes: (a) When the application uses the library module, (b) at run time, when a situation exists. This class is close to what is commonly called "aspect-oriented programming (Aspect-oriented PROGRAMMING,AOP)." We'll show you an example that we think is very chic:
Listing 7. Runtime meta-class configuration

% cat dump.py
#!/usr/bin/python
Import sys
if Len (SYS.ARGV) > 2:
  module, Metaklass = sys.argv[1:3]< C5/>m = __import__ (module, Globals (), locals (), [Metaklass])
  __metaclass__ = GetAttr (M, metaklass)
class data:< C8/>def __init__ (self):
    self.num =
    Self.lst = [' A ', ' B ', ' C ']
    self.str = ' spam ' dumps  = Lambda self: ' self '
  __str__ = Lambda self:self.dumps ()
data = data ()
print data
% dump.py
< __main__. Data instance at 1686a0>

As you would expect, the application prints a fairly general description of the data object (the regular instance object). However, if you pass a run-time parameter to an application, you can get quite different results:
Listing 8. Add an external serialization meta class

% dump.py gnosis.magic metaxmlpickler
<?xml version= "1.0"?>
<! DOCTYPE pyobject SYSTEM "Pyobjects.dtd" >
<pyobject module= "__main__" class= "Data" id= "720748" >
< attr name= "LST" type= "list" id= "980012" >
 <item type= "string" value= "a",/> <item type=
 "string" Value= "B"/>
 <item type= "string" value= "C"/>
</attr>
<attr name= "num" type= "numeric" Value= "/>"
<attr name= "str" type= "string" value= "spam"/>
</PyObject>

This particular example uses the Gnosis.xml.pickle serialization style, but the latest Gnosis.magic package also contains the meta-class serializer Metayamldump, Metapypickler, and Metaprettyprint. Also, users of dump.py "apps" can take advantage of the "Metapickler" from any Metapickler Python package that defines any expectations. Write the appropriate meta class for this purpose as follows:
Listing 9. Adding attributes with a meta class

Class Metapickler (type):
  "Metaclass for Gnosis.xml.pickle serialization"
  def __init__ (CLS, name, bases, Dict):
    from Gnosis.xml.pickle import dumps
    super (Metapickler, CLS). __INIT__ (name, bases, Dict)
    setattr (CLS, ' Dumps ', dumps)

The extraordinary thing about this arrangement is that application programmers don't need to know which serialization to use--or whether to add serialization or some other cross section to the command line.

Perhaps the most common usage of the Meta class is similar to Metapickler: adding, deleting, renaming, or replacing the methods defined in the resulting class. In our example, the native Data.dump () method is replaced by a method outside the application when the class Data (and then each subsequent instance) is created.

Other ways to solve problems with this "magic"

There is a programming environment where classes are often more important than instances. For example, the descriptive mini language (declarative mini-languages) is a Python library that directly represents its program logic in a class declaration. David studied the issue in his article "Create declarative Mini-languages". In this case, it is useful to use the Meta class to influence the class creation process.

A class-based declarative framework is gnosis.xml.validity. In this framework, you can declare a number of "validation classes" that represent a set of constraints on a valid XML document. The declarations are very close to those contained in the DTD. For example, you can configure a "dissertation" document with the following code:
Listing 10. simple_diss.py gnosis.xml.validity Rules

From gnosis.xml.validity Import *
class figure (EMPTY):   Pass
class _mixedpara (Or):   _disjoins = (PCDATA , figure)
class paragraph (Some):  _type = _mixedpara
class title (PCDATA):   Pass
class _paras (Some) :    _type = Paragraph
class chapter (SEQ):    _order = (title, _paras)
class dissertation (Some): _type = Chapter

If you try to instantiate the dissertation class without the correct component child elements, a descriptive exception is generated, and for each child element. When there is only one clear way to "elevate" the parameter to the correct type, the correct child elements are generated from simpler arguments.

Even though validation classes are often (informally) based on pre-existing DTDs, instances of these classes also print themselves into simple XML document fragments, such as:
Listing 11. The creation of basic validation class documents

>>> from Simple_diss import *
>>> ch = liftseq (chapter, (' It starts ', ' when it began ')
>&G t;> Print ch
<chapter><title>it starts</title>
<paragraph>when It began</ Paragraph>
</chapter>

By using the Meta class to create a validation class, we can generate a DTD from the class declaration (we can add one additional method to these validation classes while we do this):
Listing 12. Using the Meta class during module import

>>> from gnosis.magic import dtdgenerator, \
...             Import_with_metaclass, \
...             From_import
>>> d = import_with_metaclass (' Simple_diss ', dtdgenerator)
>>> From_import (d, ' * * ')
>>> ch = liftseq (chapter, (' It starts ', ' when it Began '))
>>> print Ch.with_internal_ Subset ()
<?xml version= ' 1.0 '?>
<! DOCTYPE Chapter [
<! ELEMENT Figure empty>
<! ELEMENT Dissertation (chapter) +>
<! ELEMENT Chapter (title,paragraph+) >
<! ELEMENT title (#PCDATA) >
<! ELEMENT paragraph ((#PCDATA |figure)) +>
]>
<chapter><title>it starts</title>
<paragraph>when it began</paragraph>
</chapter>

The package gnosis.xml.validity does not know about DTDs and internal subsets. Those concepts and capabilities are introduced entirely by the Meta class dtdgenerator, and no changes are made to gnosis.xml.validity or simple_diss.py. Dtdgenerator does not replace its own. __str__ () method with the class it produces-you can still print simple XML fragments-but the meta class can easily modify this rich "magic" method.

the convenience brought by the Yuan

To use the Meta class and some sample meta classes that can be used in aspect-oriented programming, the package Gnosis.magic contains several utilities. One of the most important utilities is import_with_metaclass (). This function used in the previous example allows you to import Third-party modules, but you want to create all module classes with a custom meta class instead of type. Whatever new capabilities you want to give to Third-party modules, you can define that capability (or get it from somewhere else) within the Meta class that you create. Gnosis.magic contains a number of pluggable serialization meta classes, and some other packages may contain tracking capabilities, object persistence, exception logging, or other capabilities.

The Import_with_metclass () function shows several properties of Meta class programming:
Listing 13. [Gnosis.magic] Import_with_metaclass ()

def import_with_metaclass (ModName, Metaklass):
  "Module Importer substituting Custom Metaclass"
  class Meta ( Object): __metaclass__ = metaklass
  DCT = {' __module__ ': modname}
  mod = __import__ (modname)
  for key, Val in MoD . __dict__.items ():
    if Inspect.isclass (val):
      setattr (mod, key, type (key, (Val,meta), DCT) return
  MoD

A notable style in this function is to generate a generic class Meta using the specified meta class. However, once the meta is added as an ancestor, the custom meta class is used to generate its descendants. In principle, classes such as meta can have either a class builder (metaclass producer) or a set of inheritable methods-meta the two aspects of the class are irrelevant.