Introduction to Meta-class programming in Python

Review Object-oriented programming

Let's take 30 seconds to review what OOP really is. In object-oriented programming languages, classes can be defined, and their purpose is to bundle related data and behavior together. These classes can inherit some or all of the properties of their parent class, but they can also define their own properties (data) or methods (behaviors). At the end of the process of defining a class, a class typically acts as a template for creating instances (and sometimes simply referred to as objects). Different instances of the same class often have different data, but "looks" are the same-for example, the Employee object Bob and Jane have. Salary and. Room_number, but both rooms and salaries vary.

Some OOP languages (including Python) allow objects to be self-introspective (also known as reflection). That is, the introspection object can describe itself: which class does the instance belong to? What are the ancestors of the class? What methods and properties can an object use? Introspection lets the function or method that handles the object make decisions based on the type of object passed to the function or method. Even without introspection, functions are often divided according to the instance data, for example, the route to Jane.room_number is different from the route to Bob.room_number, as it is both in different rooms. With introspection, you can also skip calculations for Bob while safely calculating Jane's winnings, for example, because Jane has a. Profit_share property, or because Bob is an instance of a subclass Hourly (Employee).

Meta-class programming (metaprogramming) answers

The basic OOP system outlined above is quite powerful. But one of the elements in the above description is not appreciated: in Python (and other languages), the class itself is an object that can be passed and introspective. As mentioned earlier, since you can use classes as templates to generate objects, what do you use as templates to generate classes? The answer is, of course, the meta-Class (Metaclass).

Python has always had a meta class. But the methods involved in the meta-class are better exposed to people in Python 2.2. Python V2.2 explicitly no longer uses only a special (usually hidden) meta-class to create each class object. Programmers can now create subclasses of primitive meta-class type, and can even generate classes dynamically with various meta-classes. Of course, just because you can manipulate the meta-class in Python 2.2 doesn't explain why you might want to do this.

Furthermore, you do not need to use custom meta classes to manipulate the generation of classes. A less-than-mind concept is a class factory: a common function that returns classes that are created dynamically in the body of a function. With the traditional Python syntax, you can write:
Listing 1. Old-fashioned Python 1.5. Class 2 Factory

Python 1.5.2 (#0, June 27 1999, 11:23:01) [...] Copyright 1991-1995 stichting Mathematisch Centrum, amsterdam>>> def class_with_method (func): ...   Class Klass:pass   ... SetAttr (Klass, func.__name__, func)   ... Return klass...>>> def say_foo (self): print ' foo ' ...>>> foo = class_with_method (say_foo) >>> foo = foo () >>> foo.say_foo () foo

The Factory function Class_with_method () dynamically creates a class and returns the class that contains the methods/functions passed to the factory. Before returning the class, manipulate the class itself in the body of the function. The new module provides a more concise way to encode, but the options are different from those in the class factory custom code, such as:
Listing 2. Class factory in the new module

>>> from new import classobj>>> Foo2 = classobj (' Foo2 ', (Foo,), {' Bar ': Lambda self: ' Bar '}) >>> Foo2 (). Bar () ' Bar ' >>> Foo2 (). Say_foo () foo

In all of these cases, the behavior of classes (Foo and Foo2) is not written directly as code, but instead the behavior of the class is created by invoking the function at run time with dynamic parameters. One thing to emphasize here is that not only can instances be created dynamically, but the classes themselves can also be created dynamically.

Meta-class: A solution for finding a problem?

The magic of the meta-class is so great that 99% of users have had concerns that are unnecessary. If you want to know if you need them, you can do it without them (those who actually need the meta-class really know that they need them and don't need to explain why). -python expert Tim Peters

A method (Class) can return an object as if it were a normal function. So in this sense, class factories can be classes, just as easily as they can be functions, which is obvious. In particular, Python 2.2+ provides a special class called type, which is exactly the class factory. Of course, the reader will realize that type () is not "ambitious" as the built-in function of the old Python version-Fortunately, the behavior of the old version of the type () function is maintained by the type class (in other words, type (obj) returns an object of obj, or Class). As the new type class factory, it works the same way as the function New.classobj has always been:
Listing 3. Type as class factory meta-class

>>> x = Type (' x ', (), {' Foo ': Lambda self: ' foo '}) >>> x, X (). Foo () (
 
  
   
  , ' foo ')
 
  

But since type is now a (meta) class, you can freely use it to create subclasses:
Listing 4. As a type descendant of a class factory

>>> class Chattytype (type): ...   Def __new__ (CLS, name, bases, DCT):     ... Print "Allocating memory for Class", name     ... Return type.__new__ (CLS, name, bases, DCT)   ... Def __init__ (CLS, name, bases, DCT):     ... print "Init ' ing (configuring) class", Name     ... Super (Chattytype, CLS). __INIT__ (name, bases, DCT) ...>>> X = Chattytype (' x ', (), {' Foo ': Lambda self: ' foo '}) allocating memory for Class XInit ' ing (Configuring) class X>>> X, X (). Foo () (
 
  
   
  , ' foo ')
 
  

The "Magic" of the. __new__ () and. __init__ () methods are special, but conceptually, for any other class, they work the same way: the __init__ () method enables you to configure the objects you create, and the. __new__ () method enables you to customize its allocation. Of course, the latter is not widely used, but this method exists for each Python 2.2 new style class (usually inherited rather than overwritten).

You need to be aware of a feature of type descendants; it often makes the first use of the meta-class "trap". By convention, the first parameter of these methods is called CLS, not self, because these methods operate on the generated class, not on the Meta class. In fact, there is nothing special about this; All methods are attached to their instances, and the instances of the meta classes are classes. The non-special name makes this more obvious:
Listing 5. Attaching a class method to the generated class

>>> class Printable (type): ...   def whoami (CLS): print "I am a", cls.__name__...>>> foo = Printable (' Foo ', (), {}) >>> Foo.whoami () I am a F Oo>>> Printable.whoami () Traceback (most recent call last): Typeerror:unbound method WhoAmI () [...]

All of these surprising but common practices and easy-to-grasp syntax make the use of meta-classes easier, but they also confuse new users. There are several elements for other grammars. But the parsing order of these new variants requires a bit of skill. Classes can inherit the meta-class from their ancestors-note that this is not the same as having a meta class as an ancestor (this is another often confusing place). For older classes, defining a global _metaclass_ variable can enforce the use of custom meta-classes. Most of the time, however, the safest approach is to set the class's _metaclass_ class properties when you want to create a class by customizing the meta-class. You must set the variable in the class definition itself, because if you later set the property after the class object has been created, the meta-class is not used. For example:
Listing 6. To set a meta class with class properties

>>> class Bar: ...   __metaclass__ = Printable   ... def foomethod (self): print ' foo ' ...>>> Bar.whoami () I am a bar>>> Bar (). Foomethod () foo

Use this "magic" to solve the problem.

At this point, we have learned some basic knowledge about meta-classes. However, to use a meta-class, it is more complex. The difficulty with using meta-classes is that, often in OOP design, classes do not actually do much. The inheritance structure of classes is useful for encapsulating and packaging data and methods, but in specific cases, instances are often used.

We think that the meta-class is really useful in two major types of programming tasks.

The first class, perhaps the more common one, is not exactly what the class needs to do at design time. Obviously, you know something about it, but a particular detail may depend on the information you get later. The "later" itself has two classes: (a) When the application uses the library module, and (b) at run time when a situation exists. This class is very close to what is commonly referred to as "aspect-oriented programming (aspect-oriented PROGRAMMING,AOP)". We'll show you an example of what we think is very chic:
Listing 7. Meta-class configuration at runtime

% Cat Dump.py#!/usr/bin/pythonimport sysif len (SYS.ARGV) > 2:  module, Metaklass = Sys.argv[1:3]  m = __import__ (module, Globals (), locals (), [Metaklass])  __metaclass__ = GetAttr (M, Metaklass) class Data:  def __init__ (self):    self.num =    Self.lst = [' A ', ' B ', ' C ']    self.str = ' spam '  dumps  = lambda self: ' self '  __str__ = lambda self:self.dumps () data = data () Print data% dump.py<__main__. Data instance at 1686a0>

As you would expect, the application prints out a fairly general description of the data object (a regular instance object). However, if you pass the run-time parameters to the application, you can get quite different results:
Listing 8. Add an external serialization meta class

% dump.py gnosis.magic metaxmlpickler<?xml version= "1.0"?>
 
   
  
    
  
    
  
   
 
  

This particular example uses Gnosis.xml.pickle's serialization style, but the newest Gnosis.magic package also contains the meta-class serializer Metayamldump, Metapypickler, and Metaprettyprint. Furthermore, users of the dump.py "application" can take advantage of the "Metapickler" from any Python package that defines any desired metapickler. For this purpose, write the appropriate meta-class as follows:
Listing 9. Adding attributes with a meta class

Class Metapickler (type):  "Metaclass for Gnosis.xml.pickle serialization"  def __init__ (CLS, name, bases, Dict): From    gnosis.xml.pickle import dumps    super (Metapickler, CLS). __INIT__ (name, bases, Dict)    setattr (CLS, ' Dumps ', dumps)

The advantage of this arrangement is that the application programmer does not need to know which serialization to use-not even the ability to add serialization or some other cross-section at the command line.

Perhaps the most common use of meta-classes is similar to Metapickler: adding, deleting, renaming, or replacing methods defined in the resulting class. In our example, the "native" Data.dump () method is overridden by a method other than the application when creating the class Data (and then creating each subsequent instance).

Other ways to use this "magic" to solve problems

There is a programming environment in which classes are often more important than instances. For example, a descriptive mini-language (declarative mini-languages) is a Python library that directly represents its program logic in a class declaration. David studied this issue in his article "Create declarative Mini-languages". In this case, it is useful to use a meta-class to influence the class creation process.

A class-based declarative framework is gnosis.xml.validity. In this framework, you can declare a number of "validation classes" that represent a set of constraints on valid XML documents. These declarations are very close to those contained in the DTD. For example, you can configure a "dissertation" document with the following code:
Listing 10. simple_diss.py gnosis.xml.validity Rules

From gnosis.xml.validity import *class figure (EMPTY):   passclass _mixedpara (Or):   _disjoins = (PCDATA, figure) class paragraph (Some):  _type = _mixedparaclass title (PCDATA):   passclass _paras (Some):    _type = Paragraphclass Chapter (SEQ):    _order = (title, _paras) class dissertation (Some): _type = Chapter

If you attempt to instantiate the dissertation class without the correct component child elements, a descriptive exception is generated, as is the case for each child element. When there is only one clear way to "elevate" a parameter to the correct type, the correct child element is generated from a simpler argument.

Even though the validity classes are often (informally) based on pre-existing DTDs, instances of these classes also print themselves as simple XML document fragments, for example:
Listing 11. Creation of a basic validation class document

>>> from Simple_diss import *>>> ch = liftseq (chapter, (' It starts ', ' when it began ') >>> print Ch
 
  
  
   It starts
  
   
    
   When it began
  
   
 
  

By using a meta-class to create a validation class, we can generate a DTD from the class declaration (we can add an extra method to these validation classes while doing so):
Listing 12. Using meta-classes during module import

>>> from gnosis.magic import Dtdgenerator, \ ...             Import_with_metaclass, \ ...             From_import>>> d = import_with_metaclass (' Simple_diss ', dtdgenerator) >>> From_import (d, ' * * ') > >> ch = liftseq (chapter, (' It starts ', ' when it Began ')) >>> print ch.with_internal_subset () <?xml Version= ' 1.0 '?>
 
  
 
  
 
  
 
   ]>
 
  
  
   It starts
  
   
    
   When it began
  
   
 
  

The package gnosis.xml.validity does not know the DTD and the internal subset. Those concepts and capabilities are introduced entirely by the meta-class dtdgenerator, making no changes to gnosis.xml.validity or simple_diss.py. Dtdgenerator does not replace its own. __str__ () method with the class it produces-you can still print simple XML fragments-but the meta-class can easily modify this "magic" approach.

Yuan to bring convenience

To use meta classes and some sample meta classes that can be used in aspect-oriented programming, package Gnosis.magic contains several utilities. One of the most important utilities is import_with_metaclass (). The function used in the example above allows you to import a third-party module, but you would create all the module classes with a custom meta-class instead of the type. Whatever new capabilities you want to give to third-party modules, you can define that capability within the meta-class you create (or get it from somewhere else). The gnosis.magic contains some pluggable serialization classes, and other packages may contain trace capabilities, object persistence, exception logging, or other capabilities.

The Import_with_metclass () function demonstrates several properties of meta-class programming:
Listing 13. [Gnosis.magic] of the Import_with_metaclass ()

def import_with_metaclass (ModName, Metaklass):  "Module Importer substituting Custom Metaclass"  class Meta ( Object): __metaclass__ = metaklass  DCT = {' __module__ ': modname}  mod = __import__ (modname)  for key, Val in MoD . __dict__.items ():    if Inspect.isclass (val):      setattr (mod, key, type (key, (Val,meta), DCT))  return mod

The notable style in this function is to generate a generic class meta with the specified meta-class. However, once the meta is added as an ancestor, a custom meta-class is used to generate its descendants. In principle, a class like meta can have either a meta-class generator (metaclass producer) or a set of inheritable methods-meta these two aspects of the class are irrelevant.

