A preliminary discussion of Python 3, part 2nd: Advanced Topics

Python 3 is the latest version of Guido van Rossum's powerful universal programming language. Although it breaks backwards compatibility with the 2.x version, it cleans up some grammatical issues. This article is the second in a two-part series that builds on the previous installment of this series, covering more new features and more advanced topics such as the changes in abstract base classes, meta classes, and modifiers.

Cesar Otero, consultant, freelance

May 04, 2009

• Content

The previous article about Python version 3-, python , or py3k-discussed some basic changes in python that break backwards compatibility, such as new print() functions, bytes data types, and type of change. This article is part 2nd of this series and explores more advanced topics such as abstract base classes (ABC), meta classes, function annotations and modifiers (decorator), integer literal support, numeric type hierarchies, and throwing and catching exceptions. Most of these features will still break backwards compatibility with the version 2x product line.

class modifier

In previous versions of Python, the conversion of a method must take place after the method definition. For longer methods, this requirement will define an important part of the separation from the external interface definition given by Python enhancement proposal (PEP) 318 (see Resources for links). The following code snippet illustrates this conversion requirement:

Listing 1. Method conversions in previous versions of Python 3

def myMethod (self):    # do somethingmymethod = Transformmethod (MyMethod)

To make such scenarios easier to read, and to avoid having to reuse the same method names multiple times, method modifiers were introduced in Python version 2.4.

modifiers are methods that can modify other methods and return a method or another callable object. Their comments are preceded by the modifier's name with an "at" sign ( @ )-similar to the syntax of a Java™ comment. Listing 2 shows the modifiers in the actual application.

Listing 2. A modifier method

@transformMethoddef MyMethod (self):    # do something

Modifiers are pure syntactic sugars (syntactic sugar)-or, as Wikipedia puts it, "a complement to the grammar of computer languages, which does not affect the functionality of the language, but makes it easier for people to use it." A common use of modifiers is to annotate static methods. For example, listing 1 and Listing 2 are equivalent, but listing 2 is easier for people to read.

Defining modifiers is no different than defining other methods:

Def mod (method):    method.__name__ = "John"    return method@moddef Modme ():    passprint (modme.__name__)

Even better, Python 3 now supports not only modifiers for methods, but also modifiers for classes, so instead of using the following:

Class MyClass:    Passmyclass = Dosomethingornotwithclass (MyClass)

We can use this:

@doSomethingOrNotWithClassclass MyClass:    Pass

Back to top of page

Meta class

A meta-class is a class that is an instance of these classes. Python 3 retains the built-in type for creating additional meta classes or dynamically creating classes at run time metaclass . The following syntax is still valid:

>>>aclass = Type (' ClassName ',    (object,),    {' Magicmethod ': Lambda cls:print ("Blah Blah")})

The arguments that are accepted by the preceding syntax include the string that is the class name, the tuple of the inherited object (which can be an empty tuple), and a dictionary that contains the methods that can be added (or it can be empty). Of course, you can also inherit from a type and create your own meta-class:

Class meta (type):    def __new__ (CLS, ClassName, BaseClasses, dictofmethods):        return type.__new__ (CLS, className , BaseClasses, Dictofmethods)

Only the parameters of the keyword are allowed

Python 3 has changed the way function parameters are assigned to the parameter slot (parameter slot), and you can use an asterisk () in the arguments passed in * to not accept variable-length parameters. The named arguments must be * followed-for example, def meth(*, arg1): pass . For more information, please refer to the Python documentation or PEP 3102 (see Resources for links).

Note: If the above two examples do not play any role, I highly recommend reading the series on meta-classes written by David Mertz and Michele Simionato. For related links, see resources.

Note that now, in the class definition, the keyword argument is allowed to appear after the base class list-Typically, that is class Foo(*bases, **kwds): pass . Use the keyword parameter to metaclass pass the meta-class to the class definition. Like what:

>>>class AClass (BaseClass1, baseClass2, Metaclass = ametaclass): Pass

The old meta-class syntax is to assign this meta-class to built-in properties __metaclass__ :

Class Test (object):    __metaclass__ = Type

And, now that we have a new property- __prepare__ -We can use this property to create a dictionary for the new class namespace. It is called before the class body is processed, as shown in Listing 3.

Listing 3. Use a simple meta-class of the __prepare__ attribute

def meth ():    print ("Calling method") class Mymeta (type):    @classmethod    def __prepare__ (CLS, name, baseclasses):        return {' Meth ': Meth}    def __new__ (CLS, name, BaseClasses, classdict):        return type.__new__ ( CLS, name, BaseClasses, Classdict) class Test (Metaclass = Mymeta):    def __init__ (self):        pass    attr = ' an Attribute ' t = Test () print (t.attr)

We've excerpted a more interesting example from PEP 3115, as shown in Listing 4, which creates a meta-class with a list of its method names, while preserving the order of class method declarations.

Listing 4. A meta-class that maintains the order of class members

# the Custom Dictionaryclass member_table (dict): def __init__ (self): self.member_names        = [] def __setitem__ (self, Key, value): # If the key isn't already defined, add to the # List of keys.  If key not in Self:self.member_names.append (key) # superclass dict.__setitem__ (self,  Key, Value) # The Metaclassclass orderedclass (type): # The Prepare function @classmethod def __prepare__ (Metacls, Name, bases): # keywords in this case return member_table () # The Metaclass invocation def __new__ (CLS, n Ame, bases, Classdict): # Note that we replace the classdict with a regular # dict before passing it to the        Superclass, so and we # don ' t continue to record member names after the class # have been created.  result = type.__new__ (CLS, name, bases, Dict (classdict)) Result.member_names = Classdict.member_names return Result 

There are several reasons for these changes within the Meta class. The methods of the object are generally stored in a dictionary, and the dictionary is not sequential. In some cases, however, it is useful to maintain the order of the declared class members. This can be accomplished by allowing this meta class to be created earlier when the information is still available-this is useful, for example, in the creation of C structures. This new mechanism also enables other interesting features to be implemented in the future, such as inserting symbols into the body of the created class namespace and forward references to symbols during class building. PEP 3115 mentions that there are aesthetic reasons for changing the syntax, but there is still a debate that cannot be resolved with objective criteria (see Resources for a link to PEP 3115).

Back to top of page

Abstract base class

As I discussed in Python 3, part 1th: The new features of Python 3, ABC is a class that cannot be instantiated. Programmers in the Java or C + + language should be familiar with this concept. Python 3 Adds a new framework-abc-it provides support for ABC.

This ABC module has a meta class ( ABCMeta ) and modifier ( @abstractmethod and @abstractproperty ). If an ABC has one @abstractmethod or @abstractproperty , it cannot be instantiated, but must be overridden within a subclass. For example, the following code:

>>>from ABC import *>>>class c (metaclass = Abcmeta): Pass>>>c = C ()

This code is possible, but it cannot be encoded as follows:

>>>from ABC import *>>>class C (metaclass = Abcmeta): ...    @abstractmethod ... def absmethod (self): ...        Pass>>>c = C () Traceback (most recent call last): File "<stdin>", line 1, in <module>typeerror:can ' t Instantiate abstract class C with abstract methods Absmethod

A better approach would be to use the following code:

>>>class B (C): ...    def absmethod (self): ...        Print ("Now a Concrete method") >>>b = B () >>>b.absmethod () now a concrete method

ABCMetaClass overrides the properties __instancecheck__ and __subclasscheck__ , in this can overload the built-in functions isinstance() and issubclass() . To add a virtual subclass to ABC, you can use ABCMeta the provided register() method. A simple example is as follows:

>>>class testabc (Metaclass=abcmeta): Pass>>>testabc.register (list) >>>testabc.__ INSTANCECHECK__ ([]) True

It is equivalent to using isinstance(list, TestABC) . You may have noticed that Python 3 is used, __instancecheck__ rather than, __issubclass__ used __subclasscheck__ , rather than __issubclass__ , which looks more natural. If the parameters are isinstance(subclass, superclass) reversed, for example superclass.__isinstance__(subclass) , it may cause confusion. Obviously, the syntax is a superclass.__instancecheck__(subclass) little better.

collectionswithin the module, you can use several ABC to test whether a class provides a specific interface:

>>>from Collections Import Iterable>>>issubclass (list, iterable) True

Table 1 shows the ABC for this set of frames.

Table 1. The ABC of this collection frame

ABC	Inherits
Container
Hashable
Iterable
Iterator	Iterable
Sized
Callable
Sequence	Sized, Iterable,Container
MutableSequence	Sequence
Set	Sized, Iterable,Container
MutableSet	Set
Mapping	Sized, Iterable,Container
MutableMapping	Mapping
MappingView	Sized
KeysView	MappingView,Set
ItemsView	MappingView,Set
ValuesView	MappingView

Collection

The collection framework includes the container data type, the double-ended queue (that is, deque), defaultdict and a default dictionary (that is). A deque supports appending and ejecting from the front or back. defaultdicta container is a subclass of a built-in dictionary that overwrites a method and adds a writable instance variable (according to a Python 3 document). "In addition, it serves as a dictionary. In addition, the collection framework provides a data type factory function namedtuple() .

ABC type hierarchy

Python 3 now supports type hierarchies that can represent the ABC of a numeric class. These ABC exist numbers within the module and include Number , Complex , Real , Rational and Integral . Figure 1 shows this numerical hierarchy. You can use them to implement your own numeric type or other numeric ABC.

Figure 1. Numeric hierarchies

Value Tower (numerical)

Python's numerical hierarchy is inspired by the value tower of the Scheme language.

The new module fractions can implement this value ABC Rational . This module provides support for the rational number algorithm. If dir(fractions.Fraction) you use it, you will notice that it has some properties, such as imag , real and __complex__ . According to the theory of numerical tower, the reason lies in Rationals inheriting from Reals , Reals inheriting Complex from.

Back to top of page

Throwing and catching exceptions

Within Python 3, the except statement has been modified to handle the problem of ambiguous syntax. Previously, in Python version 2.5, the try . . . except structure, such as:

>>>try:    ... x = float (' Not a number ') ... except (ValueError, Nameerror): ...    Print "Can ' t convert type"

May be incorrectly written as:

>>> Try:    ... x = float (' Not a number ') ... except ValueError, Nameerror: ...    Print "Can ' t convert type"

The problem with the latter format is that the ValueError exception will never be caught because the interpreter will catch ValueError and bind this exception object to a name NameError . This can be seen in the following example:

>>> Try:    ... x = float (' blah ') ... except ValueError, Nameerror: ...    Print "Nameerror is", nameerror ... Nameerror is invalid literal for float (): Not a number

So, in order to handle the problem of syntax ambiguity, when you want to bind this exception object to another name, the comma ( , ) is replaced with the keyword as . If you want to capture multiple exceptions, you must use parentheses ( () ). The code in Listing 5 shows two syntax-compliant examples in Python 3.

Listing 5. Exception Handling in Python 3

# bind ValueError object to local name extry:    x = float (' blah ') except ValueError as ex:    print ("Value exception OCC Urred ", ex) # catch, different exceptions simultaneouslytry:    x = float (' blah ') except (ValueError, nameerror):    Print ("Caught both types of exceptions")

Another change to exception handling is the exception chain -implicit or explicit. Listing 6 shows an example of an implicit exception chain.

Listing 6. Implicit exception chains in Python 3

Def divide (A, B):    try:        print (A/b)    except Exception as exc:        def log (exc):        FID = open (' LogFile.txt ') # missing ' W '        print (exc, file=fid)        fid.close ()        log (exc) divide (1,0)

divide()Method attempts to perform a division with a divisor of zero, which throws an exception: ZeroDivisionError . However, within the nested method of the exception statement log() , print(exc, file=fid) an attempt is made to write to a file that has not yet been opened. Python 3 throws an exception, as shown in Listing 7.

Listing 7. Tracing for an implicit exception chain example

traceback (most recent call last): File "chainexceptionexample1.py", line 3, in divide print (a b) Zerodivisionerror:int division or modulo by zeroduring handling of the above exception, another exception Occurred:trac Eback (most recent): File "chainexceptionexample1.py", line <module> divide (1,0) file "Chainex ceptionexample1.py ", line ten, in divide log (exc) File" chainexceptionexample1.py ", line 7, in log print (exc, file=f ID) file "/opt/python3.0/lib/python3.0/io.py", line 1492, in write Self.buffer.write (b) file "/opt/python3.0/lib/pyth on3.0/io.py ", line 696, in write self._unsupported (" write ") File"/opt/python3.0/lib/python3.0/io.py ", line 322, in _u Nsupported (self.__class__.__name__, name)) Io. UnsupportedOperation:BufferedReader.write () not supported 

Please note that both exceptions are handled. In earlier versions of Python, it ZeroDivisionError will be lost and not processed. So how is this implemented? __context__property, for example ZeroDivisionError , is now part of all exception objects. In this example, the property that is thrown is IOError __context__ "still" __context__ in the property ZeroDivisionError .

__context__In addition to attributes, an exception object has an __cause__ attribute, which is typically initialized None . The purpose of this property is to record the cause of an exception in an explicit way. The __cause__ properties are set by the following syntax:

>>> raise EXCEPTION from Cause

It is the same as the following code:

>>>exception = exception>>>exception.__cause__ = Cause>>>raise exception

But more elegant. The example shown in Listing 8 shows an explicit exception chain.

Listing 8. An explicit exception chain in Python 3

Class Customerror (Exception):     passtry:    FID = open ("AFile.txt") # missing ' W ' again    print ("Blah blah blah", File=fid) except IOError as exc:    raise Customerror (' Something went wrong ') from exc

As shown in the previous example, the print() function throws an exception because the file has not been opened for writing. Listing 9 shows the corresponding trace.

Listing 9. Exception tracking

Traceback (most recent):  File ' chainexceptionexample2.py ', line 5, <module>    FID = open ("Afile. TXT ")  file"/opt/python3.0/lib/python3.0/io.py ", line 278, in __new__    return open (*args, **kwargs)  file"/ opt/python3.0/lib/python3.0/io.py ", line 222, on open    closefd) File"/opt/python3.0/lib/python3.0/io.py ", line 615 , in __init__    _fileio._fileio.__init__ (self, name, mode, CLOSEFD) IOError: [Errno 2] No such file or directory: ' Afile. TXT ' The above exception was the direct cause of the following exception:traceback (most recent call last): File "Chainexcep tionexample2.py ", line 8, in <modulei>  raise Customerror (' Something went wrong ') from exc__main__. Customerror:something went wrong

Notice that a line in the exception trace "The above exception was the direct cause of the following exception," followed by another pair that caused CustomError "something went Wron G "Exception tracking.

Another property added to the exception object is __traceback__ . If the caught exception does not have its __traceback__ properties, then the new trace is set. Here is a simple example:

From Traceback import format_tbtry:    1/0except Exception as exc:    print (FORMAT_TB (exc.__traceback__) [0])

Notice that format_tb a list is returned, and there is only one exception in this list.

With statement

Starting with Python version 2.6, with it has become a keyword, and this property no longer needs to be __future__ imported.

Back to top of page

Integer support and syntax

Python supports different binary integer string literals-eight decimal (most obvious!) ) and hexadecimal-and now it's added in binary. The representation of the octal number has changed: The octal number is now preceded by a prefix 0o or 0O (that is, the number 0 followed by an uppercase or lowercase letter o). For example, the octal 13 or decimal 11, respectively, is indicated as follows:

>>>0o1311

The new binary number is preceded by a prefix 0b or 0B (that is, the number 0 followed by an uppercase or lowercase letter b). The decimal number 21 in binary notation should be:

>>>0b01010121

oct()And the hex() method has been deleted.

Back to top of page

function comment

A function Comment Associates a representation with some part of the function (such as a parameter) at compile time. In itself, function annotations are meaningless-that is, they will not be processed unless the third-party library handles them. function annotations are intended to standardize the way a function parameter or return value is commented. The function comment syntax is:

def methodName (Param1:expression1, ..., paramn:expressionn)->expressionforreturntype: ...    

For example, as shown below is a comment on the parameters of a function:

def execute (Program: "Name of the program to be executed", error: "If something Goes Wrong"):    ...

The following example notes the return value of a function. This is useful for checking the return type of a function:

Def getName (), "isstring":     ...

The full syntax of the function comment can be found in PEP 3107 (related links, see resources).

Back to top of page

Conclusion Unexpected Harvest

In my opinion, one of the best modules added for Python 3 is antigravity . Start Python, and then type at the command line import antigravity . You must not be disappointed.

The final version of Python 3 was released in early December 2008. Since then, I've looked at some blogs to try to understand how people respond to backwards incompatibilities. Although I cannot assert that the community has reached an official consensus on some level, the views I have read about the blogs are polarized. The Linux® development community seems to be less fond of transitioning to version 3, because a lot of code needs to be ported. By contrast, many WEB developers welcome the transition because of improvements in Unicode support for the new version.

The point I want to make is that before you decide whether to migrate to a new version, you should read the relevant PEP and development mailing lists carefully. These PEP explanations explain the purpose of the changes, the benefits and their implementation. It is not difficult to see that these changes have been made in a deliberate and heated discussion. The topics shown in this series are intended to give ordinary Python programmers an immediate idea of these changes without having to read through all the PEP.

A preliminary discussion of Python 3, part 2nd: Advanced Topics