On the WITH statement of Python

Add by ZHJ: The context Manager is a re-encapsulation of the try-except-finally, only to optimize the code this level of feature

Original: http://www.ibm.com/developerworks/cn/opensource/os-cn-pythonwith/#

Introduction

The WITH statement is an exception-handling feature that was introduced from Python 2.5 (which is available in version 2.5 via the From __future__ import With_statement), which is the default from the 2.6 version (see what ' s New in Python 2.6? Described in the related section of the WITH statement). The WITH statement is useful for accessing resources, ensuring that the necessary "cleanup" operations are performed regardless of whether an exception occurs during use, freeing up resources such as automatic shutdown after file use, automatic acquisition and release of locks in threads, and so on.

Back to top of page

Terms

To use the WITH statement, you first understand the concept of the context manager. With the context manager, the WITH statement can work.

The following is a set of concepts related to the context manager and with statements.

context Management Protocol: Contains methods __enter__ () and __exit__ (), supporting

The object of the Protocol is to implement both methods.

ContextManager: An object that supports the context management protocol, which implements the

__enter__ () and __exit__ () methods. The context manager defines the run-time context to be established when the WITH statement is executed.

Executes the Enter and exit operations in the context of the WITH statement block. The context manager is typically invoked with the WITH statement.

It can also be used by calling its methods directly.

Runtime Context: Created by the context manager, through the context Manager's __enter__ () and

The __exit__ () method is implemented, and the __enter__ () method enters the run-time context before the statement body executes, __exit__ ()

Exits from the run-time context after the statement body has finished executing. The With statement supports the concept of a run-time context.

Context Expression: An expression that follows the keyword with in a with statement, which

A context Manager object to return.

Statement Body (with-body): A block of code wrapped with a statement that invokes the context tube before executing the statement body

The __enter__ () method of the manager executes the __exit__ () method after the statement body is executed.

Back to top of page

Basic syntax and how it works

The syntax format for the WITH statement is as follows:

Listing 1. Syntax format for the WITH statement

    With Context_expression [as Target (s)]:        with-body

Here context_expression to return a context manager object that is not assigned to target (s) in the AS clause, and assigns the return value of the context Manager's __enter__ () method to target (s) if an AS clause is specified. Target (s) can be a single variable, or a tuple enclosed by "()" (cannot be a list of variables separated by only "," and "()" must be added).

Python has improved some of the built-in objects, adding support for the context manager, which can be used in with statements such as the ability to automatically close files, automatically fetch and release the lock, and so on. Assuming you want to manipulate a file, you can use the WITH statement to have the following code:

Listing 2. Manipulating file objects with the WITH statement

    With open (R ' Somefilename ') as Somefile: For line in        somefile:            print line            # ... more code

The WITH statement is used here to ensure that the open file handle has been closed after execution of the WITH statement, regardless of whether an exception occurred during the processing of the file. If you use the traditional try/finally paradigm, use code similar to the following:

Listing 3. Try/finally manipulating File objects

    Somefile = open (R ' somefilename ')    try: For line in        somefile:            print line            # ... more code    finally:        Somefile.close ()

By comparison, using the WITH statement can reduce the amount of coding. There are also modules threading, decimal, etc. that have been added to the context management protocol.

PEP 0343 describes the implementation of the WITH statement. The WITH statement executes in a code block similar to the following:

Listing 4. With statement execution procedure

    Context_manager = context_expression    exit = Type (Context_manager). __exit__      value = Type (Context_manager). __ enter__ (context_manager)    exc = True   # True indicates normal execution, even if an exception is omitted; False indicates a re-throw exception, need to process the exception    try:        try            : target = value  # If the AS clause            is used with-body     # executes with-body        except:            # An exception occurred during execution            exc = False            # if __exit__ returns TRUE, the exception is ignored, and if False is returned, the exception will be re-thrown            by the outer code if not            exit (Context_manager, *sys.exc_info ()):                Raise    finally:        # Normal Exit, or exit # via Break/continue/return statement in Statement-body        or ignore exception exit        if exc:< C22/>exit (Context_manager, none, none, none)         # Default return none,none is considered False in Boolean context

1. Executes the context_expression, generating the context manager Context_manager
2. Invokes the __enter__ () method of the context manager and assigns the return value of the __enter__ () method to the target (s) in the AS clause if an AS clause is used
3. Execute Statement Body With-body
4. Execution of the __exit__ () method of the context manager, regardless of whether an exception occurred during execution or not, the __exit__ () method is responsible for performing "cleanup" work, such as freeing resources. If no exception occurs during execution, or if statement Break/continue/return is executed in the body of the statement, call __exit__ (none, none, none) as a parameter with none, or use SYS.EXC_ if an exception occurs during execution. Info gets exception information for parameter call __exit__ (Exc_type, Exc_value, Exc_traceback)
5. When an exception occurs, if __exit__ (type, value, Traceback) returns FALSE, the exception is re-thrown, allowing statement logic outside of with to handle the exception, which is also a common practice, and if True, ignores the exception and no longer handles the exception

Back to top of page

Custom Context Manager

The developer can customize the classes that support the context management protocol. The custom Context Manager __enter__ () and __exit__ () two methods required to implement a context management protocol:

• context_manager.__enter__ () : Enters the context manager's run-time context and is called before the statement body executes. The WITH statement assigns the return value of the method to target in the AS clause, if the AS clause is specified
• context_manager.__exit__ (Exc_type, Exc_value, Exc_traceback) : Exits the run-time context associated with the context manager, and returns a Boolean value indicating whether the exception that occurred is being processed. The parameter represents the exception that caused the exit operation, and if no exception occurred when exiting, the 3 parameters were None. If an exception occurs, the return

  True means that the exception is not handled, or the exception is re-thrown after exiting the method to be handled by code logic other than the WITH statement. If an exception is generated inside the method, the exception generated by the statement in Statement-body is replaced. To handle an exception, do not display a re-throw exception, that is, you cannot re-throw the exception passed in by argument, just set the return value to False. After that, the context management code detects whether __exit__ () fails to handle the exception

Here's a simple example that shows how to build a custom context manager. Note that the context manager must provide both the definition of the __enter__ () and the __exit__ () method, and either the missing one will cause the Attributeerror;with statement to first check whether the __exit__ () method is provided, and then check if __ is defined Enter__ () method.

Suppose there is a resource dummyresource, which needs to be allocated before access, then released after use, and the allocation operation can be put into the __enter__ () method, and the release action can be placed in the __exit__ () method. For simplicity, the current operation is only indicated by a print statement, and there is no actual resource allocation and deallocation.

Listing 5. Customizing objects that support with statements

    Class Dummyresource:    def __init__ (self, Tag):            Self.tag = tag            print ' Resource [%s] '% tag        def __enter__ ( Self): "            print ' [Enter%s]: Allocate resource. '% Self.tag            return self  # can return different objects        def __exit__ (self, exc_ Type, Exc_value, EXC_TB):            print ' [Exit%s]: free resource. '% Self.tag            if EXC_TB is None:                print ' [Exit%s]: Exited without exception. '% Self.tag            else:                print ' [Exit%s]: Exited with exception raised. '% Self.tag                ret Urn False   # can be omitted, default none is also considered False

__enter__ () in Dummyresource returns a reference to itself, which can be assigned to the target variable in the AS clause, and the type of the return value can be set to a different type depending on the actual need, not necessarily the context Manager object itself.

The variable EXC_TB is detected in the __exit__ () method, if it is not none, indicating that an exception has occurred and that returning false indicates that the exception needs to be handled by external code logic; Notice that if no exception occurs, the default return value is None and is considered False in a Boolean environment. , but because no exception occurred, __exit__ () Three parameters are none, the context management code can detect this situation, do normal processing.

The following accesses the dummyresource in the WITH statement:

Listing 6. Using custom objects that support the WITH statement

    With Dummyresource (' Normal '):        print ' [with-body] Run without exceptions. '    With Dummyresource (' with-exception '): "        print ' [With-body] Run with Exception. '        Raise Exception        print ' [With-body] Run with Exception. Failed to finish statement-body! '

The execution result of the 1th with statement is as follows:

Listing 7. With statement 1 execution results

    Resource [Normal]    [Enter Normal]: Allocate Resource.    [With-body] Run without exceptions.    [Exit Normal]: free resource.    [Exit Normal]: Exited without exception.

As you can see, normal execution executes the statement body With-body first, and then executes the __exit__ () method to free the resource.

The execution result of the 2nd with statement is as follows:

Listing 8. With statement 2 execution results

    Resource [with-exception]    [Enter with-exception]: Allocate Resource.    [With-body] Run with exception.    [Exit with-exception]: free resource.    [Exit with-exception]: Exited with Exception raised.    Traceback (most recent):      File ' G:/demo ', line <module>       raise Exception Exception    

As you can see, With-body does not complete when an exception occurs in With-body, but the resource is guaranteed to be freed, and the resulting exception is caught by code logic outside the WITH statement.

You can customize the context Manager to manage resources in a software system, such as database connections, access control for shared resources, and so on. Python online Documentation Writing Context Managers provides a simple example of a context manager for managing database connections.

Back to top of page

Contextlib Module

The Contextlib module provides 3 objects: Adorner ContextManager, function nested, and context manager closing. Using these objects, you can wrap existing generator functions or objects, add support for context management protocols, and avoid writing context managers specifically to support with statements.

Adorner ContextManager

ContextManager is used to decorate the generator function, and after the generator function is decorated, a context manager is returned, and its __enter__ () and __exit__ () methods are provided by the ContextManager and are not the previous iterations. A decorated generator function can produce only one value, otherwise it causes an exception to be runtimeerror; The resulting value is assigned to the target in the AS clause if the AS clause is used. Let's look at a simple example.

Listing 9. Decorator ContextManager Use Example

    From contextlib import ContextManager    @contextmanager    def demo ():        print ' [Allocate resources] '        print ' Code before yield-statement executes in __enter__ '        yield ' * * * ContextManager Demo * * * '        print ' code after yield-st Atement executes in __exit__ '        print ' [free resources] ' with    demo () as Value:        print ' Assigned value:%s '% Valu E

The resulting output is as follows:

Listing 10. ContextManager using the sample execution results

    [Allocate Resources]    Code before yield-statement executes in __enter__    Assigned Value: * * * ContextManager Demo * * *    Code after yield-s Tatement executes in __exit__    [free resources]

As you can see, the statements before yield in the generator function are executed in the __enter__ () method, the statements after yield are executed in __exit__ (), and the value produced by yield is assigned to the values variable in the AS clause.

It is important to note that ContextManager only omits the writing of __enter__ ()/__exit__ (), but is not responsible for the "get" and "clean" work of resources; "GET" operations need to be defined before the yield statement, the "cleanup" operation needs to define yield Statements so that the WITH statement executes the __enter__ ()/__exit__ () method when executing these statements to get/free resources, that is, the generator function needs to implement the necessary logical controls, including throwing the appropriate exception when there is an error in resource access.

function nested

Nested can organize multiple context managers together, avoiding the use of nested with statements.

Listing 11. Nested syntax

    With nested (A (), B (), C ()) as (X, Y, Z):         # with-body code here

Similar to:

Listing 12. Nested execution process

    With A () as X: With        B () as Y: With            C () as Z:                 # with-body code here

It should be noted that after an exception occurs, if the __exit__ () method of a context manager returns FALSE for exception handling, the outer context manager does not monitor the exception.

Context Manager closing

The implementation of closing is as follows:

Listing 13. Context Management Closing implementation

    Class closing (object):        # Help Doc here        def __init__ (self, Thing):            self.thing = Thing        def __enter__ (self ):            return self.thing        def __exit__ (self, *exc_info):            self.thing.close ()

The context manager assigns the wrapped object to the target variable of the AS clause, while ensuring that the open object is closed when the with-body is finished executing. The object that is wrapped by the closing context manager must provide the definition of the close () method, or the Attributeerror error will be reported when executed.

Listing 14. Customizing objects that support closing

    Class Closingdemo (object):        def __init__ (self):            Self.acquire ()        def acquire (self):            print ' Acquire Resources. '        def free:            print ' no resources acquired. '        def close (self):            Self.free () with    closing (Closingdemo ()):        print ' Using resources '

The resulting output is as follows:

Listing 15. Customizing the output of the closing object

    Acquire resources.    Using Resources Clean up any    resources acquired.

The closing is intended for objects that provide close () implementations, such as network connections, database connections, and so on, and can perform the required resource "cleanup" by using the interface Close () when customizing the class.

Back to top of page

Summary

This paper introduces the syntax and working mechanism of the With statement, introduces how to implement the custom context Manager, and finally introduces how to use the Contextlib module to simplify the writing of the context manager.

Reference Learning

• What's the Python 2.6 version of what's new in Python 2.6? The WITH statement is described.
• See <<learning Python, Third edition>> Introduction to exception handling, with statements, and generators and adorners.
• See <<core Python Programming, Second edition>> Introduction to exception handling, with statements, and generators and adorners.
• The Python online document context Manager Types and with Statement context Managers describes the contextual manager and the WITH statement.
• Contextlib-utilities for with-statement contexts Contextlib, contextlib implementation details can be see under the Python source package lib/contextlib.py File.
• PEP 0343 describes the resulting background, specification, and use of the WITH statement.

Talking about Python with statement (GO)