Describes the usage of the with statement in Python and the pythonwith statement.

Describes the usage of the with statement in Python and the pythonwith statement.

Introduction

The with statement is a feature related to Exception Handling introduced from Python 2.5 (in Python 2.5, it can be used only after being imported from _ ure _ import with_statement ), available by default from version 2.6 (refer to What's new in Python 2.6? ). The with statement is applicable to resource access. Make sure that the necessary "clean" operation is performed to release resources no matter whether exceptions occur during use, for example, the file is automatically closed after use, and the lock in the thread is automatically obtained and released.

Terms

To use the with statement, you must first understand the context manager concept. With the context manager, with statements can work.

The following are a set of concepts related to the context manager and with statements.

Context Management Protocol: includes methods _ enter _ () and _ exit _ ().

The object of the Protocol must implement these two methods.

Context Manager: an object that supports the Context management protocol.

_ Enter _ () and _ exit _ () methods. The context manager defines the runtime context to be established when the with statement is executed,

Executes the entry and exit operations in the with statement block context. Generally, the with statement is used to call the context manager,

You can also call the method directly.

Runtime context: created by the context manager

_ Exit _ () method implementation, __enter _ () method enters the runtime context before the statement body is executed, __exit _ () in

After the statement body is executed, it exits from the runtime context. The with Statement supports the context concept during running.

Context Expression: the Expression followed by the keyword with in the with statement.

Returns a context manager object.

With-body: the block of code enclosed by the with statement. Before executing the statement body, the context manager is called.

The _ enter _ () method of the handler. After the statement body is executed, the _ exit _ () method is executed.

Basic syntax and working principles

The syntax format of the with statement is as follows:
Listing 1. Syntax format of the with statement

with context_expression [as target(s)]:
    with-body

Context_expression returns a context manager object, which is not assigned to the target (s) in the as clause. If the as clause is specified, the Return Value of the _ enter _ () method of the context manager is assigned to target (s ). Target (s) can be a single variable, or a tuple enclosed by "()" (not a list of variables separated by ",", but must be added with "()").

Python has improved some built-in objects and added support for the context manager, which can be used in with statements, such as automatic file closure and Automatic Acquisition and release of thread locks. If you want to operate a file, you can use the with statement with the following code:
Listing 2. Using the with statement to operate a file object

 with open(r'somefileName') as somefile:
    for line in somefile:
      print line
      # ...more code

The with statement is used here. whether an exception occurs during file processing or not, you can ensure that the opened file handle is closed after the with statement is executed. If the traditional try/finally paradigm is used, the following code should be used:
Listing 3. try/finally object operations

 somefile = open(r'somefileName')
  try:
    for line in somefile:
      print line
      # ...more code
  finally:
    somefile.close()

In comparison, the with statement can reduce the encoding amount. Modules such as threading and decimal have been added to the context management protocol.

PEP 0343 describes the implementation of the with statement. The execution process of the with statement is similar to the following code block:
Listing 4. with statement execution process

context_manager = context_expression
   exit = type (context_manager) .__ exit__
   value = type (context_manager) .__ enter __ (context_manager)
   exc = True # True indicates normal execution and is ignored even if there is an exception; False indicates that the exception is re-thrown and needs to be handled
   try:
     try:
       target = value # if the as clause is used
       with-body # execute with-body
     except:
       # An exception occurred during execution
       exc = False
       # If __exit__ returns True, the exception is ignored; if False is returned, the exception is re-thrown
       # Exception handling by outer code
       if not exit (context_manager, * sys.exc_info ()):
         raise
   finally:
     # Exit normally, or exit with a break / continue / return statement in statement-body
     # Or ignore abnormal exit
     if exc:
       exit (context_manager, None, None, None)
     # Returns None by default, None is considered False in a boolean context

Run context_expression to generate the context manager context_manager.
Call the _ enter _ () method of the context manager. If the as clause is used, assign the return value of the _ enter _ () method to target (s) in the as clause)
Execution statement body with-body
Whether or not exceptions occur during execution, execute the _ exit _ () method of the context manager, __exit _ () method to perform "cleaning, such as releasing resources. If no exception occurs during execution or the statement break/continue/return is executed in the statement body, call _ exit _ (None, None, None) with None as the parameter ); if an exception occurs during execution, sys. the exception information obtained by exc_info is the parameter call _ exit _ (exc_type, exc_value, exc_traceback)
When an exception occurs, if _ exit _ (type, value, traceback) returns False, an exception is thrown again to allow the statement logic outside of with to handle the exception, this is also a common practice; if True is returned, the exception is ignored and no longer processed.

Custom context Manager

Developers can customize classes that support context management protocols. The _ enter _ () and _ exit _ () methods required by the custom context manager to implement the context management protocol are as follows:

Context_manager. _ enter _ (): enter the context of the context manager runtime, which is called before the statement body is executed. The with statement assigns the return value of this method to the target in the as clause. If the as clause is specified
Context_manager. _ exit _ (exc_type, exc_value, exc_traceback): exits the runtime context related to the context manager, and returns a Boolean value indicating whether to handle exceptions. The parameter indicates an exception that causes the exit operation. If no exception occurs during exit, all three parameters are None. If an exception occurs, return

True indicates that no exception is processed. Otherwise, an exception is thrown again after exiting the method to be processed by the Code logic outside the with statement. If an internal exception occurs in this method, the exception generated by the statement in statement-body is replaced. To handle exceptions, do not throw the exception again, that is, do not throw the exception passed in through the parameter again. You only need to set the return value to False. Then, the context management code checks whether _ exit _ () fails to handle the exception.

The following uses a simple example to demonstrate how to build a custom context manager. Note: The context manager must provide definitions of the _ enter _ () and _ exit _ () methods at the same time. Missing one will cause AttributeError; the with statement first checks whether the _ exit _ () method is provided, and then checks whether the _ enter _ () method is defined.

Assume that there is a resource DummyResource, which needs to be allocated before access and then released after use; the allocation operation can be placed in the _ enter _ () method, the release operation can be put in the _ exit _ () method. For the sake of simplicity, only print the statement to indicate the current operation, and there is no actual resource allocation or release.
Listing 5. Custom objects that support the with statement

class DummyResource:
   def __init __ (self, tag):
       self.tag = tag
       print 'Resource [% s]'% tag
     def __enter __ (self):
       print '[Enter% s]: Allocate resource.'% self.tag
       return self # can return different objects
     def __exit __ (self, exc_type, exc_value, exc_tb):
       print '[Exit% s]: Free resource.'% self.tag
       if exc_tb is None:
         print '[Exit% s]: Exited without exception.'% self.tag
       else:
         print '[Exit% s]: Exited with exception raised.'% self.tag
         return False # can be omitted, the default None is also regarded as False

In DummyResource, _ enter _ () returns its own reference. This reference can be assigned to the target variable in the as clause. The type of the returned value can be set to different types as needed, it does not need to be the context manager object itself.

The _ exit _ () method checks the variable exc_tb. If it is not set to None, an exception occurs. If it is set to False, an exception must be handled by the external code logic; note that if no exception occurs, the default return value is None, which is also regarded as False in a Boolean environment. However, because no exception occurs, the three parameters of __exit _ () are None, the context management code can detect this situation and perform normal processing.

The following uses the with statement to access DummyResource:
Listing 6. Using custom objects that support the with statement

 with DummyResource('Normal'):
    print '[with-body] Run without exceptions.'

  with DummyResource('With-Exception'):
    print '[with-body] Run with exception.'
    raise Exception
    print '[with-body] Run with exception. Failed to finish statement-body!'


 

	
  


 

The execution results of the 1st with statements are as follows:
Listing 7. with Statement 1 execution result

Resource [Normal]
[Enter Normal]: Allocate resource.
[With-body] Run without exceptions.
[Exit Normal]: Free resource.
[Exit Normal]: Exited without exception.

You can see that when the statement body with-body is executed normally, the _ exit _ () method is executed to release the resource.

The execution results of the 2nd with statements are as follows:
Listing 8. with Statement 2 execution results

 Resource [With-Exception]
  [Enter With-Exception]: Allocate resource.
  [with-body] Run with exception.
  [Exit With-Exception]: Free resource.
  [Exit With-Exception]: Exited with exception raised.

  Traceback (most recent call last):
   File "G:/demo", line 20, in <module>
    raise Exception
  Exception

As you can see, when an exception occurs in the with-body, the with-body is not executed completely, but the resource will be released, exceptions generated at the same time are captured and handled by code logic other than the with statement.

You can customize the context manager to manage resources in the software system, such as database connections and resource sharing access control. The Python online document Writing Context Managers provides a simple example of a Context manager for managing database connections.

Contextlib Module

The contextlib module provides three objects: contextmanager, nested, and closing. These objects can be used to encapsulate existing generator functions or objects and support the context management protocol, avoiding the need to write the context manager to support with statements.
Contextmanager

Contextmanager is used to decorate the generator function. After the generator function is decorated, a context manager is returned, its _ enter _ () and _ exit __() contextmanager is responsible for providing the method, instead of the previous iteration. The decorated generator function can generate only one value. Otherwise, an exception RuntimeError occurs. The generated value is assigned to the target in the as clause if the as clause is used. The following is a simple example.
Listing 9. contextmanager example

from contextlib import contextmanager

  @contextmanager
  def demo():
    print '[Allocate resources]'
    print 'Code before yield-statement executes in __enter__'
    yield '*** contextmanager demo ***'
    print 'Code after yield-statement executes in __exit__'
    print '[Free resources]'

  with demo() as value:
    print 'Assigned Value: %s' % value

The output is as follows:
Listing 10. contextmanager sample execution result

 [Allocate resources]
  Code before yield-statement executes in __enter__
  Assigned Value: *** contextmanager demo ***
  Code after yield-statement executes in __exit__
  [Free resources]

The yield statement in the generator function is executed in the _ enter _ () method, and the yield statement is executed in _ exit, the value generated by yield is assigned to the value variable in the as clause.

Note that contextmanager only ignores the compilation of _ enter _ ()/_ exit, however, it is not responsible for "obtaining" and "cleaning" resources. The "getting" operation must be defined before the yield statement, and after the "cleaning" operation needs to define the yield statement, in this way, the with statement will execute the _ enter _ ()/_ exit _ () method to obtain/release resources, that is, the generator function must implement necessary logical control, including throwing an appropriate exception when an error occurs during resource access.
Function nested

Nested can organize multiple context managers together to avoid the use of nested with statements.
Listing 11. nested syntax

with nested(A(), B(), C()) as (X, Y, Z):
     # with-body code here

Similar:
Listing 12. nested Execution Process

 with A() as X:
    with B() as Y:
      with C() as Z:
         # with-body code here

Note that, if the _ exit _ () method of a context manager returns False for exception handling, the context manager on the outer layer will not detect the exception.
Context manager closing

Closing is implemented as follows:
Listing 13. Context management closing implementation

 

class closing(object):
    # help doc here
    def __init__(self, thing):
      self.thing = thing
    def __enter__(self):
      return self.thing
    def __exit__(self, *exc_info):
      self.thing.close()

The context Manager assigns the wrapped object to the target variable of the as clause, and ensures that the opened object is closed after the with-body execution. The objects encapsulated by the closing context manager must be defined by the close () method. Otherwise, an AttributeError error is reported during execution.
Listing 14. Custom closing-supported objects

class ClosingDemo(object):
    def __init__(self):
      self.acquire()
    def acquire(self):
      print 'Acquire resources.'
    def free(self):
      print 'Clean up any resources acquired.'
    def close(self):
      self.free()

  with closing(ClosingDemo()):
    print 'Using resources'

The output is as follows:
Listing 15. output results of custom closing objects

 Acquire resources.
  Using resources
  Clean up any resources acquired.

Closing is applicable to objects that provide close () implementation, such as network connections and database connections. You can also use the close () interface when customizing a class () to clear the required resources.