Python context manager contextlib and with statements

http://blog.csdn.net/pipisorry/article/details/50444736

with Statement

The WITH statement is an exception-handling feature that was introduced from Python 2.5 (which is available in version 2.5 via the From __future__ import with_statement), which is the default from the 2.6 version (see what ' s New in Python 2.6? Described in the related section of the with statement). The WITH statement is useful for accessing resources, ensuring that the necessary "cleanup" operations are performed regardless of whether an exception occurs during use, freeing up resources such as automatic shutdown after file use, automatic acquisition and release of locks in threads, and so On.

Terms

A set of concepts related to the context manager and with Statements.

context Management Protocol: Contains methods __enter__ () and __exit__ (), objects that support the protocol to implement both METHODS.

contextmanager: An object that supports the context management protocol, which implements the __enter__ () and __exit__ () methods. The context Manager defines the Run-time context to be established when the WITH statement is executed, and is responsible for performing entry and exit operations in the context of the WITH statement Block. The context manager is typically invoked with a with statement, or it can be used by calling its methods Directly.

Runtime Context: Created by the context manager, implemented by the context Manager's __enter__ () and __exit__ () methods, the __enter__ () method enters the runtime context before the statement body executes, __ exit__ () exits from the run-time context after the statement body has finished executing. The With statement supports the concept of a run-time Context.

Context Expression: the expression following the keyword with in the with statement, which returns a context manager Object.

Statement Body (with-body): A block of code wrapped with a statement that invokes the __enter__ () method of the context manager before executing the statement body and executes the __exit__ () method after the statement body is Executed.

Phi Blog

basic syntax and how it works

Syntax format for the WITH statement

    With context_expression [as target (s)]:        with-body

Here context_expression to return a context manager object that is not assigned to target (s) in the as clause, and assigns the return value of the context Manager's __enter__ () method to target (s) if an as clause is Specified. Target (s) can be a single variable, or a tuple enclosed by "()" (cannot be a list of variables separated by only "," and "()" must be added).

Python has improved some of the built-in objects, adding support for the context manager, which can be used in with statements such as the ability to automatically close files, automatically fetch and release the lock, and so On. Assuming you want to manipulate a file, you can use the WITH statement to have the following code:

manipulating file objects with the WITH statement

    With open (r ' Somefilename ') as somefile: for line in        somefile:            print line            # ... more code

The WITH statement is used here to ensure that the open file handle has been closed after execution of the with statement, regardless of whether an exception occurred during the processing of the File. If you use the traditional try/finally paradigm, use code similar to the Following:

Try/finally manipulating File objects

    somefile = Open (r ' somefilename ')    try: for line in        somefile:            print line            # ... more code    finally:< C13/>somefile.close ()

By comparison, using the WITH statement can reduce the amount of coding. There are also modules threading, decimal, etc. that have been added to the context management Protocol.

Python with statement opens two files at a time

With open (' fileToRead.txt ', ' r ') as reader, open (' fileToWrite.txt ', ' W ') as Writer:writer.write (reader.read ()) Note: 1. Open two files at the same time, one read a write, so you can save two with the nesting of statements.

2. python2.7 after the syntax, replacing the previous contextlib.nested syntax

PEP 0343 describes the implementation of the with Statement. The WITH statement executes in a code block similar to the following:

with Statement Execution Procedure

Context_manager = Context_expressionexit = Type (context_manager). __exit__  value = Type (context_manager). __enter_ _ (context_manager) exc = True   # True indicates normal execution, even if an exception is omitted; False indicates a re-throw exception, need to process the exception try:    try:        target = value  # If you use the AS clause        with-body     # execution With-body    except:        # An exception occurred during execution        exc = False        # If __exit__ returns True, the exception is ignored, and if False is returned, the exception will be re-thrown        by the outer code if not        exit (context_manager, *sys.exc_info ()):            raisefinally:    # Exit normally, or exit # via Break/continue/return statement in Statement-body    or ignore exception exit    if Exc: exit        (context_manager , none, none, none)         # Default return none,none is considered False in Boolean context

Note:

Executes the context_expression, generating the context manager Context_manager

Invokes the __enter__ () method of the context manager and assigns the return value of the __enter__ () method to the target (s) in the AS clause if an AS clause is used

Execute Statement Body With-body

Execution of the __exit__ () method of the context manager, regardless of whether an exception occurred during execution or not, the __exit__ () method is responsible for performing "cleanup" work, such as freeing Resources. If no exception occurs during execution, or if statement Break/continue/return is executed in the body of the statement, call __exit__ (none, none, none) as a parameter with none, or use SYS.EXC_ If an exception occurs during Execution. Info gets exception information for parameter call __exit__ (exc_type, exc_value, Exc_traceback)

When an exception occurs, if __exit__ (type, value, Traceback) returns False, the exception is re-thrown, allowing statement logic outside of with to handle the exception, which is also a common practice, and if True, ignores the exception and no longer handles the exception

Reading the implementation of a file with statement and its invocation procedure

class ReadFile(Object):    def__init__( self, filename):         self.file = Open(filename, ' R ')        def__enter__( self):        return self.file        def__exit__( self, type, value, Traceback):        # type, value, traceback for the wrong type, values, tracking stack, respectively         self.file.Close()        # returns True to indicate a non-throwing error, otherwise the error will be thrown by the WITH statement        return TrueWith ReadFile (' test.txt ') as File_read:
For line in File_read.readlines ():
Print (line)

At the time of the Call:

1. withThe statement first staged the ReadFile method of the class __exit__
2. Then call ReadFile the method of the class __enter__
3. __enter__method to open the file and return the result to the with statement
4. The result of the previous step is passed to the file_read parameter
5. withmanipulate parameters within a statement to file_read read each row
6. After the read is completed, with The statement calls the method that was staged before __exit__
7. __exit__Method closes the file

Note that within the method, __exit__ we closed the file but finally returned True , so the error is not thrown by the with Statement. Otherwise with , the statement throws a corresponding Error.

Phi Blog

Context Manager

The Contextlib module contains tools related to the context manager and with Declarations. Usually if you want to write a context manager, you need to define a class that contains the __enter__ method and the __exit__ method, for example:
Import time
Class Demo:
def __init__ (self, label):
Self.label = Label

def __enter__ (self):
Self.start = Time.time ()

def __exit__ (self, exc_ty, exc_val, exc_tb):
End = Time.time ()
Print (' {}: {} '. format (self.label, end-self.start))
A complete example
Import time

Class Demo:
def __init__ (self, label):
Self.label = Label

def __enter__ (self):
Self.start = Time.time ()

def __exit__ (self, exc_ty, exc_val, exc_tb):
End = Time.time ()
Print (' {}: {} '. format (self.label, end-self.start))

With demo (' counting '):
n = 10000000
While n > 0:
n-= 1

# counting:1.36000013351
The context manager is activated by a with declaration, which involves two methods.
1. __enter__ method, The __enter__ method executes when the execution stream enters the with code BLOCK. And it returns an object that can be used by the Context.
2. When the execution stream leaves the with code block, the __exit__ method is called, which cleans up the resource being Used.

Using the @contextmanager decorator to rewrite the timing example

[python Adorner decorators]
From Contextlib import ContextManager
Import time

@contextmanager
DEF demo (label):
Start = Time.time ()
Try
Yield
Finally
End = Time.time ()
Print (' {}: {} '. format (label, end-start))

With demo (' counting '):
n = 10000000
While n > 0:
n-= 1

# counting:1.32399988174
Looking at the example above, all the code before yield in the function is similar to the content of the __enter__ method in the context Manager. All code after yield is like the content of the __exit__ method. If an exception occurs during execution, it is triggered in the yield statement.

The developer can customize the classes that support the context management Protocol. The custom Context Manager __enter__ () and __exit__ () two methods required to implement a context management Protocol:

• context_manager.__enter__ () : Enters the context Manager's Run-time context and is called before the statement body Executes. The WITH statement assigns the return value of the method to target in the as clause, if the AS clause is specified

• context_manager.__exit__ (exc_type, exc_value, exc_traceback) : Exits the run-time context associated with the context manager, and returns a Boolean value indicating whether the exception that occurred is being processed. The parameter represents the exception that caused the exit operation, and if no exception occurred when exiting, the 3 parameters were None. If an exception occurs, the return

  True means that the exception is not handled, or the exception is re-thrown after exiting the method to be handled by code logic other than the with Statement. If an exception is generated inside the method, the exception generated by the statement in Statement-body is Replaced. To handle an exception, do not display a re-throw exception, that is, you cannot re-throw the exception passed in by argument, just set the return value to False. After that, the context management code detects whether __exit__ () fails to handle the exception

Here's A simple example that shows how to build a custom context Manager. Note that the context manager must provide both the definition of the __enter__ () and the __exit__ () method, and either the missing one will cause the Attributeerror;with statement to first check whether the __exit__ () method is provided, and then check if __ is defined enter__ () Method.

Suppose there is a resource dummyresource, which needs to be allocated before access, then released after use, and the allocation operation can be put into the __enter__ () method, and the release action can be placed in the __exit__ () method. For simplicity, The current operation is only indicated by a print statement, and there is no actual resource allocation and Deallocation.

customizing objects that support with statements

Class Dummyresource:    def __init__ (self, tag):        self.tag = tag        print ' Resource [%s] '% tag    def __enter__ ( self): "        print ' [Enter%s]: Allocate resource. '% self.tag        return self  # can return different objects    def __exit__ (self, exc_ type, exc_value, exc_tb):        print ' [exit%s]: free resource. '% self.tag        if EXC_TB is None:            print ' [exit%s]: E xited without Exception. '% self.tag        else:            print ' [Exit%s]: Exited with exception raised. '% Self.tag            Retu RN False   # can be omitted, the default none is also considered False

__enter__ () in Dummyresource returns a reference to itself, which can be assigned to the target variable in the as clause, and the type of the return value can be set to a different type depending on the actual need, not necessarily the context manager object Itself.

The variable EXC_TB is detected in the __exit__ () method, if it is not none, indicating that an exception has occurred and that returning false indicates that the exception needs to be handled by external code logic; notice that if no exception occurs, the default return value is None and is considered False in a Boolean environment. , but because no exception occurred, __exit__ () Three parameters are none, the context management code can detect this situation, do normal processing.

Accessing Dummyresource in the WITH statement

Using custom objects that support the WITH statement

With Dummyresource (' Normal '):    print ' [with-body] Run without exceptions. ' With Dummyresource (' with-exception '): "    print ' [with-body] Run with Exception. '    Raise Exception    print ' [with-body] Run with Exception. Failed to finish statement-body! '

Execution result of the 1th with Statement:

Resource [normal][enter Normal]: Allocate Resource. [with-body] Run without Exceptions. [Exit Normal]: Free Resource. [Exit Normal]: Exited without Exception.

As you can see, normal execution executes the statement body With-body first, and then executes the __exit__ () method to free the Resource.

Execution result of the 2nd with Statement:

Resource [with-exception][enter with-exception]: Allocate Resource. [with-body] Run with Exception. [Exit with-exception]: Free Resource. [Exit with-exception]: Exited with Exception Raised. Traceback (most recent):  File ' G:/demo ', line <module>    raise Exception Exception  

As you can see, With-body does not complete when an exception occurs in with-body, but the resource is guaranteed to be freed, and the resulting exception is caught by code logic outside the with Statement.

You can customize the context Manager to manage resources in a software system, such as database connections, access control for shared resources, and so On. Python online Documentation Writing Context Managers provides a simple example of a context manager for managing database Connections.

context using instances in database operations

Context and Control
previously, we have seen the use of exceptions to handle the flow of Control. typically, The basic steps are as follows:
Try to get Resources (files, network connections, etc.)
If it fails, clear everything left behind.
Successful access to resources is done accordingly
Write log
End of program
With that in mind, let's take a look at the example of the previous chapter of the Database. We use try-except-finally to ensure that any transactions we start are either submitted or rolled Back.
Try
# attempt to acquire a resource
Db.commit ()
Except Exception:
# If It fails, anything left behind
Log.warn ("Failure committing transaction, rolling Back")
Db.rollback ()
Else
# If It works, perform actions
# in this case, we just log success
Log.info ("Saved the new FOO")
Finally
# Clean up
Db.close ()
# program Complete
Our previous example maps almost exactly to the steps just Mentioned. Does this logic change a lot? Not Much.
Almost every time we store data, we'll do the same thing. We can write these logic into a method, or we can use the context Manager
db = Db_library.connect ("fakesql://")
# as a function
Commit_or_rollback (db)
# Context Manager
With transaction ("fakesql://") as Db:
# Retrieve data here
# Modify Data here
The context Manager protects the code snippet by setting the resource (context) that is required to run the code snippet. In our case, we need to deal with a database transaction, so the process will be:
Connecting to a database
Start at the beginning of the code snippet
Commit or rollback at the end of a code snippet
Clean up resources at the end of a code snippet
Let's create a context manager that uses the context Manager to work with the settings for our hidden Database. The ContextManager interface is very simple. The context Manager object needs to have a __enter__ () method to set the desired context, and a __exit__ (exc_type, exc_val, Exc_tb) method is called after it leaves the code snippet. If there is no exception, then all three exc_* parameters will be none.
The __enter__ method here is very simple, let's start with this Function.
Python
Class Databasetransaction (object):
def __init__ (self, connection_info):
Self.conn = Db_library.connect (connection_info)
def __enter__ (self):
Return Self.conn
The __enter__ method simply returns the database connection, in which we use the database connection to access the Data. The database connection is actually established in the __init__ method, so if the database fails to establish a connection, the code snippet will not Execute.
Now let's define how the transaction will be done in the __exit__ method. There's a lot more work to do here, because all the exceptions in the code snippet are handled, and the transaction is Closed.
Python
def __exit__ (self, exc_type, exc_val, exc_tb):
If Exc_type is not None:
Self.conn.rollback ()
Try
Self.conn.commit ()
Except Exception:
Self.conn.rollback ()
Finally
Self.conn.close ()
Now we can use the Databasetransaction class as the context manager in our Example. Inside the class, the __enter__ and __exit__ methods will start and set up the data connection and handle the aftermath Work.
Python
# Context Manager
With Databasetransaction ("fakesql://") as Db:
# Retrieve data here
# Modify Data here
To improve our (simple) transaction manager, we can add a variety of exception handling. Even now, this transaction manager has hidden a lot of complex processing for us, so you don't have to worry about database-related details every time you pull data from a database.

[python exception handling Exception:else with try except error control]

from:http://blog.csdn.net/pipisorry/article/details/50444736

Ref: talking about Python with statements

Python:try finally with introduction

Context Manager in Python

Python context manager contextlib and with statements