Context manager in Python

In Python, call the _ enter _ method before entering the code block and call the _ exit _ method object after leaving the code block as the context manager, in this article, we will go deep into the context manager in Python to see the role and usage of the context manager: in Python, call the _ enter _ method before entering the code block and call the _ exit _ method object after leaving the code block as the context manager, in this article, we will go deep into the context manager in Python to see the role and usage of the context manager:

1. what is the context manager?

For example, when writing Python code, you often put a series of operations in a statement block:

(1) when a condition is true-execute this statement Block

(2) when a condition is true, execute the statement block cyclically.

Sometimes we need to maintain a certain state when the program runs in the statement block and end this state after leaving the statement block.

Therefore, in fact, the context manager task is to prepare the code block before it is executed, and clean up the code block after it is executed.

Context manager is a feature added in Python2.5, which makes your code more readable and has fewer errors. Next, let's take a look at how to use it.

2. how to use the context manager?

Code is the best way to learn. let's see how we open a file and write "Hello World "?

filename = 'my_file.txt'mode = 'w' # Mode that allows to write to the filewriter = open(filename, mode)writer.write('Hello ')writer.write('World')writer.close()

Line 1-2: specify the file name and open mode (write ).

Line 3: open the file, write "Hello world" in lines 4-5, and close the file in line 3.

That's not enough. why do we still need the context manager? However, we ignore a small but important detail: what if we don't have a chance to close the file at line 3?

For example, the disk is full. Therefore, an exception is thrown when we try to write data to the file in row 4th, and row 6th has no chance of execution.

Of course, we can use try-finally statement block for packaging:

writer = open(filename, mode)try:  writer.write('Hello ')  writer.write('World')finally:  writer.close()

The code in the finally statement block is executed no matter what happens in the try statement block. Therefore, the file will be closed. Is there any problem with this? Of course not, but when we do something more complex than writing "Hello world", try-finally statements will become ugly. For example, if we want to open two files, read one write, and copy between the two files, the with statement can ensure that both files can be closed at the same time.

OK. Let's break down the matter:

(1) First, create a file variable named "writer.

(2) then perform some operations on writer.

(3) close writer.

Is it more elegant?

with open(filename, mode) as writer:  writer.write('Hello ')   writer.write('World')

Let us go deeper. "with" is a new keyword and always appears along with the context manager. "Open (filename, mode)" has appeared in the previous code. "As" is another keyword that refers to the content returned from the "open" function and assigned it to a new variable. "Writer" is a new variable name.

Line 2-3: enable a new code block. In this code block, we can perform any operation on writer. In this way, we use the "open" context manager, which ensures that our code is both elegant and safe. It completes the try-finally task.

Open functions can be used as simple functions and context managers. This is because the open function returns a file type variable, which implements the write method we used previously, however, some special methods must be implemented as the context manager. I will introduce them in the following section.

3. Custom context manager

Let's write an "open" context manager.

To implement the context manager, you must implement two methods: one is responsible for the preparation of the block into the statement, and the other is responsible for the aftermath of the block. At the same time, we need two parameters: file name and open mode.

The Python class contains two special methods: __enter _ and _ exit _ (double-underline as the prefix and suffix ).

When an object is used as the context manager:

(1) The _ enter _ method will be called before entering the code block.

(2) The _ exit _ method is called after the code block is left (even if an exception occurs in the code block ).

The following is an example of the context manager, which is printed when the code block is entered and left.

class PypixContextManagerDemo:   def __enter__(self):    print 'Entering the block'   def __exit__(self, *unused):    print 'Exiting the block' with PypixContextManagerDemo():  print 'In the block' #Output:#Entering the block#In the block#Exiting the block

Note:

(1) no parameters are passed.
(2) The "as" keyword is not used here.
We will discuss the parameter settings for the _ exit _ method later.
How do we pass parameters to a class? In fact, the _ init _ method can be used in any class. here we will rewrite it to receive two necessary parameters (filename, mode ).

When we enter the statement block, the open function will be used, as in the first example. When we leave the statement block, all things opened in the _ enter _ function will be closed.

The following is our code:

class PypixOpen:   def __init__(self, filename, mode):    self.filename = filename    self.mode = mode   def __enter__(self):    self.openedFile = open(self.filename, self.mode)    return self.openedFile   def __exit__(self, *unused):    self.openedFile.close() with PypixOpen(filename, mode) as writer:  writer.write("Hello World from our new Context Manager!")

Let's take a look at the changes:

(1) lines 3-5 receive two parameters through _ init.

(2) lines 7-9. open the file and return it.

(3) line 12: close the file when the statement block is left.

(4) lines 14-15 simulate open using our own context manager.

In addition, there are some things to emphasize:

4. how to handle exceptions

We completely ignore the problems that may occur inside the statement block.

If an exception occurs inside the statement block, the __exit _ method will be called, and the exception will be thrown again (re-raised ). When processing file write operations, you certainly do not want to hide these exceptions most of the time, so this is acceptable. For exceptions that do not want to be thrown again, we can make the _ exit _ method simply return True to ignore all exceptions in the statement block (this is not wise in most cases ).

We can learn more details when an exception occurs. The Complete _ exit _ function signature should be like this:

def __exit__(self, exc_type, exc_val, exc_tb)

In this way, the _ exit _ function can obtain all information about the exception (exception type, abnormal value, and exception tracing information), which will help the exception handling operation. Here I will not discuss in detail how to write an exception. The following is an example, which only throws a SyntaxErrors exception.

class RaiseOnlyIfSyntaxError:   def __enter__(self):    pass   def __exit__(self, exc_type, exc_val, exc_tb):    return SyntaxError != exc_type

Capture exceptions:
When an exception is thrown in the with block, it is passed as a parameter to _ exit __. The three parameters are used, which are the same as those returned by sys. exc_info (): type, value, and trace (traceback ). If no exception is thrown, all three parameters are None. The context manager can "swallow" an exception by returning a True value from _ exit. The exception can be easily ignored, because if _ exit _ ends directly without return, the system returns None-a False value, and then throws it again after _ exit _ ends.

The ability to capture exceptions creates interesting possibilities. A classic example from unit test -- we want to ensure that some code throws the correct type of exception:

class assert_raises(object):  # based on pytest and unittest.TestCase  def __init__(self, type):    self.type = type  def __enter__(self):    pass  def __exit__(self, type, value, traceback):    if type is None:      raise AssertionError('exception expected')    if issubclass(type, self.type):      return True # swallow the expected exception    raise AssertionError('wrong exception type')with assert_raises(KeyError):  {}['foo']

5. talk about the content of the upper and lower libraries (contextlib ).

Contextlib is a Python module that provides an easy-to-use context manager.

(1) contextlib. closing

Suppose we have a database creation function that returns a database object and closes related resources (database connection sessions, etc.) after use)

We can process it as before or through the context manager:

with contextlib.closing(CreateDatabase()) as database:  database.query()

The contextlib. closing method calls the database closing method after the statement block is completed.

(2) contextlib. nested

Another cool feature can effectively help us reduce nesting:

Suppose we have two files, one reading and one writing, and we need to copy them.

We do not advocate the following:

with open('toReadFile', 'r') as reader:  with open('toWriteFile', 'w') as writer:    writer.writer(reader.read())

You can use contextlib. nested to simplify the process:

with contextlib.nested(open('fileToRead.txt', 'r'),            open('fileToWrite.txt', 'w')) as (reader, writer):  writer.write(reader.read())

In Python2.7, this method is replaced by a new syntax:

with open('fileToRead.txt', 'r') as reader, \    open('fileToWrite.txt', 'w') as writer:    writer.write(reader.read())contextlib.contextmanager

For advanced Python players, any function that can be split into two parts by the yield keyword can be implemented through the context manager decorated by the decorator. Any content before yield can be seen as an operation before the code block is executed, and any operation after yield can be placed in the exit function.

Here is an example of a thread lock:

The following is an example of thread-safe function writing:

import threading lock = threading.Lock() def safeWriteToFile(openedFile, content):  lock.acquire()  openedFile.write(content)  lock.release()

Next, let's use the context manager to implement it. let's look back at the previous analysis on yield and contextlib:

@contextlib.contextmanagerdef loudLock():  print 'Locking'  lock.acquire()  yield  print 'Releasing'  lock.release() with loudLock():  print 'Lock is locked: %s' % lock.locked()  print 'Doing something that needs locking' #Output:#Locking#Lock is locked: True#Doing something that needs locking#Releasing

Note that this is not an exception safe statement. If you want to ensure exceptional security, use the try Statement on yield. Fortunately, threading. Lock is already a context manager, so we only need to simply:

@contextlib.contextmanagerdef loudLock():  print 'Locking'  with lock:    yield  print 'Releasing'

Because threading. lock will return False through the _ exit _ function in case of an exception, which will be thrown again when yield is called. In this case, the lock will be released, but the call to "print 'releasing '" will not be executed unless we rewrite try-finally.

If you want to use the "as" keyword in the context manager, use yield to return the value you need. it will assign a value to the new variable through the as keyword. Next, let's take a closer look.

6. use the generator to define the context manager
When discussing generators, it is said that we are more inclined to generate them than the iterators implementing classes because they are shorter and more convenient, and their statuses are locally saved rather than in instances and variables. On the other hand, as described in the two-way communication section, the data stream between the generator and its callers can be two-way. Including exceptions, which can be passed directly to the generator. We want to implement the context manager as a special generator function. In fact, the generator protocol is designed to support this use case.

@contextlib.contextmanagerdef some_generator():  
 
    try:    yield 
  
     finally:    
   
  
 

Contextlib. contextmanager decorations a generator and converts it to the context manager. The generator must follow some rules that are enforced by wrapper functions-the most important thing is that it is at least yield once. Yield is executed from _ enter _. the code block in the context manager is executed when the generator stops at yield, and the rest is executed in _ exit. If an exception is thrown, the interpreter passes it to the packaging function through the _ exit _ parameter, and the packaging function throws an exception in the yield statement. By using the generator, the context manager becomes shorter and more refined.

Let's use a generator to override closing:

@contextlib.contextmanagerdef closing(obj):  try:    yield obj  finally:    obj.close()

Then rewrite assert_raises into a generator:

@contextlib.contextmanagerdef assert_raises(type):  try:    yield  except type:    return  except Exception as value:    raise AssertionError('wrong exception type')  else:    raise AssertionError('exception expected')

Here we use the decorator to convert the generated function into the context manager!

For more articles about the context manager in Python, refer to the PHP Chinese website!