In-depth analysis of the context manager in Python,

In-depth analysis of the context manager in Python,

1. What is the context manager?

For example, when writing Python code, you often put a series of operations in a statement block:

(1) When a condition is true-execute this statement Block

(2) When a condition is true, execute the statement block cyclically.

Sometimes we need to maintain a certain state when the program runs in the statement block and end this state after leaving the statement block.

Therefore, in fact, the context manager task is to prepare the code block before it is executed, and clean up the code block after it is executed.

Context manager is a feature added in Python2.5, which makes your code more readable and has fewer errors. Next, let's take a look at how to use it.

2. How to Use the context manager?

Code is the best way to learn. Let's see how we open a file and write "Hello World "?

filename = 'my_file.txt'mode = 'w' # Mode that allows to write to the filewriter = open(filename, mode)writer.write('Hello ')writer.write('World')writer.close()

Line 1-2: Specify the file name and open mode (write ).

Line 3: open the file, write "Hello world" in lines 4-5, and close the file in line 3.

That's not enough. Why do we still need the context manager? However, we ignore a small but important detail: What if we don't have a chance to close the file at line 3?

For example, the disk is full. Therefore, an exception is thrown when we try to write data to the file in row 4th, and row 6th has no chance of execution.

Of course, we can use try-finally statement block for packaging:

writer = open(filename, mode)try:  writer.write('Hello ')  writer.write('World')finally:  writer.close()

The code in the finally statement block is executed no matter what happens in the try statement block. Therefore, the file will be closed. Is there any problem with this? Of course not, but when we do something more complex than writing "Hello world", try-finally statements will become ugly. For example, if we want to open two files, read one write, and copy between the two files, the with statement can ensure that both files can be closed at the same time.

OK. Let's break down the matter:

(1) first, create a file variable named "writer.

(2) then perform some operations on writer.

(3) Close writer.

Is it more elegant?

with open(filename, mode) as writer:  writer.write('Hello ')   writer.write('World')

Let us go deeper. "with" is a new keyword and always appears along with the context manager. "Open (filename, mode)" has appeared in the previous code. "As" is another keyword that refers to the content returned from the "open" function and assigned it to a new variable. "Writer" is a new variable name.

Line 2-3: Enable a new code block. In this Code block, we can perform any operation on writer. In this way, we use the "open" context manager, which ensures that our code is both elegant and safe. It completes the try-finally task.

Open functions can be used as simple functions and context managers. This is because the open function returns a file type variable, which implements the write method we used previously, however, some special methods must be implemented as the context manager. I will introduce them in the following section.

3. Custom context Manager

Let's write an "open" context manager.

To implement the context manager, you must implement two methods: one is responsible for the preparation of the block into the statement, and the other is responsible for the aftermath of the block. At the same time, we need two parameters: File Name and open mode.

The Python class contains two special methods: __enter _ and _ exit _ (double-underline as the prefix and suffix ).

When an object is used as the context Manager:

(1) The _ enter _ method will be called before entering the code block.

(2) The _ exit _ method is called after the code block is left (even if an exception occurs in the code block ).

The following is an example of the context manager, which is printed when the code block is entered and left.

class PypixContextManagerDemo:   def __enter__(self):    print 'Entering the block'   def __exit__(self, *unused):    print 'Exiting the block' with PypixContextManagerDemo():  print 'In the block' #Output:#Entering the block#In the block#Exiting the block

Note:

(1) No parameters are passed.
(2) The "as" keyword is not used here.
We will discuss the parameter settings for the _ exit _ method later.
How do we pass parameters to a class? In fact, the _ init _ method can be used in any class. Here we will rewrite it to receive two necessary parameters (filename, mode ).

When we enter the statement block, the open function will be used, as in the first example. When we leave the statement block, all things opened in the _ enter _ function will be closed.

The following is our code:

class PypixOpen:   def __init__(self, filename, mode):    self.filename = filename    self.mode = mode   def __enter__(self):    self.openedFile = open(self.filename, self.mode)    return self.openedFile   def __exit__(self, *unused):    self.openedFile.close() with PypixOpen(filename, mode) as writer:  writer.write("Hello World from our new Context Manager!")

Let's take a look at the changes:

(1) lines 3-5 receive two parameters through _ init.

(2) lines 7-9. open the file and return it.

(3) line 12: Close the file when the statement block is left.

(4) lines 14-15 simulate open using our own context manager.

In addition, there are some things to emphasize:

4. How to handle exceptions

We completely ignore the problems that may occur inside the statement block.

If an exception occurs inside the statement block, the __exit _ method will be called, and the exception will be thrown again (re-raised ). When processing file write operations, you certainly do not want to hide these exceptions most of the time, so this is acceptable. For exceptions that do not want to be thrown again, we can make the _ exit _ method simply return True to ignore all exceptions in the statement block (this is not wise in most cases ).

We can learn more details when an exception occurs. The complete _ exit _ FUNCTION signature should be like this:

def __exit__(self, exc_type, exc_val, exc_tb)

In this way, the _ exit _ function can obtain all information about the exception (exception type, abnormal value, and exception tracing information), which will help the exception handling operation. Here I will not discuss in detail how to write an exception. The following is an example, which only throws a SyntaxErrors exception.

class RaiseOnlyIfSyntaxError:   def __enter__(self):    pass   def __exit__(self, exc_type, exc_val, exc_tb):    return SyntaxError != exc_type

Capture exceptions:
When an exception is thrown in the with block, it is passed as a parameter to _ exit __. The three parameters are used, which are the same as those returned by sys. exc_info (): type, value, and trace (traceback ). If no exception is thrown, all three parameters are None. The context manager can "Swallow" an exception by returning a True value from _ exit. The exception can be easily ignored, because if _ exit _ ends directly without return, the system returns None-a False value, and then throws it again after _ exit _ ends.

The ability to capture exceptions creates interesting possibilities. A classic example from unit test -- we want to ensure that some code throws the correct type of exception:

class assert_raises(object):  # based on pytest and unittest.TestCase  def __init__(self, type):    self.type = type  def __enter__(self):    pass  def __exit__(self, type, value, traceback):    if type is None:      raise AssertionError('exception expected')    if issubclass(type, self.type):      return True # swallow the expected exception    raise AssertionError('wrong exception type')with assert_raises(KeyError):  {}['foo']

5. Talk about the content of the upper and lower libraries (contextlib ).

Contextlib is a Python module that provides an easy-to-use context manager.

(1) contextlib. closing

Suppose we have a database creation function that returns a database object and closes related resources (database connection sessions, etc.) after use)

We can process it as before or through the context Manager:

with contextlib.closing(CreateDatabase()) as database:  database.query()

The contextlib. closing method calls the database closing method after the statement block is completed.

(2) contextlib. nested

Another cool feature can effectively help us reduce nesting:

Suppose we have two files, one reading and one writing, and we need to copy them.

We do not advocate the following:

with open('toReadFile', 'r') as reader:  with open('toWriteFile', 'w') as writer:    writer.writer(reader.read())

You can use contextlib. nested to simplify the process:

with contextlib.nested(open('fileToRead.txt', 'r'),            open('fileToWrite.txt', 'w')) as (reader, writer):  writer.write(reader.read())

In Python2.7, this method is replaced by a new syntax:

with open('fileToRead.txt', 'r') as reader, \    open('fileToWrite.txt', 'w') as writer:    writer.write(reader.read())contextlib.contextmanager

For advanced Python players, any function that can be split into two parts by the yield keyword can be implemented through the context manager decorated by the decorator. Any content before yield can be seen as an operation before the code block is executed, and any operation after yield can be placed in the exit function.

Here is an example of a thread lock:

The lock mechanism ensures that the two segments of Code do not interfere with each other during simultaneous execution. For example, if we have two pieces of concurrently executed code to write a file at the same time, we will get an error file mixed with two inputs. But if we can have a lock, any code that wants to write a file must first obtain the lock, then it is easy to do. If you want to know