Use mock libraries in Python to test Python code

How to test without patience

Often, the software we write interacts directly with what we call a "dirty" service. In layman's terms, services are critical to our application, and the interaction between them is designed for us, but this leads to undesirable side effects--those that we don't want when we test ourselves.

For example, maybe we're writing a social software program and want to test "features posted to Facebook," but we don't want to be posted on Facebook every time we run a test set.

Python's UnitTest Library has a sub-package called unittest.mock--or you declare it as a dependency, simplifying the module to mock--provides a very powerful and useful way to simulate or screen out these aspects that are not in our hope.
Note: The mock is recently included in the Python 3.3 standard library, and the previously released version must be downloaded from the mock library via PyPI.

Fear system call

For another example, consider system calls, and we'll discuss them in the remainder of the article. It is not difficult to find that these can be considered using simulations: whether you want to write a script to pop up a CD driver, or a Web service to delete a cache file in the/tmp directory, or a socket service to bind a TCP port, these calls are not expected in your unit testing.

As a developer, you are more concerned about whether your library has successfully called system functions to eject CDs, rather than experiencing each test with the CD tray open.

As a developer, you are more concerned about whether your library has successfully called system functions to eject CDs (with the correct parameters, etc.). Instead of experiencing each test, the CD tray is turned on (or worse, many times, when a unit test runs, many test points involve pop-up code).

Similarly, keeping your unit testing efficient and performance means keeping some "slow code" outside of automated testing, such as access to file systems and networks.

For our first example, we are going to refactor a standard Python test case from the original to the use of a mock. We will prove how to write a test case with a mock to make our tests smarter, faster, and to expose more questions about the work of our software.

A simple delete function

Sometimes we need to delete files from the file system, so we can write a function like this in Python, this function will make it easier to be our script to do this thing.

#!/usr/bin/env python#-*-coding:utf-8-*-import osdef rm (filename):  os.remove (filename)

Obviously, at this point in time, our RM method does not provide more functionality than the basic Os.remove method, but our code will be improved, allowing us to add more functionality here.

Let's write a traditional test case, that is, without a mock test:

#!/usr/bin/env python#-*-coding:utf-8-*-from mymodule import rmimport os.pathimport tempfileimport unittestclass RmTes TCase (unittest. TestCase):   Tmpfilepath = Os.path.join (Tempfile.gettempdir (), "Tmp-testfile")  def setUp (self): with    Open (Self.tmpfilepath, "WB") as F:      f.write ("Delete me!")      def test_rm (self):    # Remove the file    rm (Self.tmpfilepath)    # Test This it was actually removed    Self.assertfalse (Os.path.isfile (Self.tempfile), "Failed to remove the file.")

Our test case is fairly simple, but when it is run each time, a temporary file is created and then deleted. Furthermore, we have no way to test whether our RM method passes parameters to Os.remove. We can assume that it is based on the above test, but there are still many needs to be confirmed.

Refactoring and simulation testing

Let's refactor our test cases using a mock:

#!/usr/bin/env python#-*-coding:utf-8-*-from mymodule import rmimport mockimport unittestclass RmTestCase (unittest. TestCase):     @mock. Patch (' Mymodule.os ')  def test_rm (self, mock_os):    rm ("no Path")    # test that RM Called Os.remove with the right parameters    Mock_os.remove.assert_called_with ("Any path")

For these refactorings, we have fundamentally changed the way the test is run. Now, we have an internal object that allows us to use another functional validation.
Potential pitfalls.

The first thing to note is that we use the Mock.patch method to decorate the Mymodule.os simulation object and inject it into our test case simulation method. Is it more meaningful to emulate the OS, or is it more meaningful in Mymodule.os reference?

Of course, when Python appears in the Import and Management module, the usage is very flexible. At run time, the MyModule module has its own OS operating system--modules that are introduced into its own scope. Therefore, if we simulate the OS system, we will not see the impact of the simulation test on the MyModule module.

This sentence needs to be deeply remembered:

Copy the Code code as follows:

To simulate a project, you just need to know where it is used, not where it comes from.

If you need to simulate the Tempfile model for Myproject.app.MyElaborateClass, you may need to simulate each module of Myproject.app.tempfile to keep its own imports.

This is the way to simulate tests in a trap.

Add validation to ' RM '

The previously defined RM method is fairly straightforward. Before the blind deletion, we will take it to verify that a path exists and verify that it is a file. Let's refactor RM to make it smarter:

#!/usr/bin/env python#-*-coding:utf-8-*-import osimport os.pathdef rm (filename):  if Os.path.isfile (filename): C2/>os.remove (filename)

Very good. Now, let's tweak our test cases to keep the test coverage.

#!/usr/bin/env python#-*-coding:utf-8-*-from mymodule import rmimport mockimport unittestclass RmTestCase (unittest. TestCase):     @mock. Patch (' Mymodule.os.path ')  @mock. Patch (' Mymodule.os ')  def test_rm (self, mock_os, mock  _path):    # Set up the mock    mock_path.isfile.return_value = False         rm ("any path")        # test, the Remove call Was is not called.    Self.assertfalse (mock_os.remove.called, "Failed to not remove the file if not present.")        # make the file ' exist '    mock_path.isfile.return_value = True         rm ("any path")         Mock_os.remove.assert_ Called_with ("Any path")

Our test paradigm has changed completely. We can now verify and verify that the internal functionality of the method has any side effects.

Remove functionality as a service

So far, we have only provided simulation tests for function functions, and we have not tested the methods of objects and instances that need to pass parameters. Next we will show you how to test the method of the object.

First, we first re-compose the RM method into a service class. Actually converting such a simple function into an object does not require much tweaking, but it can help us understand the key concepts of mocks. The following is the refactoring code:

#!/usr/bin/env python#-*-coding:utf-8-*-import osimport os.pathclass Removalservice (object): "" "  A service for REM Oving objects from the filesystem.   "" " def rm (filename):    if Os.path.isfile (filename):      os.remove (filename)

You can see that our test cases don't actually make much of a change:

#!/usr/bin/env python#-*-coding:utf-8-*-from mymodule import removalserviceimport mockimport unittestclass RemovalSer Vicetestcase (unittest. TestCase):     @mock. Patch (' Mymodule.os.path ')  @mock. Patch (' Mymodule.os ')  def test_rm (self, mock_os, mock _path):    # Instantiate our service    reference = Removalservice ()        # Set up the mock    mock_ Path.isfile.return_value = False         reference.rm ("Any path") # test, the Remove call is not        called.    Self.assertfalse (mock_os.remove.called, "Failed to not remove the file if not present.")        # make the file ' exist '    mock_path.isfile.return_value = True         reference.rm ("Any path")         Mock_ Os.remove.assert_called_with ("Any path")

Very well, removalservice work as we planned. Next, let's create another service with this object as a dependency:

#!/usr/bin/env python#-*-coding:utf-8-*-import osimport os.pathclass Removalservice (object): "" "  A service for REM Oving objects from the filesystem.   "" " def rm (filename):    if Os.path.isfile (filename):      os.remove (filename)       class Uploadservice (object):   def __init__ (self, removal_service):    self.removal_service = Removal_service      def upload_complete (filename):    self.removal_service.rm (filename)

So far, our tests have covered removalservice, and we will not validate the Uploadservice internal function rm in our test case. Instead, we'll call Uploadservice's Removalservice.rm method for a simple test (in order not to produce any other side effects), and we'll know that it works correctly with the previous test case.

There are two ways to achieve these requirements:

1. Simulates the Removalservice.rm method itself.
2. A mock instance is provided in the constructor of the Uploadservice class.

Because both of these methods are very important in unit testing, we will review both methods at the same time.

Option 1: Methods for simulating instances

The simulation library has a special method to decorate the methods and parameters of the simulated object instance. @mock. Patch.object for decoration:

#!/usr/bin/env python#-*-coding:utf-8-*-from mymodule import removalservice, Uploadserviceimport mockimport UNITTESTC Lass Removalservicetestcase (unittest. TestCase): @mock. Patch (' Mymodule.os.path ') @mock. Patch (' Mymodule.os ') def test_rm (self, Mock_os, Mock_path): # in         Stantiate our service reference = Removalservice () # Set up the mock Mock_path.isfile.return_value = False    REFERENCE.RM ("Any path") # test, the Remove call is not called.        Self.assertfalse (mock_os.remove.called, "Failed to not remove the file if not present.") # make the file ' exist ' Mock_path.isfile.return_value = True reference.rm ("Any path") mock_os.remove.as Sert_called_with ("Any path") class Uploadservicetestcase (UnitTest. TestCase): @mock. Patch.object (Removalservice, ' rm ') def test_upload_complete (self, MOCK_RM): # Build our Dependencie s removal_service = Removalservice () reference = Uploadservice (removal_service) # call UPload_complete, which should, in turn, call ' RM ': Reference.upload_complete ("My uploaded file") # Check that it  Called the RM method of any Removalservice mock_rm.assert_called_with ("My uploaded file") # Check that it called The RM method of _our_ Removal_service removal_service.rm.assert_called_with ("My uploaded file")

That's great! We verified that the upload service successfully called the RM method of the instance. Did you notice the interesting part of this? This patching mechanism actually replaces the RM method of the Delete service instance of our test method. This means that we can actually examine the instance itself. If you want to learn more, you can try to get a better understanding of how this patching mechanism works by breaking breakpoints in the code of the simulation test.

Traps: Order of decorations

The order of decorations is important when decorating test methods with multiple decorative methods, but it is easy to confuse. Basically, when decorating methods are mapped to test methods with parameters, the work order of the decoration method is reversed. For example, the following:

  @mock. Patch (' Mymodule.sys ')  @mock. Patch (' Mymodule.os ')  @mock. Patch (' Mymodule.os.path ')  def test_ Something (self, mock_os_path, Mock_os, Mock_sys):    Pass

Have you noticed that the parameters of our decorating method are in reverse matching? This is partly because of how Python works. The following is the actual code execution order when using multiple decorating methods:

Patch_sys (Patch_os (Patch_os_path (test_something)))

Since this patch on SYS is at the outermost layer, it will be executed at the end, making it the last parameter of the actual test method. Please pay special attention to this and use the debugger in doing the testing to ensure that the correct parameters are injected in the correct order.

Option 2: Create a mock Test interface

We can provide a mock test instance in the Uploadservice constructor, rather than simulating the creation of a specific simulation test method. I recommend using option 1 as it is more precise, but in most cases option 2 is necessary and more efficient. Let's refactor our test instance again:

#!/usr/bin/env python#-*-coding:utf-8-*-from mymodule import removalservice, Uploadserviceimport mockimport UNITTESTC Lass Removalservicetestcase (unittest. TestCase): @mock. Patch (' Mymodule.os.path ') @mock. Patch (' Mymodule.os ') def test_rm (self, Mock_os, Mock_path): # in         Stantiate our service reference = Removalservice () # Set up the mock Mock_path.isfile.return_value = False    REFERENCE.RM ("Any path") # test, the Remove call is not called.        Self.assertfalse (mock_os.remove.called, "Failed to not remove the file if not present.") # make the file ' exist ' Mock_path.isfile.return_value = True reference.rm ("Any path") mock_os.remove.as Sert_called_with ("Any path") class Uploadservicetestcase (UnitTest. TestCase): Def test_upload_complete (self, MOCK_RM): # build our dependencies Mock_removal_service = Mock.create_au Tospec (removalservice) reference = Uploadservice (mock_removal_service) # call Upload_compleTe, which should, in turn, call ' RM ': Reference.upload_complete ("My uploaded file") # test that it called the RM Method Mock_removal_service.rm.assert_called_with ("My uploaded file")

In this case, we don't even need to add any functionality, just create a Removalservice class with the Auto-spec method and inject that instance into Uploadservice to validate the method.

Mock.create_autospec provides an equivalent instance of a function for the class. This means that, in practice, when interacting with the returned instance, an exception is thrown if an illegal method is used. More specifically, if a method is called with an incorrect number of arguments, an exception is thrown. This is very important for refactoring. When a library changes, the interruption test is exactly what is expected. If you do not use Auto-spec, even if the underlying implementation has been compromised, our tests will still pass.

Trap: Mock. Mock and Mock.magicmock classes

The mock library contains two important class mocks. Mock and mock.magicmock, most intrinsic functions are built on top of these two classes. Use a mock in the selection. Mock instances, mock. When magicmock an instance or Auto-spec method, it is often preferred to use the Auto-spec method, as it is able to maintain the rationality of testing for future changes. This is because of the mock. Mocks and Mock.magicmock ignore the underlying API, accepting all method calls and parameter assignments. For example, the following use case:

Class Target (object):  def apply (value):    return Valuedef method (Target, value):  return target.apply (value )

We use mocks like this. Mock instances to do the testing:

Class Methodtestcase (UnitTest. TestCase):   def test_method (self):    target = mock. Mock ()     method (target, "value")     target.apply.assert_called_with ("value")

This logic seems reasonable, but if we modify the Target.apply method to accept more parameters:

Class Target (object):  def apply (value, are_you_sure):    if are_you_sure:      return value    else:      Return None

Rerun your test, and then you will find it still able to pass. This is because it is not created for your API. This is why you should always use the Create_autospec method, and use the Autospec parameter when using @patch and @patch.object decorating methods.

Real-world example: imitating a Facebook API call

At the end of the day, let me write a more practical example of the real world, which has been mentioned in our introductory section: Send a message to Facebook. We'll write a nice wrapper class, and a test case that generates a response.

Import Facebookclass Simplefacebook (object):     def __init__ (self, Oauth_token):    self.graph = Facebook. Graphapi (Oauth_token)  def post_message (self, Message): "" "    Posts a message to the Facebook wall.    " " Self.graph.put_object ("Me", "feed", Message=message)

Here is our test case, which checks that I sent the message, but does not actually send out this message (on Facebook):

Import facebookimport simple_facebookimport mockimport unittestclass simplefacebooktestcase (unittest. TestCase):     @mock. Patch.object (Facebook. Graphapi, ' Put_object ', autospec=true)  def test_post_message (self, mock_put_object):    SF = Simple_facebook. Simplefacebook ("Fake OAuth token")    sf.post_message ("Hello world!")    # Verify    Mock_put_object.assert_called_with (message= "Hello world!")

As far as we can see, it's really easy to start writing smarter tests in Python using mock-up.
Summarize

Python's mock library, which is used to confuse ideas, is a game changer for unit testing. We've shown some of the usual usage scenarios by starting with mock-up in unit tests, and hopefully this article will help Python overcome the first hurdle and write good, testable code.

