Simulate and test Python code with mock libraries in Python _python

How to test without patience

Often, the software we write will interact directly with what we call "dirty" services. In layman's terms, services are critical to our applications, and the interactions between them are designed, but this leads to undesirable side effects-things we don't want to do when we test ourselves.

For example, maybe we're writing a social software and want to test the "post to Facebook feature," but we don't want to be posted to Facebook every time we run the test set.

Python's unittest Library has a child package called unittest.mock--or you declare it as a dependency, simplifying the mock--module provides a powerful and useful way to simulate or screen out these areas that are not what we want.
Note: Mocks are recently included in the Python 3.3 standard library; the previously released version must download the mock library via PyPI.

Fear system call

To take another example, consider system calls, which we will discuss in the remainder of the article. It's not hard to see that these can be considered using simulations: whether you want to write a script to eject a CD driver, or a Web service to delete a cached file in the/tmp directory, or a socket service to bind a TCP port, these calls are not expected when you unit tests.

As a developer, you're more concerned about whether your library has successfully invoked a system function to eject a CD, rather than experiencing a CD tray opening every time you test.

As a developer, you are more concerned about whether your library successfully invokes the system function to eject the CD (with the correct parameters, etc.). Instead of experiencing every test the CD tray is turned on (or worse, many times, when a unit test runs, many test points involve pop-up code).

Similarly, keeping your unit testing efficiency and performance means you need to keep some "slow code" other than automated tests, such as file systems and network access.

For our first example, we'll refactor a standard Python test case from original to mock use. We'll show you how to write a test case with a mock to make our tests smarter, faster, and to expose more questions about our software work.

A simple deletion function

Sometimes we need to delete files from the file system, so we can write a function like this in Python, this function will make it easier to be our script to do this thing.

#!/usr/bin/env python#-*-coding:utf-8-*-import osdef rm (filename):
  os.remove (filename)

Obviously, at this point in time, our RM method does not offer more functionality than the basic Os.remove method, but our code will be improved to allow us to add more functionality here.

Let's write a traditional test case, that is, without a mock test:

#!/usr/bin/env python#-*-coding:utf-8-*-from mymodule import rmimport os.pathimport tempfileimport Unittestclass Tcase (unittest. TestCase):
 
  Tmpfilepath = Os.path.join (Tempfile.gettempdir (), "Tmp-testfile")  def setUp (self): with
    Open (Self.tmpfilepath, "WB") as F:
      f.write ("Delete me!")    
  def test_rm (self):
    # Remove the file
    rm (Self.tmpfilepath)    # test that it is actually removed
    Self.assertfalse (Os.path.isfile (Self.tempfile), "Failed to remove the file.")

Our test case is fairly simple, but when it runs each time, a temporary file is created and then deleted. In addition, we have no way to test whether our RM method passes parameters to Os.remove. We can assume that it is based on the test above, but there is still much to be proved.

Refactoring and simulation test

Let's use a mock to refactor our test cases:

#!/usr/bin/env python#-*-coding:utf-8-*-from mymodule import rmimport mockimport unittestclass RmTestCase (unittest. TestCase):
   
  @mock. Patch (' Mymodule.os ')
  def test_rm (self, mock_os):
    rm ("any path")    # test that RM Called Os.remove with the right parameters
    mock_os.remove.assert_called_with (' Any path ')

For these refactorings, we have fundamentally changed the way the test is run. Now we have an internal object that allows us to use another feature validation.
potential pitfalls.

The first thing to note is that the Mock.patch method we use is decorated in the Mymodule.os mock object and injected into our test case simulation method. Is it more meaningful to simulate an OS, or is it more meaningful to Mymodule.os reference?

Of course, when Python appears in the Import and Management module, the usage is very flexible. At runtime, the MyModule module has its own OS operating system-a module that is introduced into its own scope. Therefore, if we simulate an OS system, we will not see the impact of the simulation test on the MyModule module.

This sentence needs to be deeply remembered:

Copy Code code as follows:

Simulate testing a project, only need to know where it is used, not where it comes from.

If you need to simulate a tempfile model for Myproject.app.MyElaborateClass, you may need to simulate each module of Myproject.app.tempfile to keep your own imports.

This is the way to simulate the test with a trap.

Add validation to ' RM '

The RM method defined earlier is fairly straightforward. Before the blind deletion, we will take it to verify that a path exists and verify that it is a file. Let's refactor RM to make it smarter:

#!/usr/bin/env python#-*-coding:utf-8-*-import osimport os.pathdef rm (filename):
  if Os.path.isfile (filename): C3/>os.remove (filename)

Very good. Now, let's adjust our test cases to keep the test coverage level.

#!/usr/bin/env python#-*-coding:utf-8-*-from mymodule import rmimport mockimport unittestclass RmTestCase (unittest. TestCase):
   
  @mock. Patch (' Mymodule.os.path ')
  @mock. Patch (' Mymodule.os ')
  def test_rm (self, mock_os, mock  _path):
    # Set up the mock
    mock_path.isfile.return_value = False
     
    rm (' any path ')    
    # test that's the remove call is not called.
    Self.assertfalse (mock_os.remove.called, "Failed to not remove the file if not present.")    
    # make the file ' exist '
    mock_path.isfile.return_value = True
     
    rm (' any path ')
     
    Mock_os.remove.assert_ Called_with ("Any path")

Our test paradigm has changed completely. Now we can verify and verify that the internal functionality of the method has any side effects.

To use the Delete function as a service

So far, we've only provided mock tests for function functions, and we don't simulate the methods of objects and instances that need to pass parameters. Next we'll describe how to simulate the methods of an object.

First, we'll first refactor the RM method to form a service class. Actually converting such a simple function into an object does not require much tweaking, but it can help us understand the key concepts of mocks. Here is the refactoring code:

#!/usr/bin/env python#-*-coding:utf-8-*-import osimport os.pathclass Removalservice (object):
  "" A Service for REM Oving objects from the filesystem.
 
  "" " def rm (filename):
    if Os.path.isfile (filename):
      os.remove (filename)

You can see that our test cases do not actually make much of a difference:

#!/usr/bin/env python#-*-coding:utf-8-*-from mymodule import removalserviceimport mockimport unittestclass RemovalSer Vicetestcase (unittest. TestCase):
   
  @mock. Patch (' Mymodule.os.path ')
  @mock. Patch (' Mymodule.os ')
  def test_rm (self, mock_os, mock _path):
    # Instantiate our service
    reference = Removalservice ()    
    # Set up the mock
    mock_ Path.isfile.return_value = False
     
    reference.rm (' Any path ') # test that's the remove call is not    
    called.
    Self.assertfalse (mock_os.remove.called, "Failed to not remove the file if not present.")    
    # make the file ' exist '
    mock_path.isfile.return_value = True
     
    reference.rm (' Any path ')
     
    Mock_ Os.remove.assert_called_with ("Any path")

Well, Removalservice is working as we planned. Next let's create another service with this object as a dependency:

#!/usr/bin/env python#-*-coding:utf-8-*-import osimport os.pathclass Removalservice (object):
  "" A Service for REM Oving objects from the filesystem.
 
  "" " def rm (filename):
    if Os.path.isfile (filename):
      os.remove (filename)      
 
class Uploadservice (object):
 
  def __init__ (self, removal_service):
    self.removal_service = Removal_service    
  def upload_complete (filename):
    self.removal_service.rm (filename)

So far, our tests have covered removalservice, and we will not validate the internal function RM for Uploadservice in our test cases. Instead, we'll call the Uploadservice Removalservice.rm method for a simple test (in order not to produce any other side effects), and we can tell by the previous test cases that it works correctly.

There are two ways to achieve these requirements:

1. Simulate the Removalservice.rm method itself.
2. Provides a mock instance in the constructor of the Uploadservice class.

Since both of these methods are very important in unit testing, we will review both approaches at the same time.

Option 1: Methods for simulating an instance

The simulation library has a special way to decorate methods and parameters for simulating object instances. @mock. Patch.object for decoration:
 

#!/usr/bin/env python#-*-coding:utf-8-*-from mymodule import removalservice. Uploadserviceimport mockimport UNITTESTC Lass Removalservicetestcase (unittest.
    TestCase): @mock. Patch (' Mymodule.os.path ') @mock. Patch (' Mymodule.os ') def test_rm (self, Mock_os, Mock_path): # Instantiate our service reference = Removalservice () # Set up the mock Mock_path.isfile.return_value =
    False reference.rm ("Any path") # test which is the remove call is not called.    
    Self.assertfalse (mock_os.remove.called, "Failed to not remove the file if not present.") # make the file ' exist ' Mock_path.isfile.return_value = True reference.rm (' Any path ') Mock_os.remo Ve.assert_called_with ("Any path") class Uploadservicetestcase (UnitTest. TestCase): @mock. Patch.object (Removalservice, ' rm ') def test_upload_complete (self, MOCK_RM): # Build our Depende Ncies Removal_service = Removalservice () reference = Uploadservice (RemovAl_service) # call Upload_complete, which should, in turn, called ' RM ': Reference.upload_complete ("My uploaded F Ile ") # Check that it called ' rm method ' any removalservice mock_rm.assert_called_with (' My uploaded file ' # Check that it called the RM method of _our_ Removal_service removal_service.rm.assert_called_with (' My Uplo
 aded file ")

That's great! We validated the RM method that the upload service successfully invoked the instance. Did you notice the interesting part of it? This patching mechanism actually replaces the RM method of the Delete service instance of our test method. This means that we can actually examine the instance itself. If you want to learn more, try the next breakpoint in the code that simulates the test to better understand how the patching mechanism works.

Traps: The order of the decorations

When using multiple decorative methods to decorate the test method, the order of decoration is important, but it is easily confusing. Basically, when the decoration method is mapped to the test method with parameters, the working order of the decoration method is reversed. For example, the following example:

  @mock. Patch (' Mymodule.sys ')
  @mock. Patch (' Mymodule.os ')
  @mock. Patch (' Mymodule.os.path ')
  def test_ Something (self, mock_os_path, Mock_os, Mock_sys): Pass
    

Have you noticed that the parameters of our decorating method are reverse-matched? This is partly because of how Python works. The following is the actual sequence of code execution when using multiple decoration methods:

Patch_sys (Patch_os (Patch_os_path (test_something))

Because this patch on SYS is at the outermost layer, it will be executed at the end, making it the last parameter of the actual test method. Please pay special attention to this and use the debugger to make sure that the correct parameters are injected in the correct order in the test.

Option 2: Create a mock Test interface

We can provide a mock test instance in the Uploadservice constructor, rather than simulate the creation of specific mock test methods. I recommend using option 1 because it's more accurate, but in most cases option 2 is necessary and more efficient. Let's refactor our test example again:
 

#!/usr/bin/env python#-*-coding:utf-8-*-from mymodule import removalservice. Uploadserviceimport mockimport UNITTESTC Lass Removalservicetestcase (unittest.
    TestCase): @mock. Patch (' Mymodule.os.path ') @mock. Patch (' Mymodule.os ') def test_rm (self, Mock_os, Mock_path): # Instantiate our service reference = Removalservice () # Set up the mock Mock_path.isfile.return_value =
    False reference.rm ("Any path") # test which is the remove call is not called.    
    Self.assertfalse (mock_os.remove.called, "Failed to not remove the file if not present.") # make the file ' exist ' Mock_path.isfile.return_value = True reference.rm (' Any path ') Mock_os.remo Ve.assert_called_with ("Any path") class Uploadservicetestcase (UnitTest. TestCase): Def test_upload_complete (self, MOCK_RM): # build our dependencies Mock_removal_service = Mock.creat  E_autospec (removalservice) reference = Uploadservice (Mock_removal_service)  
    # call Upload_complete, which should, in turn, called ' RM ': Reference.upload_complete ("My uploaded file")
 # test that it called ' rm method Mock_removal_service.rm.assert_called_with (' My uploaded file ')

In this case, we don't even need to add any functionality, just create a Removalservice class with the Auto-spec method, and then inject that instance into the Uploadservice to validate the method.

Mock.create_autospec provides an equivalent instance of functionality for the class. This means that, in practice, when interacting with the returned instance, an exception is thrown if an illegal method is used. More specifically, if a method is invoked with an incorrect number of arguments, an exception is thrown. This is very important for refactoring. When a library changes, interrupt testing is exactly what is expected. If you do not use Auto-spec, even if the underlying implementation has been compromised, our tests will still pass.

Pitfalls: Mocks. Mock and Mock.magicmock classes

The mock library contains two important class mocks. Mocks and mock.magicmock, most internal functions are built on these two classes. When you choose to use a mock. Mock instances, mocks. When magicmock an instance or Auto-spec method, it is usually preferred to use the Auto-spec method, because it can keep the test reasonable for future changes. This is because of the mock. Mocks and Mock.magicmock ignore the underlying APIs and accept all method calls and parameter assignments. For example, the following use case:

Class Target (object):
  def apply (value): Return
    Valuedef Method (Target, value): Return
  target.apply (value )

We use mocks as follows. Mock instances to test:

Class Methodtestcase (UnitTest. TestCase):
 
  def test_method (self):
    target = mock. Mock () method
 
    (target, "value")
 
    target.apply.assert_called_with ("value")

This logic seems reasonable, but if we modify the Target.apply method to accept more parameters:

Class Target (object):
  def apply (value, are_you_sure):
    if are_you_sure: return      value    else:      Return None

Rerun your test, and then you will find it still able to pass. This is because it is not created for your API. This is why you should always use the Create_autospec method and use the Autospec parameter when using the @patch and @patch.object decoration methods.

Real-world example: imitating a Facebook API call

At the end of the day, let me write a more practical example of the real world, which was mentioned in our introductory section: Sending a message to Facebook. We'll write a nice encapsulation class, and a test case that produces a response.

Import Facebookclass Simplefacebook (object):
   
  def __init__ (self, Oauth_token):
    self.graph = Facebook. Graphapi (Oauth_token)  def post_message (self, Message): "" "Posts a message to the
    Facebook wall.
    " " Self.graph.put_object ("Me", "feed", Message=message)

Here is our test case, which checks that I sent the message, but does not actually send this message (to Facebook):

Import facebookimport simple_facebookimport mockimport unittestclass simplefacebooktestcase (unittest. TestCase):
   
  @mock. Patch.object (Facebook). Graphapi, ' Put_object ', Autospec=true
  def test_post_message (self, mock_put_object):
    SF = Simple_facebook. Simplefacebook ("Fake OAuth token")
    sf.post_message ("Hello world!")    # Verify
    Mock_put_object.assert_called_with (message= "Hello world!")

As far as we can see, it's really easy to start writing smarter tests in Python with a mock.
Summary

Python's mock library, used to be an idea-obsessed, is a unit-testing game-changing player. We've shown some common usage scenarios by starting using mocks in unit tests, and hopefully this article will help Python overcome a first hurdle and write good code that can withstand testing.