tutorial on the use of test modules UnitTest and doctest in Python

I want to confess a little. Although I am a creator of a fairly wide range of public domain Python libraries, the unit tests introduced in my module are very non-systematic. In fact, most of those tests were included in Gnosis.xml.pickle's Gnosis Utilities, and were written by contributors to the sub-package (subpackage). I also found that most of the third-party Python packages I downloaded lacked a complete set of unit tests.

Not only that, the existing tests in Gnosis Utilities are also trapped in another flaw: you often need to infer the desired output in a very large amount of detail to determine the success or failure of the test. Testing is actually--in many cases--more like a small utility that uses parts of the library. These tests (or utilities) support the output of input and/or descriptive data formats from any data source (type correct). In fact, these test utilities are more useful when you need to debug some minor errors. But for the self-explanatory integrity check (sanity checks) of changes between libraries, these kinds of tests are not up to date.

In this installment, I try to use the Python standard library module doctest and unittest to improve the testing in my utility toolset and lead you to experience with me (and point out some of the best methods).

Script gnosis/xml/objectify/test/test_basic.py gives a typical example of the shortcomings of the current test and the solution. The following is the latest version of the script:

Listing 1. test_basic.py

"Read and print and objectified XML file" Import sysfrom gnosis.xml.objectify import xml_objectify, Pyobj_printerif len (sys . argv) > 1:for filename in sys.argv[1:]:  for parser in (' DOM ', ' EXPAT '):   try:    xml_obj = xml_objectify (file Name, parser=parser)    py_obj = xml_obj.make_instance ()    print Pyobj_printer (py_obj). Encode (' UTF-8 ')    Sys.stderr.write ("+ + SUCCESS (using" +parser+ ") \ n")    print "=" *50   except:    sys.stderr.write ("+ + FAILED ( Using "+parser+") \ n ")    print" = "*50else:print" Please specify one or more XML files to Objectify. "

The utility function Pyobj_printer () generates any Python object (specifically an object that does not use any of the gnosis.xml.objectify's other utilities, nor does it use the Gnosis Utilities Any other thing) is a non--xml representation. In a later release, I'll probably move this function somewhere else in the Gnosis package. In any case, Pyobj_printer () uses the indentation and notation of various class-python to describe objects and their properties (similar to Pprint, but extends the instance, not just the extended built-in data type).

If some special XML may not be properly "objectified", the test_basic.py script provides a good debugging tool-you can visually view the properties and values of the resulting object. In addition, if you redirect STDOUT, you can view a simple message on STDERR, as in this example:

Listing 2. Analyzing STDERR result messages

$ python test_basic.py testns.xml >/dev/null++ SUCCESS (using DOM) + + FAILED (using EXPAT)

However, the definition of success or failure in the example run above is not obvious: success simply means that there is no exception, not that the (redirected) output is correct.
Using Doctest

The Doctest module allows you to embed annotations within a document string (docstrings) to show the expected behavior of various statements, especially the results of functions and methods. This is much like making a document string look like an interactive shell session; An easy way to do this is from a Python interactive shell (or from Idel, Pythonwin, Macpython, or other Ides with interactive sessions). Medium) copy-paste. This improved test_basic.py script illustrates the addition of self-diagnostic features:

Listing 3. test_basic.py scripts with self-diagnostic capabilities

Import sysfrom gnosis.xml.objectify import xml_objectify, Pyobj_printer, EXPAT, DOMLF = "\ n" def show (XML_SRC, parser): "" " Self test using simple or user-specified XML data >>> XML = "<?xml version=" 1.0 "?> ......
 
  
   
  ...
 
   
    Some text about eggs. 
 
    ... 
   
    Ode to Spam
    
   ...
 ">>> squeeze = lambda s:s.replace (LF*2,LF). Strip () >>> print squeeze (show (Xml,dom) [0])-----* _xo_s Pam *-----{Eggs} pcdata=some text about Eggs.  {Morespam} Pcdata=ode to Spam >>> print squeeze (show (XML,EXPAT) [0])-----* _xo_spam *-----{Eggs} pcdata=some text about eg Gs.  {Morespam} Pcdata=ode to Spam pcdata= "" "Try:xml_obj = Xml_objectify (xml_src, parser=parser) py_obj = Xml_obj.make_instance () re Turn (Pyobj_printer (py_obj) encode (' UTF-8 '), "+ + SUCCESS (using" +parser+ ") \ n") Except:return ("", "+ + FAILED (using" +parser+ ") \ n") if __name__ = = "__main__": If Len (SYS.ARGV) ==1 or sys.argv[1]== "-V": Import doctest, Test_basic Doctest.te Stmod (Test_basic) elif sys.argv[1] In ('-H ', '-help ', '--help '): print "You could specify XML files to objectify instead of s   Elf-test "Print" (use '-V ' for verbose output, otherwise no-message means success) "Else:for filename in sys.argv[1:]: For parser in (DOM, EXPAT): output, message = Show (filename, parseR) Print output sys.stderr.write (message) print "=" *50 

Note that I put the main code block in the improved (and extended) test script so that if you specify an XML file on the command line, the script will continue to perform the previous behavior. This allows you to continue to analyze the XML outside of the test case and focus only on the results--or find out the errors in Gnosis.xml.objectify's work, or simply understand their purpose. In a standard way, you can use the-H or--help parameter to get a description of the usage.

When you run test_basic.py without any parameters (or with a-v parameter that is only used by doctest), you will find interesting new features. In this example, we run Doctest on the module/script itself-you can see that we actually import the test_basic into the script's own namespace so that we can simply import the other modules that we want to test. The Doctest.testmod () function goes through the module itself, its functions, and all the document strings in its class to find out all the content of a similar interactive shell session, in this case, a session is found in the show () function.

The document string for Show () illustrates several small "traps" (gotchas) during a well-designed doctest session. Unfortunately, when parsing an explicit session, Doctest handles the empty line as the end of the session-so output such as the return value of Pyobj_printer () requires some protection (be munged slightly) for testing. The simplest approach is to use a function such as squeeze () as defined by the document string itself (it simply removes the newline immediately following it). In addition, because the document string is a string swap (escape) after all, the sequence is expanded so that it is slightly confusing to swap lines within the code sample. You can use \\n, but I find that the definition of LF solves these problems.

The self-test defined in the document string of Show () does not only make sure that no exception occurs (against the original test script). At least one simple XML document should be checked for the correct "objectification". Of course, it's still possible that some other XML document could not be handled correctly-for example, the namespace XML document Testns.xml We tried above encountered a EXPAT parser failure. Document strings processed by Doctest may contain backtracking (traceback) within them, but in special cases, a better approach is to use UnitTest.

Using UnitTest

Another test contained in the gnosis.xml.objectify is test_expat.py. The main reason for this test is that a child package that uses the EXPAT parser often needs to invoke a special setup function to enable the processing of the namespace-aware XML document (the reality is that it evolved rather than designed and may change later). The old test attempts to print the object without the help of the setting, captures it if an exception occurs, and then prints it (and gives a message about what happened) if necessary.

And if you use test_basic.py, the test_expat.py tool allows you to analyze how Gnosis.xml.objectify describes a novel XML document. But as before, there are a lot of specific behaviors that we might want to verify. An enhanced, extended version of test_expat.py uses UnitTest to analyze what happens when various actions are executed, including assertions that hold a particular condition or (approximate) equation, or some unexpected exception that is expected. Take a look at:

Listing 4. Self-diagnosing test_expat.py scripts

"Objectify using Expat parser, namespace setup where needed" import UnitTest, sys, cstringiofrom os.path import Isfilefrom Gnosis.xml.objectify import make_instance, config_nspace_sep,\ xml_objectifybasic, NS = ' test.xml ', ' Testns.xml ' CLA SS Prerequisite (UnitTest. TestCase): def testhavelibrary (self): "Import the Gnosis.xml.objectify Library" import gnosis.xml.objectify def testhave  Files (self): "Check for sample XML files, NS and BASIC" Self.failunless (Isfile (BASIC)) self.failunless (Isfile (NS)) class Expattest (unittest. TestCase): def setUp (self): Self.orig_nspace = XML_Objectify.expat_kwargs.get (' nspace_sep ', ') def testnonamespace ( Self): ' Objectify namespace-free XML document ' o = Make_instance (BASIC) def testnamespacefailure (self): "Raise syntaxer Ror on Non-setup namespace XML "Self.assertraises (SyntaxError, Make_instance, NS) def testnamespacesuccess (self):" Suces sfully objectify NS after Setup "Config_nspace_sep (None) o = Make_instance (NS) def testnspacebasic (self):  "Successfully objectify basic despite extra setup" Config_nspace_sep (None) o = Make_instance (BASIC) def tearDown (self) : xml_objectify.expat_kwargs[' nspace_sep ' = self.orig_nspaceif __name__ = = ' __main__ ': If Len (sys.argv) = = 1:unittest. Main () Elif sys.argv[1] In ('-Q ', '--quiet '): Suite = UnitTest. TestSuite () suite.addtest (Unittest.makesuite (prerequisite)) Suite.addtest (Unittest.makesuite (expattest)) out = Cstringio.stringio () results = UnitTest. Texttestrunner (Stream=out). Run (Suite) if not results.wassuccessful (): For failure in Results.failures:print "FAIL:" , Failure[0] for the error in Results.errors:print "error:", Error[0] elif sys.argv[1].startswith ('-'): # Pass the args to U  Nittest Unittest.main () else:from gnosis.xml.objectify import pyobj_printer as show Config_nspace_sep (None) for fname In sys.argv[1:]: Print Show (Make_instance (fname)). Encode (' UTF-8 ')

The use of UnitTest adds quite a lot of power to the simpler doctest approach. We can divide our tests into classes, each of which inherits from UnitTest. TestCase. Within each test class, each method that starts with the name ". Test" is considered to be another test. The two additional classes defined for Expattest are interesting: run. SetUp () before each use of the class and run. TearDown () at the end of the test (whether the test is successful, failed, or has an error). In our example above, we do a little bookkeeping for the dedicated Expat_kwargs dictionary to ensure that each test runs independently.

By the way, the difference between failure (failure) and errors (error) is important. A test may fail because some specific assertion is invalid (the assertion method starts with ". Fail" or begins with ". Assert"). In a sense, failure is expected-at least in a way that we have specifically analyzed. On the other hand, the error is an unexpected problem--because we don't know where it went wrong beforehand, we need to analyze the backtracking in the actual test run to diagnose the problem. However, we can design a hint for failure to give a diagnostic error. For example, if Prerequisite.havefiles () fails, errors will occur in some testexpat tests, and if the former is successful, you will have to go somewhere else to find the source of the error.

In UnitTest. In the TestCase inheritance class, some of the specific test methods may be included. Assert ... () or. Fail ... () method, but it may just have a series of actions that we believe should be executed successfully. If the test method does not run as expected, we will get an error (and the backtracking that describes the error).

The _main_ block in test_expat.py is also worth seeing. In the simplest case, we can run the test case using only Unittest.main (), which will determine what needs to be run. With this approach, the UnitTest module will accept a-v option to give more detailed output. Based on the specified file name, after the namespace setting has been performed, we print out the representation of the specified XML file, which roughly preserves backward compatibility with the older version of the tool.

The most interesting branch of _main_ is the one that expects the-Q or--quiet tags. As you will expect, this branch will be silent unless there is a failure or error (quiet, which minimizes output). Not only that, because it is silent, it only shows a single row for each question about the failure/error location of the report, not the entire diagnostic backtracking. In addition to the direct use of the silent output style, this branch provides examples of custom tests versus test suites and control of the results report. A little bit longer unittest. The default output of Texttestrunner () is directed to stringio out-if you want to see it, you are welcome to Out.getvalue () to find it. However, the result object lets us test for full success and, if not completely successful, lets us handle failures and errors. Obviously, because they are values in variables, you can easily record the contents of the result object into the log, or display it in the GUI, no matter what, not just print to STDOUT.

Combination Test

Perhaps the best feature of the UnitTest framework is that you can easily combine tests that contain different modules. In fact, if you use Python 2.3+, you can even convert the doctest test into a unittest suite. Let's combine the tests we've created so far into a script test_all.py (admittedly, it's a bit of an exaggeration to say that it's the test we've done so far):

Listing 5. TEST_ALL.PY combines unit Testing

"Combine Tests for Gnosis.xml.objectify package (req 2.3+)" Import UnitTest, Doctest, test_basic, test_expatsuite = doctest . Doctestsuite (Test_basic) suite.addtest (Unittest.makesuite (Test_expat. Prerequisite)) Suite.addtest (Unittest.makesuite (Test_expat. expattest)) UnitTest. Texttestrunner (verbosity=2). Run (Suite)

Because test_expat.py only contains test classes, they can be easily added to a local test suite. Doctest. The Doctestsuite () function performs a conversion of a document string test. Let's take a look at what test_all.py does when it runs:

Listing 6. Successful output from the test_all.py

$ python2.3 test_all.pydoctest of test_basic.show ... okcheck for sample XML files, NS and basic ... okimport the gnosis.x Ml.objectify Library ... okraise syntaxerror on Non-setup namespace XML ... oksucessfully objectify NS after Setup ... OkO Bjectify namespace-free XML Document ... oksuccessfully objectify BASIC despite extra setup ... ok------------------------ ----------------------------------------------Ran 7 tests in 0.052sOK

Note the description of the tests performed: In the case of using the UnitTest test method, their descriptions are derived from the corresponding docstring functions. If you do not specify a document string, the class and method names are used as the most appropriate description. Let's take a look at what we get when some tests fail, just as interesting (the backtracking details are removed for this article):

Listing 7. Results when some tests fail

$ mv testns.xml testns.xml# && python2.3 test_all.py 2>&1 | Head-7doctest of Test_basic.show ... okcheck for sample XML files, NS and basic ... Failimport the Gnosis.xml.objectify Library ... okraise syntaxerror on Non-setup namespace XML ... errorsucessfully objectify NS after Setup ... Errorobjectify namespace-free XML Document ... oksuccessfully objectify BASIC despite extra setup ... ok

Casually mentioned, this failure to write to STDERR's last line is "FAILED (Failures=1, errors=2)", if you need this is a good summary (relative to the success of the final "OK").

Start from here

This article introduces you to some typical uses of UnitTest and doctest, which have improved the testing in my own software. Read the Python documentation to learn more about the full range of methods available for testing suites, test cases, and test results. They all follow the pattern described in the example.

Making yourself compliant with Python's standard test module prescribes methodology as a good software practice. The development of test-driven (Test-driven) is popular in many software cycles, but it is clear that Python is a language suitable for test-driven models. Also, if you are only considering that a package is more likely to work as planned, a package or library can be more useful to users than a package or library that lacks these tests if accompanied by a comprehensive set of tests.

