Test modules in Python unittest and doctest tutorial _python

I want to be honest. Although I am the creator of a fairly broad public domain Python Library, the unit tests introduced in my module are very systematic. In fact, most of those tests were included in Gnosis.xml.pickle's Gnosis Utilities, and were written by contributors to the child package (subpackage). I also found that most of the Third-party Python packages I downloaded lacked a complete set of unit tests.

Not only that, the existing tests in Gnosis Utilities are also plagued by another flaw: you often need to infer the desired output in an extremely large amount of detail to determine the success or failure of the test. Testing is actually--in many cases--more like a small utility that uses parts of the library. These tests (or utilities) support output from any input and/or descriptive data format from any data source (of the correct type). In fact, these test utilities are more useful when you need to debug some minor errors. However, these types of tests are not up to the sanity checks for a custom-interpreted integrity check that changes between library versions.

In this installment, I tried using the Python standard library module doctest and unittest to improve the testing in my utility set and lead you to experience with me (and point out some of the best ways).

The script gnosis/xml/objectify/test/test_basic.py gives a typical example of the shortcomings of the current test and the solution. The following is the latest version of the script:

Listing 1. test_basic.py

"Read and print and objectified XML file"
import sys
from gnosis.xml.objectify import xml_objectify, Pyobj_printe R
If Len (SYS.ARGV) > 1: for
 filename in sys.argv[1:]:
  for parser in (' DOM ', ' EXPAT '):
   try:
    Xml_ obj = xml_objectify (filename, parser=parser)
    py_obj = xml_obj.make_instance ()
    print Pyobj_printer (py_obj). Encode (' UTF-8 ')
    sys.stderr.write ("+ + SUCCESS (using" +parser+ ") \ n")
    print "=" *50
   except:
    Sys.stderr.write ("+ + FAILED" (using "+parser+") \ n ")
    print" = "*50
else:
 print" Please specify one or more XML files to Objectify. "

Utility function Pyobj_printer () generates an arbitrary Python object (specifically an object that does not use any of the other utilities of the gnosis.xml.objectify, nor does it use the Gnosis Utilities Any other thing) is a-xml expression. In a future release, I'll probably move this function somewhere else in the Gnosis package. In any case, Pyobj_printer () uses a variety of class--python indents and symbols to describe objects and their properties (similar to Pprint, but extends the instance, not limited to extending the built-in data type).

If some particular XML may not be "objectified" correctly, the test_basic.py script provides a good debugging tool-you can visually view the properties and values of the resulting object. In addition, if you redirect STDOUT, you can view simple messages on STDERR, as in this example:

Listing 2. Analyzing STDERR result messages

$ python test_basic.py testns.xml >/dev/null
+ + SUCCESS (using DOM)
+ + FAILED (using EXPAT)

However, the definition of success or failure in the example running above is not obvious: success simply means that there is no exception, and that the output is not correct (redirected).
using Doctest

The Doctest module allows you to embed comments within a document string (docstrings) to show the expected behavior of various statements, especially the results of functions and methods. This is much like making a document string look like an interactive shell session; An easy way to do this is from a Python interactive shell (or from Idel, Pythonwin, Macpython, or other Ides with interactive sessions) Medium) copy-paste. This improved test_basic.py script illustrates the addition of the self-diagnosis feature:

Listing 3. test_basic.py scripts with self-diagnostic capabilities

Import sys from gnosis.xml.objectify import xml_objectify, Pyobj_printer, EXPAT, DOM LF = "\ n" def show (XML_SRC, parser): ' "' Self test using simple or user-specified XML data >>> XML = ' <?xml version= ' 1.0?> ... <! DOCTYPE Spam SYSTEM "Spam.dtd" > ... <Spam> ... <eggs>some text about eggs.</eggs> ... <morespa M>ode to spam</morespam> ... </Spam> ' >>> squeeze = lambda s:s.replace (LF*2,LF). Strip () >
 >> Print Squeeze (show (Xml,dom) [0])-----* _xo_spam *-----{Eggs} pcdata=some text about Eggs.
  {Morespam} Pcdata=ode to Spam >>> print squeeze (show (XML,EXPAT) [0])-----* _xo_spam *-----{Eggs} pcdata=some text Abou
 T eggs.
  {Morespam} Pcdata=ode to Spam pcdata= "" "Try:xml_obj = Xml_objectify (xml_src, parser=parser) py_obj = Xml_obj.make_instance (  ) Return (Pyobj_printer (py_obj). Encode (' UTF-8 '), "+ + SUCCESS (using" +parser+ ") \ n") Except:return ("", "+ + FAILED (using "+parser+ ") \ n") if __name__ = = "__main__": If Len (SYS.ARGV) ==1 or sys.argv[1]== "-V": Import doctest, Test_basic doctest.t Estmod (Test_basic) elif sys.argv[1] In ('-H ', '-help ', '--help '): print ' You-May-Specify XML files to objectify instead o F self-test "print" (use '-V ' for verbose output, otherwise no message means success) "Else:for filename in sys.argv [1:]: For parser in (DOM, EXPAT): output, message = Show (filename, parser) print output sys.stderr.write (Me

 ssage) print "=" *50

Notice that I put the main block of code in the improved (and extended) test script so that if you specify an XML file on the command line, the script will continue to perform the previous behavior. This allows you to continue to analyze the other XML in addition to the test case and focus on the results--or find out what gnosis.xml.objectify is doing, or just understand its purpose. In a standard way, you can use the-h or--help parameters to obtain a description of the usage.

Interesting new features are found when you run test_basic.py without any arguments (or with the-v argument used only by doctest). In this example, we run Doctest on the module/script itself--as you can see, we actually import test_basic into the script's own namespace so we can simply import other modules that we want to test. The Doctest.testmod () function goes through the module itself, its functions, and all the document strings in its class to find all the content that resembles an interactive shell session, and in this case a session is found in the show () function.

The document string for Show () illustrates several small "traps (gotchas)" that are in the process of designing a doctest session. Unfortunately, when parsing an explicit session, Doctest handles the empty row as the end of the session-so output such as the return value of Pyobj_printer () needs some protection (be munged slightly) for testing. The easiest way to do this is to use a function such as squeeze () as defined by the document string itself (it just drops the newline that follows). In addition, because the document string is, after all, a string-changing (escape), this sequence is extended so that swapping lines within the code sample is somewhat confusing. You can use \\n, but I find that the definition of LF solves these problems.

The self-test that is defined in the document string of Show () does not only make sure that the exception does not occur (in contrast to the original test script). Check for at least one simple XML document for the correct objectification. Of course, some other XML documents may still not be handled correctly-for example, the namespace XML document we tried above Testns.xml encountered a EXPAT parser failure. A document string processed by Doctest may include backtracking within it (Traceback), but in particular, the better approach is to use UnitTest.

Using UnitTest

Another test included in the Gnosis.xml.objectify is test_expat.py. The main reason for creating this test is that a child package using the EXPAT parser often needs to invoke a special set function to enable processing of namespace-enabled XML documents (the fact is that it evolved rather than designed, and may change later). The old test tries to print the object without the help of the settings, catch it if an exception occurs, and then print it (and give a message about what happened) if necessary.

With test_basic.py, the test_expat.py tool allows you to analyze how Gnosis.xml.objectify describes a novel XML document. But as before, there are a lot of specific behaviors that we might want to verify. An enhanced, extended version of test_expat.py uses UnitTest to analyze what happens when a variety of actions are performed, including assertions that hold a particular condition or (approximate) equation, or some unexpected exception that is expected. Take a look at:

Listing 4. Self-diagnosing test_expat.py scripts

"Objectify using Expat parser, namespace setup where needed" import UnitTest, sys, Cstringio from os.path import isfile F Rom gnosis.xml.objectify import make_instance, config_nspace_sep,\ xml_objectify BASIC, NS = ' test.xml ', ' testns.x ML ' class prerequisite (unittest. TestCase): def testhavelibrary (self): "Import the Gnosis.xml.objectify Library" import gnosis.xml.objectify def test Havefiles (self): "Check for sample XML files, NS and BASIC" Self.failunless (Isfile (Basic)) self.failunless (Isfile (NS ) class Expattest (UnitTest). TestCase): def setUp (self): Self.orig_nspace = XML_Objectify.expat_kwargs.get (' nspace_sep ', ') def testnonamespace (SE LF): "Objectify namespace-free XML document" o = Make_instance (BASIC) def testnamespacefailure (self): Raise Syntax Error on Non-setup namespace XML "Self.assertraises (SyntaxError, Make_instance, NS) def testnamespacesuccess (self):" sucessfully objectify ns after Setup "Config_nspace_sep (None) o = make_instance (ns)
 def testnspacebasic (self): "successfully objectify BASIC despite extra setup" Config_nspace_sep (None) o = make_ins 
 Tance (BASIC) def teardown (self): xml_objectify.expat_kwargs[' nspace_sep '] = self.orig_nspace if __name__ = ' __main__ ': If Len (sys.argv) = = 1:unittest.main () elif sys.argv[1] in (' Q ', '--quiet '): Suite = UnitTest. TestSuite () suite.addtest (Unittest.makesuite (prerequisite)) Suite.addtest (Unittest.makesuite (expattest)) out = CStr Ingio.stringio () results = UnitTest. Texttestrunner (Stream=out). Run (Suite) if not results.wassuccessful (): For failure in Results.failures:print "FAI L: ", failure[0] for error in Results.errors:print" error: ", Error[0] elif sys.argv[1].startswith ('-'): # pass ARG
  S to UnitTest Unittest.main () else:from gnosis.xml.objectify import pyobj_printer as show Config_nspace_sep (None)

 For fname in sys.argv[1:]: Print Show (Make_instance (fname)). Encode (' UTF-8 ')

The use of UnitTest adds a considerable amount of capacity to the simpler doctest approach. We can divide our tests into several classes, each of which inherits from UnitTest. TestCase. Within each test class, each method that starts with ". Test" is considered to be another test. The two extra classes that are defined for Expattest are interesting: run before each use of a class to perform a test. SetUp (), run at the end of the test. Teardown () (regardless of whether the test was successful, failed, or error). In our example above, we do a little bookkeeping for a dedicated Expat_kwargs dictionary to ensure that each test runs independently.

Incidentally, the difference between failure (failure) and errors (error) is important. A test may fail because some specific assertion is invalid (the assertion method either starts with ". Fail" or begins with ". Assert"). In a sense, failure is the expectation-at least in a sense we have analyzed it in detail. On the other hand, errors are an unexpected problem--because we don't know where to go wrong beforehand, we need to analyze the backtracking in the actual test run to diagnose the problem. However, we can design a hint that fails to diagnose errors. For example, if Prerequisite.havefiles () fails, an error occurs in some Testexpat tests, and if the former is successful, you will have to go somewhere else to find the source of the error.

In UnitTest. In the TestCase inheritance class, some of the specific test methods may be included. Assert ... () or ... fail. () method, but it may only have a series of actions that we believe should be executed successfully. If the test method does not work as expected, we will get an error (and a backtracking that describes the error).

The _MAIN_ program block in test_expat.py is also worth seeing. In the simplest case, we can run the test case using only Unittest.main (), which will determine what needs to be run. When you use this approach, the UnitTest module will accept a-v option to give more detailed output. Based on the specified file name, after the namespace setting is executed, we print out the representation of the specified XML file, thus maintaining a general backward compatibility of the older version of the tool.

The most interesting branch of _main_ is the one that expects the-Q or--quiet tag. As you will expect, unless a failure or error occurs, this branch will be silent (quiet, i.e. minimizing output). Not only that, because it is silent, it will only display a row of reports about the failure/error location for each problem, rather than the entire diagnostic backtracking. In addition to the direct use of the silent output style, this branch provides examples of custom testing relative to the test suite and control of the results report. A little bit of long unittest. The default output for Texttestrunner () is directed to stringio out--if you want to see it, you are welcome to Out.getvalue () to find it. However, the result object allows us to test the overall success, if not completely successful, and let us deal with failures and errors. Obviously, because they are values in a variable, you can easily record the contents of a result object into a log, or display it in a GUI, no matter what, not just print to STDOUT.

Combination Test

Perhaps the best feature of the UnitTest framework is that you can easily combine tests that contain different modules. In fact, if you use the Python 2.3+, you can even convert the doctest test into a unittest suite. Let's combine the tests we've created so far into a script test_all.py (admittedly, it's a bit exaggerated by the tests we've done so far):

Listing 5. test_all.py Combination of unit tests

"Combine Tests for Gnosis.xml.objectify package (req 2.3+)"
import UnitTest, doctest, Test_basic, Test_expat
Suite = Doctest. Doctestsuite (Test_basic)
suite.addtest (Unittest.makesuite (Test_expat). Prerequisite))
suite.addtest (Unittest.makesuite) (Test_expat. expattest))
 unittest. Texttestrunner (verbosity=2). Run (Suite)

Because test_expat.py only contains test classes, they can be easily added to a local test suite. Doctest. The Doctestsuite () function performs a transformation of the document string test. Let's see what test_all.py does when it's running:

Listing 6. Successful output from test_all.py

$ python2.3 test_all.py
doctest of test_basic.show. OK
Check for sample XML files, NS and basic ... ok
impo RT the Gnosis.xml.objectify library ... ok
Raise syntaxerror on Non-setup namespace xml ... ok
sucessfully objecti FY NS after Setup ... ok
Objectify namespace-free XML Document ... ok
successfully Objectify BASIC despite extra s Etup. OK
----------------------------------------------------------------------
Ran 7 tests in 0.052s
OK

Note the description of the tests performed: In the case of using the UnitTest test method, their description comes from the corresponding docstring function. If you do not specify a document string, the class and method names are used as the most appropriate description. It's also interesting to see what we get when some tests fail (the backtracking details are removed for this article):

Listing 7. Results when some tests fail

$ mv testns.xml testns.xml# && python2.3 test_all.py 2>&1 | Head-7
doctest of test_basic.show. OK
Check for sample XML files, NS and basic ... FAIL
Import The gnosis.xml.objectify library ... ok
Raise syntaxerror on Non-setup namespace XML ... ERROR
sucessfully objectify NS after Setup ... ERROR
Objectify namespace-free XML Document ... ok
successfully Objectify BASIC despite extra setup ... ok

Casually mentioned, this failure to write to the last line of STDERR is "FAILED (Failures=1, errors=2)", if you need it is a good summary (relative to the success of the final "OK").

Start from here

This article introduces you to some of the typical uses of UnitTest and doctest, which have improved the tests in my own software. Read the Python documentation to learn more about the methods available for the full range of test suites, test cases, and test results. All of them follow the pattern described in the example.

The methodology that allows you to comply with Python's standard test modules is good software practice. The development of test drivers (Test-driven) is popular in many software cycles, but it is clear that Python is a language suitable for test-driven models. Furthermore, if you are only considering that a package is more likely to work as planned, a package or library with a comprehensive set of tests will be more useful to users than a package or library that lacks these tests.