Summary of several methods for Python code debugging

This article mainly introduces several methods to debug Python code. This article comes from the technical documentation on the IBM official website. For more information, see Use pdb for debugging

Pdb is a python package that provides an interactive source code debugging function for python programs, main features include breakpoint setting, single-step debugging, function debugging, viewing current code, viewing stack fragments, and dynamically changing variable values. Pdb provides some common debugging commands. for details, see Table 1.
Table 1. common pdb commands

The following describes how to use pdb for debugging based on a specific instance.
Listing 1. test sample code

import pdb  a = "aaa" pdb.set_trace()  b = "bbb" c = "ccc" final = a + b + c  print final

Start Debugging: directly run the script and it will stay at the pdb. set_trace (). Select n + enter to execute the current statement. After you press n + enter for the first time, you can press enter to execute the previous debug command again.
List 2. debug with pdb

[root@rcc-pok-idg-2255 ~]# python epdb1.py  > /root/epdb1.py(4)?()  -> b = "bbb" (Pdb) n  > /root/epdb1.py(5)?()  -> c = "ccc" (Pdb)  > /root/epdb1.py(6)?()  -> final = a + b + c  (Pdb) list  1   import pdb  2   a = "aaa" 3   pdb.set_trace()  4   b = "bbb" 5   c = "ccc" 6 -> final = a + b + c  7   print final  [EOF]  (Pdb)  [EOF]  (Pdb) n  > /root/epdb1.py(7)?()  -> print final  (Pdb)

Exit debug: quit or q can be used to exit the current debug, but quit will exit the program in a very rude way, and the result is a direct crash.
Listing 3. exit debug

[root@rcc-pok-idg-2255 ~]# python epdb1.py  > /root/epdb1.py(4)?()  -> b = "bbb" (Pdb) n  > /root/epdb1.py(5)?()  -> c = "ccc" (Pdb) q  Traceback (most recent call last):  File "epdb1.py", line 5, in ?   c = "ccc" File "epdb1.py", line 5, in ?   c = "ccc" File "/usr/lib64/python2.4/bdb.py", line 48, in trace_dispatch   return self.dispatch_line(frame)  File "/usr/lib64/python2.4/bdb.py", line 67, in dispatch_line   if self.quitting: raise BdbQuit  bdb.BdbQuit

Print the value of a variable: to print the value of a variable during debugging, you can directly add the variable name with p, however, it must be noted that only after the current statement has been executed can you see the specific value. otherwise, NameError: <exceptions. nameError... ...> Error.
Listing 4. printing variables during debug

[root@rcc-pok-idg-2255 ~]# python epdb1.py  > /root/epdb1.py(4)?()  -> b = "bbb" (Pdb) n  > /root/epdb1.py(5)?()  -> c = "ccc" (Pdb) p b 'bbb' (Pdb) 'bbb' (Pdb) n  > /root/epdb1.py(6)?()  -> final = a + b + c  (Pdb) p c 'ccc' (Pdb) p final  *** NameError: 
 
    (Pdb) n  > /root/epdb1.py(7)?()  -> print final  (Pdb) p final 'aaabbbccc' (Pdb)
 

Use c to stop the current debug program so that the program can continue to run. If the set_statement () statement is continued in the following program, it enters the debug state again. you can add set_trace () verification before the code print final.
Listing 5. stop debug and continue executing the program

[root@rcc-pok-idg-2255 ~]# python epdb1.py  > /root/epdb1.py(4)?()  -> b = "bbb" (Pdb) n  > /root/epdb1.py(5)?()  -> c = "ccc" (Pdb) c  aaabbbccc

Display code: the current code block may not be remembered during debugging. to view a specific code block, you can use the list or l command to display it. List uses the arrow-> to point to the current debug statement.
Listing 6. displaying code during debug

[root@rcc-pok-idg-2255 ~]# python epdb1.py  > /root/epdb1.py(4)?()  -> b = "bbb" (Pdb) list  1   import pdb  2   a = "aaa" 3   pdb.set_trace()  4 -> b = "bbb" 5   c = "ccc" 6   final = a + b + c  7   pdb.set_trace()  8   print final  [EOF]  (Pdb) c  > /root/epdb1.py(8)?()  -> print final  (Pdb) list  3   pdb.set_trace()  4   b = "bbb" 5   c = "ccc" 6   final = a + b + c  7   pdb.set_trace()  8 -> print final  [EOF]  (Pdb)

Debug a function
Listing 7. examples of using functions

import pdb  def combine(s1,s2):   # define subroutine combine, which...   s3 = s1 + s2 + s1  # sandwiches s2 between copies of s1, ...   s3 = '"' + s3 +'"'  # encloses it in double quotes,...   return s3      # and returns it.  a = "aaa" pdb.set_trace()  b = "bbb" c = "ccc" final = combine(a,b)  print final

If you use n for debug directly, the final = combine (a, B) statement will be treated as a normal value assignment statement and enter print final. What if I want to debug the function? You can directly use s to enter the function block. The single-step debugging in the function is similar to the previous introduction. If you do not want to perform one-step debugging in the function, you can directly press r at the breakpoint to exit the called place.
Listing 8. debug functions

[root@rcc-pok-idg-2255 ~]# python epdb2.py  > /root/epdb2.py(10)?()  -> b = "bbb" (Pdb) n  > /root/epdb2.py(11)?()  -> c = "ccc" (Pdb) n  > /root/epdb2.py(12)?()  -> final = combine(a,b)  (Pdb) s  --Call--  > /root/epdb2.py(3)combine()  -> def combine(s1,s2):   # define subroutine combine, which...  (Pdb) n  > /root/epdb2.py(4)combine()  -> s3 = s1 + s2 + s1  # sandwiches s2 between copies of s1, ...  (Pdb) list  1   import pdb  2  3   def combine(s1,s2):   # define subroutine combine, which...  4 ->   s3 = s1 + s2 + s1  # sandwiches s2 between copies of s1, ...  5     s3 = '"' + s3 +'"'  # encloses it in double quotes,...  6     return s3      # and returns it.  7  8   a = "aaa" 9   pdb.set_trace()  10   b = "bbb" 11   c = "ccc" (Pdb) n  > /root/epdb2.py(5)combine()  -> s3 = '"' + s3 +'"'  # encloses it in double quotes,...  (Pdb) n  > /root/epdb2.py(6)combine()  -> return s3      # and returns it.  (Pdb) n  --Return--  > /root/epdb2.py(6)combine()->'"aaabbbaaa"' -> return s3      # and returns it.  (Pdb) n  > /root/epdb2.py(13)?()  -> print final  (Pdb)

The value is dynamically changed during debugging. You can dynamically change the variable value during debugging, as shown in the following example. Note that the following error occurs because B has been assigned a value. if you want to change the value of B again, use it! B.
Listing 9. dynamically changing the value during debugging

[root@rcc-pok-idg-2255 ~]# python epdb2.py  > /root/epdb2.py(10)?()  -> b = "bbb" (Pdb) var = "1234" (Pdb) b = "avfe" *** The specified object '= "avfe"' is not a function  or was not found along sys.path.  (Pdb) !b="afdfd" (Pdb)

One obvious drawback of pdb debugging is that it does not support multithreading, remote debugging, and other operations well. it does not have a more intuitive interface display and is not suitable for large-scale python projects. In a large python project, these debugging requirements are common, so you need to use more advanced debugging tools. Next we will introduce the debugging method of PyCharm IDE.
Use PyCharm for debugging

PyCharm is a Python IDE created by JetBrains. it has functions such as syntax highlighting, Project management, code redirection, smart prompts, automatic completion, unit testing, and version control, it also provides support for Django development and Google App Engine. It can be divided into personal and commercial editions, and requires license support. you can also get a free 30-day trial. You can download the trial version of Pycharm from the official website: http://www.jetbrains.com/pycharm/download/index.html. PyCharm also provides comprehensive debugging functions, including multi-threading and remote debugging. it supports breakpoint settings, single-step mode, expression evaluation, and variable viewing. The debugging window layout of PyCharm IDE is shown in 1.
Figure 1. PyCharm IDE window layout

The following example shows how to use PyCharm for multi-thread debugging. For the code examples used for debugging, see list 10.
Listing 10. PyCharm debugging code example

__author__ = 'zhangying' #!/usr/bin/python  import thread  import time  # Define a function for the thread  def print_time( threadName, delay):  count = 0  while count < 5:   count += 1   print "%s: %s" % ( threadName, time.ctime(time.time()) )  def check_sum(threadName,valueA,valueB):  print "to calculate the sum of two number her" result=sum(valueA,valueB)  print "the result is" ,result;  def sum(valueA,valueB):  if valueA >0 and valueB>0:   return valueA+valueB  def readFile(threadName, filename):  file = open(filename)  for line in file.xreadlines():   print line  try:  thread.start_new_thread( print_time, ("Thread-1", 2, ) )  thread.start_new_thread( check_sum, ("Thread-2", 4,5, ) )  thread.start_new_thread( readFile, ("Thread-3","test.txt",))  except:  print "Error: unable to start thread" while 1:  # print "end" pass

You usually need to set a breakpoint before debugging. a breakpoint can be set at a loop or conditional expression or a key point of the program. The method for setting breakpoints is very simple: in the code editing box, move the cursor to the row where the breakpoint needs to be set, press Ctrl + F8 or select the menu "Run"-> "Toggle Line Break Point". the more direct way is to double-click the left edge of the code editing area, the red dot (2) is displayed ). When debugging starts, the code being executed is displayed in blue. Three breakpoints are set, and the code being executed is highlighted in blue.
Figure 2. breakpoint settings

Expression evaluation: some expressions need to be tracked during debugging to find problems in the program. Pycharm supports expression evaluation. you can select this expression, select "Run"-> "Evaluate Expression" and select "Evaluate" in the displayed window.

Pychar provides both the Variables and Watches windows. the values of specific Variables involved in the debugging steps can be viewed directly in the variable column.
Figure 3. View variables

If you want to dynamically monitor a variable, you can directly select the variable and choose "Run"> "Add Watch" from the menu to Add it to the watches bar. When debugging the statement where the variable is located, you can view the specific value of the variable in this window.
Figure 4. monitoring variables

For multi-threaded programs, there are usually multiple threads. when the breakpoint of debug needs to be set in the corresponding thread bodies of different threads, IDE usually needs to support good multi-threaded Debugging functions. In Pycharm, a virtual thread with the name starting with Dummy is automatically generated when the main thread starts the startup thread. each frame corresponds to its own debugging frame. 5. There are four threads in this instance. the main thread generates three threads: Dummy-4, Dummy-5, and Dummy-6. dummy-4 corresponds to thread 1, while others correspond to thread 2 and thread 3 respectively.
Figure 5. multithreading window

When debugging enters the subprograms of various threads, the Frame will automatically switch to its corresponding frame, and the corresponding variable column will also display the corresponding variables for this process, 6, directly control the debugging button, such as setp in. step over can facilitate debugging.
Figure 6. subthread debugging

Use PyDev for debugging

PyDev is an open-source plugin that can be easily integrated with Eclipse and provide convenient and powerful debugging functions. As an excellent Python IDE, it also provides powerful functions such as syntax error prompt, source code editing assistant, Quick Outline, Globals Browser, Hierarchy View, and running. The following describes how to integrate PyDev with Eclipse. Before installing PyDev, you must first install Java 1.4 or later, Eclipse, and Python. Step 1: Start Eclipse, find the Help bar in the Eclipse menu bar, select Help> Install New Software, and select Add button to Add the Ptdev download site http://pydev.org/updates. Select PyDev and complete the remaining steps to install PyDev.
Figure 7. install PyDev

After installation, configure the Python Interpreter. on the Eclipse menu bar, choose Window> Preferences> Pydev> Interpreter-Python. Python is installed in the C: \ Python27 path. Click New and select Python interpreter python.exe. a window containing multiple check boxes is displayed. select the path to be added to the system PYTHONPATH and click OK.
Figure 8. configure PyDev

After configuring Pydev, you can choose File> New> Project> Pydev Project from the Eclipse menu bar and click Next to create a Python Project. the following content assumes that the python Project has been created, there is a script remote to be debugged. py (the specific content is as follows) is a script for logging on to a remote machine to execute some commands. some parameters need to be input during running, the following describes how to input parameters during debugging.
Listing 11. sample code for Pydev debugging

 #!/usr/bin/env python  import os  def telnetdo(HOST=None, USER=None, PASS=None, COMMAND=None): #define a function  import telnetlib, sys  if not HOST:  try:  HOST = sys.argv[1]  USER = sys.argv[2]  PASS = sys.argv[3]  COMMAND = sys.argv[4]  except:  print "Usage: remote.py host user pass command" return  tn = telnetlib.Telnet() #  try:  tn.open(HOST)  except:  print "Cannot open host" return  tn.read_until("login:")  tn.write(USER + '\n')  if PASS:  tn.read_until("Password:")  tn.write(PASS + '\n')  tn.write(COMMAND + '\n')  tn.write("exit\n")  tmp = tn.read_all()  tn.close()  del tn  return tmp   if __name__ == '__main__':  print telnetdo()

Some parameters need to be input during debugging. before debugging, you need to configure the parameters to receive the required parameters and select the program to be debugged (in this example, remote. in the debug process, enter four parameters: host, user, password, and command. In the eclipse Project Directory, select the program to debug, right-click, select "Debug As"-> "Debug deployments", and select "Variables" on the Arguments Tab ". See Figure 9.
Figure 9. configure variables

Select "Select Variable" in the window and then Select "Edit Varuables". The following window is displayed. Select "New" in the window and enter the corresponding Variable name and value in the window that appears. Note that there must be spaces behind the value, otherwise all parameters will be read as the first parameter.
Figure 10. add specific variables

Configure all the parameters in sequence according to the preceding method, and then select the corresponding variables in the order required to install the parameters in the select variable window. The status after configuration is complete is shown in Figure 11.
Figure 11. Complete configuration

Select Debug to Start program debugging. the debugging method is similar to the built-in debugging function of eclipse and supports multi-thread debugging. There are many articles in this regard, you can search for and read it on your own, or refer to the article "debugging using the Eclipse platform.
Use the log function for debugging

Log information is a very useful method for debugging during software development, especially when many related personnel need to collaborate in large-scale software development. By adding some specific event information in the code that can record the software running process, developers can identify problems in the code. This information may include the time, description, and specific context information when an error or exception occurs. The original debug method is to embed the print statement in the code and output relevant information to locate program problems. However, this method has some defects. the normal program output and debug information are mixed together, which makes the analysis difficult. when debugging ends, debugging output is no longer needed, there is usually no simple way to block or locate the print information. The logging module in python can easily solve these problems. It provides the log function, which divides the logger level into five levels and can be set through Logger. setLevel (lvl. The default level is warning.
Table 2. log level

Ogging lib contains four main objects

1. Logger: logger is the interface for program information output. It is scattered in different codes so that the program can record the corresponding information during running, based on the configured log level or filter, determine which information needs to be output and distribute the information to the associated handler. Common methods include Logger. setLevel (), Logger. addHandler (), Logger. removeHandler (), Logger. addFilter (), Logger. debug (), Logger.info (), Logger. warning (), Logger. error (), getLogger (), etc. Logger supports hierarchical Inheritance. the name of the sub-logger is usually the parent logger. name. If you do not create a logger instance, use the default root logger to get the root logger instance through logging. getLogger () or logging. getLogger.
2. Handler: Handler is used to process information output. it can output information to the console, file, or network. You can use Logger. addHandler () to add handler to a logger object. commonly used handler classes include StreamHandler and FileHandler. StreamHandler sends error messages to the stream, while the FileHandler class is used to output log information to the file, which are defined in the core modules of logging. Other handers are defined in the logging. handles module, such as HTTPHandler and SocketHandler.
3. Formatter: Formatter determines the format of log information, which is defined in a format similar to % (<dictionary key>) s, for example, '% (asctime) s-% (levelname) s-% (message) S'. the supported keys can be viewed in the LogRecord attributes file provided by python.
4. Filter: used to determine which information needs to be output. It can be used by handler and logger and supports hierarchical relationships. for example, if A filter is set, it is called. b's logger, the information of the logger and its sub-logger will be output, such as. b,. B .C.

List 12. sample log usage

Import logging
LOG1 = logging. getLogger ('B. c ')
LOG2 = logging. getLogger ('D. e ')
Filehandler = logging. FileHandler ('test. log', 'A ')
Formatter = logging. Formatter ('% (name) s % (asctime) s % (levelname) s % (message) s ')
Filehandler. setFormatter (formatter)
Filter = logging. Filter ('B ')
Filehandler. addFilter (filter)
LOG1.addHandler (filehandler)
LOG2.addHandler (filehandler)
LOG1.setLevel (logging. INFO)
LOG2.setLevel (logging. DEBUG)
LOG1.debug ('It is a debug info for log1 ')
LOG1.info ('normal infor log1 ')
LOG1.warning ('warning info for log1: B. c ')
LOG1.error ('error info for log1: ABCD ')
LOG1.critical ('critical info for log1: not worked ')
LOG2.debug ('debug info for log2 ')
LOG2.info ('normal info for log2 ')
LOG2.warning ('warning info for log2 ')
LOG2.error ('Error: B. c ')
LOG2.critical ('critical ')

If filter B is set in the above example, then B. c is the sub-logger of B, so the log information related to the logger that meets the filter condition will be output, while other logger that does not meet the filter condition (here is d. e) will be filtered out.
Listing 13. output results

B. c 2011-11-25 11:07:29, 733 INFO normal infor for log1
B. c 2011-11-25 11:07:29, 733 WARNING warning info for log1: B. c
B. c 2011-11-25 11:07:29, 733 ERROR error info for log1: abcd
B. c 2011-11-25 11:07:29, 733 CRITICAL critical info for log1: not worked

Logging is easy to use and thread-safe. the following uses a multi-thread example to describe how to debug logging.
Listing 14. using logging with multiple threads

logging.conf  [loggers]  keys=root,simpleExample  [handlers]  keys=consoleHandler  [formatters]  keys=simpleFormatter  [logger_root]  level=DEBUG  handlers=consoleHandler  [logger_simpleExample]  level=DEBUG  handlers=consoleHandler  qualname=simpleExample  propagate=0  [handler_consoleHandler]  class=StreamHandler  level=DEBUG  formatter=simpleFormatter  args=(sys.stdout,)  [formatter_simpleFormatter]  format=%(asctime)s - %(name)s - %(levelname)s - %(message)s  datefmt=  code example:  #!/usr/bin/python  import thread  import time  import logging  import logging.config  logging.config.fileConfig('logging.conf')  # create logger  logger = logging.getLogger('simpleExample')  # Define a function for the thread  def print_time( threadName, delay):  logger.debug('thread 1 call print_time function body')  count = 0  logger.debug('count:%s',count)

Summary

The full text introduces several different debug methods in python, including pdb module, debugging using PyDev and Eclipse Integration, PyCharm and Debug log, hoping to provide a reference for related python users.