Python logging module

Many programmers have the need to log, and the log contains information that has normal program access logs, there may be errors, warnings and other information output, Python logging module provides a standard log interface, you can store a variety of logs through it, you can store various forms of log , the logging log can be divided into five levels: Debug (), info (), warning (), error (), and Critical ().

By default, Python's logging module prints the logs to standard output and only displays logs that are greater than or equal to the warning level, indicating that the default logging level is set to warning (log level level critical > ERROR > WARNING > INFO > DEBUG), the default log format is log level: Logger name: User output message.

One, logging module introduction

Logging module is a python built-in standard module, mainly used for output log, you can set the level of output logs, log save path, log file rollback, etc., compared to print, has the following advantages:

1. You can only output important information in release version by setting different log levels, without having to display a lot of debugging information;
2. Print prints all the information to the standard output, which seriously affects the developer viewing other data from the standard output, and logging can be used by the developer to decide where to export the information and how to output it;

Two, basic usage

Import Logging  logging.debug (' Debug message ')  logging.info (' info message ')  logging.warning (' Warning Message ')  logging.error (' error message ')  logging.critical (' critical message ')  

Output Result:

WARNING:root:warning messageERROR:root:error messageCRITICAL:root:critical Message

Visible, by default, Python's logging module prints the logs to standard output and only displays logs that are greater than or equal to the warning level, indicating that the default logging level is set to warning (log level level critical > ERROR > WARNING > INFO > DEBUG > NOTSET), the default log format is log level: Logger name: User output message.

Three, flexible configuration log level, log format, output location

Import Logging  logging.basicconfig (level=logging. DEBUG,                      format= '% (asctime) s% (filename) s[line:% (Lineno) d]                                 % (levelname) s% (message) s ',                      datefmt= '%a,%d b%Y%h:%m:%s ',                      filename= ' Test.log ',                      filemode= ' W ')    logging.debug (' Debug message ')  logging.info (' Info message ') logging.warning (' warning message ') logging.error ('  error message ')  logging.critical (' Critical message ')            

To view the output in Test.log:

Wed, Mar 2018 09:39:09 log module. py[line:24] Debug Debug messagewed, Mar 2018 09:39:09 log module. py[line:25] Info Info Messa gewed, Mar 2018 09:39:09 log module. py[line:26] WARNING WARNING messagewed, Mar 2018 09:39:09 log module. py[line:27] ERROR Err or messagewed, Mar 2018 09:39:09 log module. py[line:28] CRITICAL CRITICAL message

The Logging.basicconfig () function can be seen to change the default behavior of the logging module through specific parameters, with the parameters available

FileName: Creates a filedhandler with the specified file name (the concept of handler is explained in the back) so that the                log is stored in the specified file. FileMode: File is opened by using this parameter when filename is specified, and the default value is "a" and can be specified as "W". Format: Specifies the log display format used by handler. DATEFMT: Specifies the date time format. Level: Set Rootlogger (The following explains the specific concept) stream: Create Streamhandler with the specified stream. You can specify the output to Sys.stderr,sys.stdout            or the file (F=open (' Test.log ', ' W ')), and the default is Sys.stderr.            If you list both the filename and stream two parameters, the stream parameter is ignored. Formatted string that may be used in the format parameter:% (name) s logger name% (Levelno) s number form log level% (levelname) s text form log level% (pathname) s The full pathname of the module that calls the log output function, The file name of the module that may not have% (filename) s call log output function% (module) s call log output function module name% (funcName) s call log output function name% (Lineno) d Call Log output function The statement that is located in the line% ( Created) F current time, the current time in the number of milliseconds (asctime) s string as a percentage of the time that the UNIX standard represents a floating-point (relativecreated) d output log information, since logger was created. The default format is "2003-07-08 16:49:45,896". The comma is followed by the millisecond% (thread) d thread ID. There may not be a% (threadname) s thread name. There may not be a% (process) d process ID. Messages that may not have a% (message) s user output    

Four, Logger object

In the above examples we have learned about Logging.debug (), Logging.info (), logging.warning (), Logging.error (), logging.critical () (used to record different levels of log information), Logging.basicconfig () (using the default log format (Formatter) to establish a default stream processor (Streamhandler) for the log system, Set the base configuration (such as log level, etc.) and add to root logger (root logger) These several logging module-level functions, plus a module-level function is Logging.getlogger ([name]) (return a Logger object, If no name is specified will return root logger)

Let's look at one of the simplest processes:

# _*_ coding:utf-8 _*_ Import Logginglogger = Logging.getlogger () # Create a handler for writing to the log file, the file output object fh = logging. Filehandler (' Test1.log ') # Creates a handler for output to the console, and the screen output object ch = logging. The Streamhandler () #format is the output format formatter = logging. Formatter ('% (asctime) s-% (name) s-% (levelname) s-% (message) s ') Fh.setformatter (Formatter) ch.setformatter (Formatter ) Logger.addhandler (FH) #logger对象可以添加多个fh和ch对象logger. AddHandler (CH) logger.debug (' Logger debug message ') Logger.info (' Logger info message ') logger.warning (' Logger warning message ') logger.error (' Logger error message ') logger.critical (' Logger critical message ')

Results:

2018-03-14 10:19:39,478-root-warning-logger WARNING message2018-03-14 10:19:39,478-root-error-logger ERROR mes SAGE2018-03-14 10:19:39,478-root-critical-logger CRITICAL Message

4.1 Writing a log to a file

Set logging, create a filehandler, set the format of the output message, add it to the logger, and then write the log to the specified file.

Import Logginglogger = Logging.getlogger (__name__) logger.setlevel (level = logging.info) handler = logging. Filehandler ("Log.txt") Handler.setlevel (logging.info) formatter = logging. Formatter ('% (asctime) s-% (name) s-% (levelname) s-% (message) s ') Handler.setformatter (Formatter) Logger.addhandler ( Handler) Logger.info ("Start print Log") Logger.debug ("Do Something") logger.warning ("Something maybe fail.") Logger.info ("Finish")

The log output data in Log.txt is:

2018-03-14 22:47:14,497-__main__-Info-start print log2018-03-14 22:47:14,523-__main__-warning-something maybe Fail.2018-03-14 22:47:14,523-__main__-Info-finish

　　

Five, log output to screen and file at the same time

If you want to print log to the screen and the file log at the same time, you need to know a little bit of complicated knowledge.

Python uses the logging module to log logs involving the main passing classes, which are most appropriate to use in the official documentation:

• Logger provides an interface that the application can use directly
• Handle send (logger created) log records to the appropriate destination output
• Filter provides the details of the device to determine which log record to output
• Formatter determining the final output format of the log record

The relationship between them is as follows:

Logger

Each program before outputting the information to obtain a logger,logger usually corresponds to the program's module name, such as the chat tool's graphical interface module can get its logger:

LOG = Logging.getlogger ("Chat,gui")

And the core module can do this:

Log=logging.getlogger ("Chat.kernel")

You can also tie handle and filters.

Logger.setlevel (LEL): Specifies the lowest log level, and the level below LEL is ignored.                            Debug is the lowest built-in level, critical up to Logger.addfilter (filt), Logger.removefilter (filt): Add or remove specified Filterlogger.addhandler (HDLR), Logger.removehandler (HDLR): Adds or deletes the specified handler

Logger.debug (), Logger.info (), logger.warning (), Logger.error (), logger.critical (): Log levels you can set

Handler

Handler object is responsible for sending relevant information to the specified destination, Python's log system has a variety of handler can be used, some handler can output information to the console, some handler can export information to the file, There are some handler can send information to the network, if it is not enough, you can also write your own handler, you can add multiple handler through the AddHandler () method

Handler.setlevel (LEL): Specifies the level of information being processed, information below lel level is ignored Handler.setformatter (): Select a format for this Handler handler.addfilter (filt), Handler.removefilter (filt): Add or remove a filter object

Multiple handler can be attached to each logger. Next, let's introduce some common handler:

1,logging. Streamhandler uses this handler to output information to any file object, such as Sys.stdout or Sys.stderr.

2,logging. Filehandler and Streamhandler are similar for outputting log information to a file. But Filehandler will open this file for you.

3,logging.handlers.rotatingfilehandler

This handler is similar to the filehandler above, but it can manage the file size. When the file reaches a certain size, it automatically renames the current log file and then creates a new log file with the same name to continue the output. For example, the log file is Chat.log. When the chat.log reaches the specified size, Rotatingfilehandler automatically renames the file to Chat.log.1. However, if chat.log.1 already exists, it will first rename chat.log.1 to chat.log.2 ... Finally, re-create the Chat.log and continue to output the log information. Its functions are:

Rotatingfilehandler (filename[, mode[, maxbytes[, Backupcount]])

　　

where filename and mode two parameters are the same as Filehandler.

• The maxbytes is used to specify the maximum file size for the log file. If MaxBytes is 0, it means that the log file can be infinitely large, and the renaming process described above does not occur.
• The backupcount is used to specify the number of reserved backup files. For example, if you specify 2, when the renaming process described above occurs, the original chat.log.2 is not renamed, but is deleted.

4,logging.handlers.timedrotatingfilehandler

This handler is similar to Rotatingfilehandler, however, it does not determine when to recreate the log file by judging the size of the file, but instead automatically creates a new log file at a certain time interval. The process of renaming is similar to Rotatingfilehandler, but the new file is not an appended number but the current time. Its functions are:

Timedrotatingfilehandler (filename [, when [, interval [, Backupcount]])

where the filename parameter and the Backupcount parameter and the Rotatingfilehandler have the same meaning.

Interval is the time interval

The When parameter is a string that represents the unit of time interval, is case-insensitive, and it has the following values:

s seconds m min h hours d days W per week (Interval==0 on behalf of Monday) midnight every morning

　　

Formatter components

The formatter of the log is a separate component that can be combined with the root handler

FH = logging. Filehandler ("Access.log") formatter = logging. Formatter ('% (asctime) s-% (name) s-% (levelname) s-% (message) s ') Fh.setformatter (Formatter) #把formmater绑定到fh上

　　

Filter component

If you filter the contents of the log file, you can customize a filter

Class Ignorebackuplogfilter (logging. Filter): "" "    ignores log with DB Backup" "    def filter (self, record): #固定写法        return   " DB backup "not in Record.getmessage ()

Note that the filter function returns TRUE or False,logger determines whether this log is output based on this value

Then add this filter to the logger

Logger.addfilter (Ignorebackuplogfilter ())

The following log will filter out filters that match the filter condition

Logger.debug ("Test ...") logger.info ("Test info ...") logger.warning ("Start to run DB Backup job ....") Logger.error (" Test error .... ")

VI. Application Example 6.1 a complete example of output to screen, file, with filter

Import Loggingclass ignorebackuplogfilter (logging. Filter): "" "    ignores log with DB Backup" "    def filter (self, record): #固定写法        return   " DB backup "not in Record.getmessage () #console Handlerch = logging. Streamhandler () ch.setlevel (logging.info) #file HANDLERFH = logging. Filehandler (' Mysql.log ') #fh. SetLevel (logging. WARNING) #formatterformatter = logging. Formatter ('% (asctime) s-% (name) s-% (levelname) s-% (message) s ') #bind Formatter to Chch.setformatter (Formatter) Fh.setformatter (formatter) logger = Logging.getlogger ("Mysql") logger.setlevel (logging. DEBUG) #logger #add handler to   logger instancelogger.addhandler (CH) logger.addhandler (FH) with priority higher than other output channels #add Filterlogger.addfilter (Ignorebackuplogfilter ()) logger.debug ("Test ....") logger.info ("Test info ....") Logger.warning ("Start to run DB Backup job .....") Logger.error ("Test error ....")

　

6.2 File Automatic Truncation example

Import loggingfrom Logging Import Handlerslogger = Logging.getlogger (__name__) log_file = "Timelog.log" #fh = handlers. Rotatingfilehandler (filename=log_file,maxbytes=10,backupcount=3) FH = handlers. Timedrotatingfilehandler (filename=log_file,when= "S", interval=5,backupcount=3) formatter = logging. Formatter ('% (asctime) s% (module) s:% (Lineno) d% (message) s ') Fh.setformatter (Formatter) Logger.addhandler (FH) Logger.warning ("Test1") logger.warning ("test12") logger.warning ("test13") logger.warning ("test14")

　　

6.3 Examples of log files applied to ATM programs

Import osimport timeimport loggingfrom config import settingsdef get_logger (Card_num, struct_time):    if struct_ Time.tm_mday <:        file_name = "%s_%s_%d"% (Struct_time.tm_year, Struct_time.tm_mon, $)    else:        file_ Name = "%s_%s_%d"% (Struct_time.tm_year, struct_time.tm_mon+1,)    File_handler = logging. Filehandler (        os.path.join (settings. User_dir_folder, Card_num, ' record ', file_name),        encoding= ' utf-8 '    )    FMT = logging. Formatter (fmt= "% (asctime) s:  % (message) s")    File_handler.setformatter (FMT)    logger1 = logging. Logger (' User_logger ', level=logging.info)    Logger1.addhandler (File_handler)    return Logger1

　　

Python logging module