Python logger模組,pythonlogger模組
1 logging模組簡介
logging模組是Python內建的標準模組,主要用於輸出作業記錄,可以設定輸出日誌的等級、日誌儲存路徑、記錄檔復原等;相比print,具備如下優點:
Logger從來不直接執行個體化,經常通過logging模組層級方法(Module-Level Function)logging.getLogger(name)來獲得,其中如果name不給定就用root。名字是以點號分割的命名方式命名的(a.b.c)。對同一個名字的多個調用logging.getLogger()方法會返回同一個logger對象。這種命名方式裡面,後面的loggers是前面logger的子logger,自動繼承父loggers的log資訊,正因為此,沒有必要把一個應用的所有logger都配置一遍,只要把頂層的logger配置好了,然後子logger根據需要繼承就行了。 logging.Logger對象扮演了三重角色: 首先,它暴露給應用幾個方法以便應用可以在運行時寫log. 其次,Logger對象按照log資訊的嚴重程度或者根據filter對象來決定如何處理log資訊(預設的過濾功能). 最後,logger還負責把log資訊傳送給相關的handlers.2 logging模組使用2.1 基本使用
配置logging基本的設定,然後在控制台輸出日誌,
import logginglogging.basicConfig(level = logging.INFO,format = '%(asctime)s - %(name)s - %(levelname)s - %(message)s')logger = logging.getLogger(__name__)logger.info("Start print log")logger.debug("Do something")logger.warning("Something maybe fail.")logger.info("Finish")
運行時,控制台輸出,
1 2016-10-09 19:11:19,434 - __main__ - INFO - Start print log2 2016-10-09 19:11:19,434 - __main__ - WARNING - Something maybe fail.3 2016-10-09 19:11:19,434 - __main__ - INFO - Finish
logging中可以選擇很多訊息層級,如:DEBUG,INFO,WARNING,ERROR,CRITICAL,通過賦予logger或者handler不同的層級,開發人員就可以只輸出錯誤資訊到特定的記錄檔案,或者在調試時只記錄調試資訊。
將logger的層級改為DEBUG,再觀察一下輸出結果
logging.basicConfig(level = logging.DEBUG,format = '%(asctime)s - %(name)s - %(levelname)s - %(message)s')
從輸出結果可以看到,輸出了debug的日誌記錄
2016-10-09 19:12:08,289 - __main__ - INFO - Start print log2016-10-09 19:12:08,289 - __main__ - DEBUG - Do something2016-10-09 19:12:08,289 - __main__ - WARNING - Something maybe fail.2016-10-09 19:12:08,289 - __main__ - INFO - Finish
logging.basicConfig函數各參數:
filename:指定記錄檔名;filemode:和file函數意義相同,指定記錄檔的開啟模式,'w'或者'a';format:指定輸出的格式和內容,format可以輸出很多有用的資訊,datefmt:指定時間格式,同time.strftime();level:設定記錄層級,預設為logging.WARNNING;stream:指定將日誌的輸出資料流,可以指定輸出到sys.stderr,sys.stdout或者檔案,預設輸出到sys.stderr,當stream和filename同時指定時,stream被忽略;
Formatters定義了Logger記錄的輸出格式。 定義了最終log資訊的內容格式,應用可以直接執行個體化Foamatter類。資訊格式字串用%(<dictionary key>)s風格的字串做替換。
| 屬性名稱 |
格式 |
說明 |
| name |
%(name)s |
日誌的名稱 |
| asctime |
%(asctime)s |
可讀時間,預設格式‘2003-07-08 16:49:45,896’,逗號之後是毫秒 |
| filename |
%(filename)s |
檔案名稱,pathname的一部分 |
| pathname |
%(pathname)s |
檔案的全路徑名稱 |
| funcName |
%(funcName)s |
調用日誌多對應的方法名 |
| levelname |
%(levelname)s |
日誌的等級 |
| levelno |
%(levelno)s |
數字化的日誌等級 |
| lineno |
%(lineno)d |
被記錄日誌在源碼中的行數 |
| module |
%(module)s |
模組名 |
| msecs |
%(msecs)d |
時間中的毫秒部分 |
| process |
%(process)d |
進程的ID |
| processName |
%(processName)s |
進程的名稱 |
| thread |
%(thread)d |
線程的ID |
| threadName |
%(threadName)s |
線程的名稱 |
| relativeCreated |
%(relativeCreated)d |
日誌被建立的相對時間,以毫秒為單位 |
2.2 將日誌寫入到檔案
2.2.1 將日誌寫入到檔案
設定logging,建立一個FileHandler,並對輸出訊息的格式進行設定,將其添加到logger,然後將日誌寫入到指定的檔案中,
import logginglogger = logging.getLogger(__name__)logger.setLevel(level = logging.INFO)handler = logging.FileHandler("log.txt")handler.setLevel(logging.INFO)formatter = logging.Formatter('%(asctime)s - %(name)s - %(levelname)s - %(message)s')handler.setFormatter(formatter)logger.addHandler(handler)logger.info("Start print log")logger.debug("Do something")logger.warning("Something maybe fail.")logger.info("Finish")
log.txt中日誌資料為:
2017-07-25 15:02:09,905 - __main__ - INFO - Start print log
2017-07-25 15:02:09,905 - __main__ - WARNING - Something maybe fail.
2017-07-25 15:02:09,905 - __main__ - INFO - Finish
2.2.2 將日誌同時輸出到螢幕和記錄檔
logger中添加StreamHandler,可以將日誌輸出到螢幕上,
import logginglogger = logging.getLogger(__name__)logger.setLevel(level = logging.INFO)handler = logging.FileHandler("log.txt")handler.setLevel(logging.INFO)formatter = logging.Formatter('%(asctime)s - %(name)s - %(levelname)s - %(message)s')handler.setFormatter(formatter)console = logging.StreamHandler()console.setLevel(logging.INFO)logger.addHandler(handler)logger.addHandler(console)logger.info("Start print log")logger.debug("Do something")logger.warning("Something maybe fail.")logger.info("Finish")
可以在log.txt檔案和控制台中看到
2017-07-25 15:03:05,075 - __main__ - INFO - Start print log
2017-07-25 15:03:05,075 - __main__ - WARNING - Something maybe fail.
2017-07-25 15:03:05,075 - __main__ - INFO - Finish
可以發現,logging有一個Tlog的主對象,其他處理方式都是通過addHandler添加進去,logging中包含的handler主要有如下幾種,
handler名稱:位置;作用StreamHandler:logging.StreamHandler;日誌輸出到流,可以是sys.stderr,sys.stdout或者檔案FileHandler:logging.FileHandler;日誌輸出到檔案BaseRotatingHandler:logging.handlers.BaseRotatingHandler;基本的日誌復原方式RotatingHandler:logging.handlers.RotatingHandler;日誌復原方式,支援記錄檔最大數量和記錄檔復原TimeRotatingHandler:logging.handlers.TimeRotatingHandler;日誌復原方式,在一定時間地區內復原記錄檔SocketHandler:logging.handlers.SocketHandler;遠程輸出日誌到TCP/IP socketsDatagramHandler:logging.handlers.DatagramHandler;遠程輸出日誌到UDP socketsSMTPHandler:logging.handlers.SMTPHandler;遠程輸出日誌到郵件地址SysLogHandler:logging.handlers.SysLogHandler;日誌輸出到syslogNTEventLogHandler:logging.handlers.NTEventLogHandler;遠程輸出日誌到Windows NT/2000/XP的事件記錄MemoryHandler:logging.handlers.MemoryHandler;日誌輸出到記憶體中的指定bufferHTTPHandler:logging.handlers.HTTPHandler;通過"GET"或者"POST"遠程輸出到HTTP伺服器
2.2.3 日誌復原
使用RotatingFileHandler,可以實現日誌復原,
import loggingfrom logging.handlers import RotatingFileHandlerlogger = logging.getLogger(__name__)logger.setLevel(level = logging.INFO)#定義一個RotatingFileHandler,最多備份3個記錄檔,每個記錄檔最大1KrHandler = RotatingFileHandler("log.txt",maxBytes = 1*1024,backupCount = 3)rHandler.setLevel(logging.INFO)formatter = logging.Formatter('%(asctime)s - %(name)s - %(levelname)s - %(message)s')rHandler.setFormatter(formatter)console = logging.StreamHandler()console.setLevel(logging.INFO)console.setFormatter(formatter)logger.addHandler(rHandler)logger.addHandler(console)logger.info("Start print log")logger.debug("Do something")logger.warning("Something maybe fail.")logger.info("Finish")
可以在工程目錄中看到,備份的記錄檔,
.3 設定訊息的等級
可以設定不同的日誌等級,用於控制日誌的輸出
日誌等級:使用範圍FATAL:致命錯誤CRITICAL:特別糟糕的事情,如記憶體耗盡、磁碟空間為空白,一般很少使用ERROR:發生錯誤時,如IO操作失敗或者串連問題WARNING:發生很重要的事件,但是並不是錯誤時,如使用者登入密碼錯誤INFO:處理請求或者狀態變化等日常事務DEBUG:調試過程中使用DEBUG等級,如演算法中每個迴圈的中間狀態
setLevel(lvl) 定義處理log的最低等級,內建的層級為:DEBUG,INFO,WARNING,ERROR,CRITICAL;是層級對應數值
2.4 捕獲traceback
Python中的traceback模組被用於跟蹤異常返回資訊,可以在logging中記錄下traceback
import logginglogger = logging.getLogger(__name__)logger.setLevel(level = logging.INFO)handler = logging.FileHandler("log.txt")handler.setLevel(logging.INFO)formatter = logging.Formatter('%(asctime)s - %(name)s - %(levelname)s - %(message)s')handler.setFormatter(formatter)console = logging.StreamHandler()console.setLevel(logging.INFO)logger.addHandler(handler)logger.addHandler(console)logger.info("Start print log")logger.debug("Do something")logger.warning("Something maybe fail.")try: open("sklearn.txt","rb")except (SystemExit,KeyboardInterrupt): raiseexcept Exception: logger.error("Faild to open sklearn.txt from logger.error",exc_info = True)logger.info("Finish")
控制台和記錄檔log.txt中輸出
1 2017-07-25 15:04:24,045 - __main__ - INFO - Start print log2 2017-07-25 15:04:24,045 - __main__ - WARNING - Something maybe fail.3 2017-07-25 15:04:24,046 - __main__ - ERROR - Faild to open sklearn.txt from logger.error4 Traceback (most recent call last):5 File "E:\PYTHON\Eclipse\eclipse\Doc\14day5\Logger模組\Logging.py", line 71, in <module>6 open("sklearn.txt","rb")7 IOError: [Errno 2] No such file or directory: 'sklearn.txt'8 2017-07-25 15:04:24,049 - __main__ - INFO - FinishView Code
也可以使用logger.exception(msg,_args),它等價於logger.error(msg,exc_info = True,_args),
將logger.error("Faild to open sklearn.txt from logger.error",exc_info = True)替換為,logger.exception("Failed to open sklearn.txt from logger.exception")
2.5 多模組使用logging
主模組mainModule.py
import loggingimport subModulelogger = logging.getLogger("mainModule")logger.setLevel(level = logging.INFO)handler = logging.FileHandler("log.txt")handler.setLevel(logging.INFO)formatter = logging.Formatter('%(asctime)s - %(name)s - %(levelname)s - %(message)s')handler.setFormatter(formatter)console = logging.StreamHandler()console.setLevel(logging.INFO)console.setFormatter(formatter)logger.addHandler(handler)logger.addHandler(console)logger.info("creating an instance of subModule.subModuleClass")a = subModule.SubModuleClass()logger.info("calling subModule.subModuleClass.doSomething")a.doSomething()logger.info("done with subModule.subModuleClass.doSomething")logger.info("calling subModule.some_function")subModule.som_function()logger.info("done with subModule.some_function")
子模組subModule.py
import loggingmodule_logger = logging.getLogger("mainModule.sub")class SubModuleClass(object): def __init__(self): self.logger = logging.getLogger("mainModule.sub.module") self.logger.info("creating an instance in SubModuleClass") def doSomething(self): self.logger.info("do something in SubModule") a = [] a.append(1) self.logger.debug("list a = " + str(a)) self.logger.info("finish something in SubModuleClass")def som_function(): module_logger.info("call function some_function")
執行之後,在控制和記錄檔log.txt中輸出
1 2017-07-25 15:05:07,427 - mainModule - INFO - creating an instance of subModule.subModuleClass2 2017-07-25 15:05:07,427 - mainModule.sub.module - INFO - creating an instance in SubModuleClass3 2017-07-25 15:05:07,427 - mainModule - INFO - calling subModule.subModuleClass.doSomething4 2017-07-25 15:05:07,427 - mainModule.sub.module - INFO - do something in SubModule5 2017-07-25 15:05:07,427 - mainModule.sub.module - INFO - finish something in SubModuleClass6 2017-07-25 15:05:07,427 - mainModule - INFO - done with subModule.subModuleClass.doSomething7 2017-07-25 15:05:07,427 - mainModule - INFO - calling subModule.some_function8 2017-07-25 15:05:07,427 - mainModule.sub - INFO - call function some_function9 2017-07-25 15:05:07,428 - mainModule - INFO - done with subModule.some_function
View Code
說明:
首先在主模組定義了logger'mainModule',並對它進行了配置,就可以在解譯器進程裡面的其他地方通過getLogger('mainModule')得到的對象都是一樣的,不需要重新設定,可以直接使用。定義的該logger的子logger,都可以共用父logger的定義和配置,所謂的父子logger是通過命名來識別,任意以'mainModule'開頭的logger都是它的子logger,例如'mainModule.sub'。
實際開發一個application,首先可以通過logging設定檔編寫好這個application所對應的配置,可以產生一個根logger,如'PythonAPP',然後在主函數中通過fileConfig載入logging配置,接著在application的其他地方、不同的模組中,可以使用根logger的子logger,如'PythonAPP.Core','PythonAPP.Web'來進行log,而不需要反覆的定義和配置各個模組的logger。
3 通過JSON或者YAML檔案配置logging模組
儘管可以在Python代碼中配置logging,但是這樣並不夠靈活,最好的方法是使用一個設定檔來配置。在Python 2.7及以後的版本中,可以從字典中載入logging配置,也就意味著可以通過JSON或者YAML檔案載入日誌的配置。
3.1 通過JSON檔案配置
JSON設定檔
{ "version":1, "disable_existing_loggers":false, "formatters":{ "simple":{ "format":"%(asctime)s - %(name)s - %(levelname)s - %(message)s" } }, "handlers":{ "console":{ "class":"logging.StreamHandler", "level":"DEBUG", "formatter":"simple", "stream":"ext://sys.stdout" }, "info_file_handler":{ "class":"logging.handlers.RotatingFileHandler", "level":"INFO", "formatter":"simple", "filename":"info.log", "maxBytes":"10485760", "backupCount":20, "encoding":"utf8" }, "error_file_handler":{ "class":"logging.handlers.RotatingFileHandler", "level":"ERROR", "formatter":"simple", "filename":"errors.log", "maxBytes":10485760, "backupCount":20, "encoding":"utf8" } }, "loggers":{ "my_module":{ "level":"ERROR", "handlers":["info_file_handler"], "propagate":"no" } }, "root":{ "level":"INFO", "handlers":["console","info_file_handler","error_file_handler"] }}
通過JSON載入設定檔,然後通過logging.dictConfig配置logging,
import jsonimport logging.configimport osdef setup_logging(default_path = "logging.json",default_level = logging.INFO,env_key = "LOG_CFG"): path = default_path value = os.getenv(env_key,None) if value: path = value if os.path.exists(path): with open(path,"r") as f: config = json.load(f) logging.config.dictConfig(config) else: logging.basicConfig(level = default_level)def func(): logging.info("start func") logging.info("exec func") logging.info("end func")if __name__ == "__main__": setup_logging(default_path = "logging.json") func()
3.2 通過YAML檔案配置
通過YAML檔案進行配置,比JSON看起來更加簡介明了,
version: 1disable_existing_loggers: Falseformatters: simple: format: "%(asctime)s - %(name)s - %(levelname)s - %(message)s"handlers: console: class: logging.StreamHandler level: DEBUG formatter: simple stream: ext://sys.stdout info_file_handler: class: logging.handlers.RotatingFileHandler level: INFO formatter: simple filename: info.log maxBytes: 10485760 backupCount: 20 encoding: utf8 error_file_handler: class: logging.handlers.RotatingFileHandler level: ERROR formatter: simple filename: errors.log maxBytes: 10485760 backupCount: 20 encoding: utf8loggers: my_module: level: ERROR handlers: [info_file_handler] propagate: noroot: level: INFO handlers: [console,info_file_handler,error_file_handler]
通過YAML載入設定檔,然後通過logging.dictConfig配置logging
import yamlimport logging.configimport osdef setup_logging(default_path = "logging.yaml",default_level = logging.INFO,env_key = "LOG_CFG"): path = default_path value = os.getenv(env_key,None) if value: path = value if os.path.exists(path): with open(path,"r") as f: config = yaml.load(f) logging.config.dictConfig(config) else: logging.basicConfig(level = default_level)def func(): logging.info("start func") logging.info("exec func") logging.info("end func")if __name__ == "__main__": setup_logging(default_path = "logging.yaml") func()
4 Reference
http://wjdadi-gmail-com.iteye.com/blog/1984354
關於 logging 的一些瑣事python logging 重複寫日誌問題
本文摘自:http://www.cnblogs.com/zhbzz2007/p/5943685.html