Python logs, support for color printing and file size slices writing and writing to MongoDB

A log that supports different logger name writes to different files, different logger name logs are written to different MongoDB collection. Logmanager is easier to call, because the internal methods are all using underscores, the use of the underscore is that the protection and private methods do not need to call the outside world, do not need to understand him, at the time of the call Pycharm will not automatically complete the hint of these underlined irrelevant method, only exposed get_ And_add_handlers and Get_without_handlers two methods that may need to be called, Pyrcharm can automatically complement the two methods.

The main ideas and patterns are:

Logger add handler, each write Logger.debug/info support log a variety of records, because logger and handler is Publisher and subscriber relationship, in design mode is called Observer mode. When Logger.debug is in progress, logger calls its own _log method, _log method calls the handle method, handle calls the call handler method, Callhandler finds all subscribers, The handle method that invokes the Subscriber (the subscriber here is a variety of handler objects that have been added to the list by AddHandler), and the handler method of handler calls the Emit method. So you only need to inherit logging when writing custom handler. Handler class, overriding the emit method is possible. If you have special needs to pass in other parameters, you will need to override the Init method in addition to the emit method.

So a total of two main design patterns in the inside, logger and various handler objects with the pattern is the observer pattern, various patterns of the handler class and the basic handler class is the use of template mode (template mode is the main method of the steps in the base class write, Some of the different steps in the template class are set for abstract methods, and subclasses must be overridden. ）

# coding = utf8

"" "

Log management, support log printing to the console or writing to a file or mongodb

Use it as logger = LogManager (‘logger_name’). Get_and_add_handlers (log_level_int = 1, is_add_stream_handler = True, log_path = None, log_filename = None, log_file_size = 10, mongo_url = None, formatter_template = 2)

Or logger = LogManager (‘logger_name’). Get_without_handlers (), this kind of logs are not recorded immediately, after that, all logs can be captured and recorded by get_and_add_handlers according to loggerame at a single unified main gate.

"" "

import os

import unittest

import time

import re

from collections import OrderedDict

import pymongo

import logging

from logging.handlers import RotatingFileHandler

if os.name == ‘posix’:

   from cloghandler import ConcurrentRotatingFileHandler

formatter_dict = {

   1: logging.Formatter ('log time [% (asctime) s]-log name [% (name) s]-file [% (filename) s]-line [% (lineno) d]-log level [% (levelname) s]-log message [% (message) s] ', "% Y-% m-% d% H:% M:% S"),

   2: logging.Formatter ('% (asctime) s-% (name) s-% (filename) s-% (lineno) d-% (levelname) s-% (message) s', "% Y-% m -% d% H:% M:% S "),

}

class LogLevelException (Exception):

   def __init __ (self, log_level):

       err = ‘the set log level is {0}, the setting is wrong, please set it to a number in the range 1 2 3 4 5’.format (log_level)

       Exception .__ init __ (self, err)

class MongoHandler (logging.Handler):

   "" "

   A mongodb log handler that supports creating different collections by loggername and writing to mongodb

   "" "

   msg_pattern = re.compile ('(\ d +-\ d +-\ d + \ d +: \ d +: \ d +)-(\ S *?)-(\ S *?)-(\ d +)-(\ S *? )-([\ s \ S] *) ')

   def __init __ (self, mongo_url, mongo_database = ‘logs’):

       "" "

       : param mongo_url: mongo connection

       : param mongo_database: save the log ide database, the logs database is used by default

       "" "

       logging.Handler .__ init __ (self)

       mongo_client = pymongo.MongoClient (mongo_url)

       self.mongo_db = mongo_client.get_database (mongo_database)

   def emit (self, record):

       try:

           "" "The following method is used to extract the field" "

           # msg = self.format (record)

           # logging.LogRecord

           # msg_match = self.msg_pattern.search (msg)

           # log_info_dict = {‘time’: msg_match.group (1),

           # ‘Name’: msg_match.group (2),

           # ‘File_name’: msg_match.group (3),

           # ‘Line_no’: msg_match.group (4),

           # ‘Log_level’: msg_match.group (5),

           # ‘Detail_msg’: msg_match.group (6),

           #}

           level_str = None

           if record.levelno == 10:

               level_str = ‘DEBUG’

           elif record.levelno == 20:

               level_str = ‘INFO’

           elif record.levelno == 30:

               level_str = ‘WARNING’

           elif record.levelno == 40:

               level_str = ‘ERROR’

           elif record.levelno == 50:

               level_str = ‘CRITICAL’

           log_info_dict = OrderedDict ()

           log_info_dict [‘time’] = time.strftime (‘% Y-% m-% d% H:% M:% S’)

           log_info_dict [‘name’] = record.name

           log_info_dict [‘file_path’] = record.pathname

           log_info_dict [‘file_name’] = record.filename

           log_info_dict [‘func_name’] = record.funcName

           log_info_dict [‘line_no’] = record.lineno

           log_info_dict [‘log_level’] = level_str

           log_info_dict [‘detail_msg’] = record.msg

           col = self.mongo_db.get_collection (record.name)

           col.insert_one (log_info_dict)

       except (KeyboardInterrupt, SystemExit):

           raise

       except:

           self.handleError (record)

class ColorHandler (logging.StreamHandler):

   "" "Color log, showing different colors according to different levels of log" ""

   def emit (self, record):

       "" "

       0 40 black

       31 41 red

       32 42 green

       33 43 yellow

       34 44 blue

       35 45 fuchsia

       36 46 cyan

       37 47 white

       : param record:

       : return:

       "" "

       try:

           # logging.LogRecord.levelno

           msg = self.format (record)

           if record.levelno == 10:

               print (‘\ 033 [0; 32m% s \ 033 [0m’% msg) # green

           elif record.levelno == 20:

               print (‘\ 033 [0; 36m% s \ 033 [0m’% msg) # cyan

           elif record.levelno == 30:

               print (‘\ 033 [0; 34m% s \ 033 [0m’% msg) # blue

           elif record.levelno == 40:

               print (‘\ 033 [0; 35m% s \ 033 [0m’% msg) # fuchsia

           elif record.levelno == 50:

               print (‘\ 033 [0; 31m% s \ 033 [0m’% msg) # blood red

       except (KeyboardInterrupt, SystemExit):

           raise

       except:

           self.handleError (record)

class LogManager (object):

   "" "

   A log class for creating and capturing logs, supporting printing to the console, printing, and writing to log files.

   "" "

   def __init __ (self, logger_name = None):

       "" "

       : param logger_name: log name, print all logs when None

       "" "

       self.logger = logging.getLogger (logger_name)

       self._logger_level = None

       self._is_add_stream_handler = None

       self._log_path = None

       self._log_filename = None

       self._log_file_size = None

       self._mongo_url = None

       self._formatter = None

   def get_logger_and_add_handlers (self, log_level_intg ‘)

       LogManager (‘test’). Get_logger_and_add_handlers (1, log_path = ‘.. / logs’, log_filename = ‘test.log’)

       test_log = LogManager (‘test’). get_logger_and_add_handlers (1, log_path = ‘.. / logs’, log_filename = ‘test.log’)

       print (‘The following sentence does not repeat printing four times and write to the log four times’)

       time.sleep (1)

       test_log.debug (‘this sentence will not be printed four times and written to the log four times’)

   @ unittest.skip

   def test_get_logger_without_hanlders (self):

       "" "Testing logs without handlers" ""

       log = LogManager (‘test2’). get_logger_without_handlers ()

       print ('The following sentence will not be printed')

       time.sleep (1)

       log.info (‘this sentence will not be printed’)

   @ unittest.skip

   def test_add_handlers (self):

       "" "This allows you to write debug and info level logs at any specific place. You only need to specify the level at the main gate to filter, which is very convenient.

       LogManager (‘test3’). Get_logger_and_add_handlers (2)

       log1 = LogManager (‘test3’). get_logger_without_handlers ()

       print (‘The following sentence is info level and can be printed’)

       time.sleep (1)

       log1.info (‘This sentence is info level and can be printed’)

       print (‘The following sentence is a debug level and cannot be printed’)

       time.sleep (1)

       log1.debug (‘This sentence is a debug level and cannot be printed’)

   @ unittest.skip

   def test_only_write_log_to_file (self):

       "" "Write to log file only" ""

       log5 = LogManager (‘test5’). get_logger_and_add_handlers (is_add_stream_handler = False, log_path = ‘.. / logs’, log_filename = ‘test5.log’)

       print ('The following sentence is only written to the file')

       log5.debug ('This sentence is only written to the file')

   def test_color_and_mongo_hanlder (self):

       "" "Testing colored logs and log writing to mongodb" ""

       from app import config

       logger = LogManager (‘helloMongo’). get_logger_and_add_handlers (mongo_url = config.connect_url)

       logger.debug ('a debug-level log')

       logger.info ('an info-level log')

       logger.warning (‘a warning level’ ’)

       logger.error (‘an error-level log’)

       logger.critical (‘A critical level log’)

if __name__ == "__main__":

   unittest.main ()

The effect of Test_color_and_mongo_hanlder is as follows.

Because the custom Colorhandler is added to logger, the log recorded by the console is colored.

MongoDB inside

Using the Name property of the log logger as the name of the collection, different Loggername create different collection. The main saved field has the journal name, which is the level of log and log details in what file of what function/method is printed.

Python logs, support for color printing and file size slices writing and writing to MongoDB