Analysis of the Log Module in openstack

1. The module is generally in openstack/common/log. py. In fact, the most important thing is to call the logging module in Python;

The entry function is in
Def setup (product_name, version = 'unknown '):
"Setup logging ."""
If Conf. log_config_append:
_ Load_log_config (Conf. log_config_append)
Else:
_ Setup_logging_from_conf (product_name, Version)
SYS. Export thook = _ create_logging_export thook (product_name)

If log_config_append is set in the configuration file, log_config_append is the configuration of logging, as shown below:

Implemented in function _ load_log_config:
Def _ load_log_config (log_config_append ):
Try:
Logging. config. fileconfig (log_config_append,
Disable_existing_loggers = false)
Failed t moves. configparser. Error as exc:
Raise logconfigerror (log_config_append, STR (EXC ))

Logconfigerror is a custom exception, which is also in log. py;

If log_config_append is not set, log settings are performed through the _ setup_logging_from_conf function;
508 log_root = getlogger (none). Logger
509 for handler in log_root.handlers:
510 log_root.removehandler (handler)
511
512 if Conf. use_syslog:
513 facility = _ find_facility_from_conf ()
514 # todo (bogdando) use the format provided by rfcsysloghandler
515 # After existing Syslog Format deprecation in J
516 if Conf. use_syslog_rfc_format:
517 syslog = rfcsysloghandler (address = '/dev/log ',
518 facility = facility)
519 else:
520 syslog = logging. Handlers. sysloghandler (address = '/dev/log ',
521 facility = facility)
522 log_root.addhandler (syslog)
523
524 logpath = _ get_log_file_path ()
525 if logpath:
526 filelog = logging. Handlers. watchedfilehandler (logpath)
527 log_root.addhandler (filelog)
528
529 if Conf. use_stderr:
530 streamlog = colorhandler ()
531 log_root.addhandler (streamlog)
532
533 Elif not logpath:
534 # Pass SYS. stdout as a positional argument
535 # python2.6 callthe argument STRM, in 2.7 It's Stream
536 streamlog = logging. streamhandler (SYS. stdout)
537 log_root.addhandler (streamlog)
538
539 if Conf. publish_errors:
540 handler = importutils. import_object (
541 "openstack. Common. log_handler.publisherrorshandler ",
542 logging. Error)
543 log_root.addhandler (handler)
544
545 datefmt = Conf. log_date_format
546 for handler in log_root.handlers:
547 # Note (alaski): Conf. log_format overrides everything currently. This
548 # shocould be deprecated in favor of Context Aware formatting.
549 if Conf. log_format:
550 handler. setformatter (logging. formatter (FMt = Conf. log_format,
551 datefmt = datefmt ))
552 log_root.info ('precated: log_format is now deprecated and will'
553 'be removed in the next release ')
554 else:
555 handler. setformatter (contextformatter (project = project,
556 version = version,
557 datefmt = datefmt ))
558
559 if Conf. Debug:
560 log_root.setlevel (logging. Debug)
561 Elif Conf. verbose:
562 log_root.setlevel (logging. info)
563 else:
564 log_root.setlevel (logging. Warning)
565
566 for pair in Conf. default_log_levels:
567 mod, _ SEP, level_name = pair. Partition ('= ')
568 level = logging. getlevelname (level_name)
569 logger = logging. getlogger (MOD)
570 logger. setlevel (level)

508 ~ Row 510; first, get the root logger. in logging, there is a default rootlogger. Any other logger is inherited by this rootlogger, and then the handler is cleared. Logger can set handler, handler is responsible for displaying information sent to logger, such as stdout and output to files;

512 ~ Row 3: whether to write syslog data to the system. In Linux, the syslog data is/dev/log, and in Mac, the syslog data is/var/run/syslog.

Rows 524 to 527, The _ get_log_file_path function obtains the Log Path, which is actually used to determine whether the conf. LOG_FILE and Conf. log_dir are set in the conf;
Watchedfilehandler outputs log information to the file. The specific meaning of watched is that when the output to the file is changed (a file is deemed to have changed if its device or inode have changed ), when a file changes,
The previous file stream will be closed, and the file will be opened again with a new file stream; this is useful when the log rotation mechanism is used.

Lines 529 to 531. If Conf. stderr is set, use colorhandler.

Rows 533 to 537. If no logpath is set, stream handler (standard output) is added to the root log;

Rows 539 to 543. If publish_errors is set, publisherrorhandler is called.
22 class publisherrorshandler (logging. Handler ):
23 def emit (self, record ):
24 if ('openstack. Common. notifier. log_notifier' in
25 cfg. conf. icationication_driver ):
26 return
27 notifier. API. Policy (none, 'error. Her her ',
28 'error _ notification ',
29 notifier. API. error,
30 dict (error = record. getmessage ()))
Publisherrorhandler overrides the emit method and sends the log information using notifier;

Lines 545 to 557 set the log format;

559 to 564 select one from debug <info <warning as the Log Level of rootlogger. If other logs are not set separately, the level of rootlogger is inherited by default; debug and verbose can be configured through the configuration file;

566,570 you can use Conf./default_log_levels to set different levels for different modules;

After the basic analysis of the setup function is complete, you can use the setup function as follows;
From Higgs. Agent. Common import config
Config. setup_logging (CFG. conf)

But the common method at the beginning is
From openstack. Common import log as logging
Log = logging. getlogger (_ name __)

In this case, it calls the contextadapter class in openstack/common/log. py in real time;
The contextadapter class inherits the baseloggeradapter class from the logging. loggeradapter class.
This allows you to process logs in a unified manner and add context information (contextual information) when printing logs. For example, in a network application, you may want to include some client information in the log.
(For example, the username and IPaddress of the client ).
The implementation of loggeradapter is more like a proxy or adapter. when instantiating a loggeradapter, you need to pass a logger instance and a dict-like object that contains your context information. Then, loggeradapter also requires info, debug,
Errors, critical, and other methods, and their function signatures are the same as those of logger itself, and then what loggeradapter does is similar to the following snippet;

Def debug (self, MSG, * ARGs, ** kwargs ):
"Delegate a debug call to the underlying logger,
After adding contextual information from this adapter instance ."""
MSG, kwargs = self. Process (MSG, kwargs)
Self. Logger. debug (MSG, * ARGs, ** kwargs)

First, call the process method to process the parameters, and then pass the processed parameters to the underlying logger for calling;
In openstack, some project, version, and other information are added to the log;

There is also a lazylogger that delays loading logger;

Analysis of the Log Module in openstack