Python common module (ii)

I. JSON and PICKLE

JSON and pickle modules are designed to complete the serialization of data.

Serialization refers to the process of turning an object (variable) from memory into a storage or transfer, called picking in Python, and other words in other languages, but it is a meaning

Serialization is to solve two problems of data persistence State and cross-platform data interaction.

How is serialization implemented?

　　Json

JSON (JavaScript object Notation, JS Simplified) is a lightweight data interchange format. It is based on a subset of ECMAScript (the JS specification developed by the European Computer Association) that uses text formats that are completely independent of the programming language to store and represent data. The simplicity and clarity of the hierarchy makes JSON an ideal data exchange language. Easy for people to read and write, but also easy to machine analysis and generation, and effectively improve the network transmission efficiency.

If we are going to pass objects between different programming languages, we have to serialize the object to a standard format, the best way is to serialize it to JSON, because JSON represents a string that can be read by all languages, easily stored on disk or transmitted over the network, and JSON is not just a standard format, Flat and faster than XML, and can be read directly on the Web page, very convenient

Because JSON represents objects that are standard JavaScript language objects, JSON and Python have built-in data types such as the following:

　　

　　Pickle

Pickle is the same as JSON for a module that implements serialization, but the problem with pickle is that it can only be used in Python, and almost all of the data types in Python are serialized with Pickle, but the readability of the data after pickle serialization is poor and generally unrecognized.

Pickle.dump (obj, file[, Protocol])
Serializes the object and writes the resulting data stream to the file object. The parameter protocol is a serialization mode with a default value of 0, which indicates serialization as text. The value of the protocol can also be 1 or 2, which is serialized as a binary representation.

Pickle.load (file)
Deserializes the object. Resolves the data in a file to a Python object. It is important to note that in the case of load (file), Python is able to find the definition of the class, otherwise it will give an error:

Two. Logging module

1. Log Related concepts

A log is a way to track events that occur when some software is running. Software developers can call logging related methods to their code to indicate that something has happened. An event can be described with a message that can contain optional variable data. In addition, events also have the concept of importance, and this importance can also be referred to as severity level.

Logs are used to enable us to log and analyze logs to see if a system or software program is functioning properly, or to quickly locate problems in the event of an application failure.

2. Level of log

CRITICAL = 50

ERROR = 40

WARNING = 30

INFO = 20

DEBUG = 10

NOTSET = 0

The default level is warning

code example:

　　　　

Execution Result:

　　　　

Use of 3.logging modules

Logging the module-level common functions defined by the module:

Logging.debug (Msg,*args,**kwargs) Create a log record of level debug

Logging. Info (Msg,*args,**kwargs) Create a log record of level info

Logging.warning (Msg,*args,**kwargs) Create a log record of level warning

Logging.error (Msg,*args,**kwargs) Create a log record of level error

Logging.critical (Msg,*args,**kwargs) Create a log record of level critical

Logging.log (Level,*args,**kwargs) to create a level of log record

Logging.basicconfig (**kwargs) One-time configuration of root logger

Formatter,handler,logger,filter Objects for 4.logging modules

Logger: The object that generated the log
Filter: Object that filters logs
Handler: Receive logs and then control print to a different place, Filehandler used to print to the file, Streamhandler used to print to the terminal
Formatter objects: You can customize different log format objects and then bind to different handler objects to control the different handler log formats

Logger is the first level of filtering, and then to handler, we can give logger and handler at the same time to set levels

Logging configuration, code example:

LOGGING = {
　　　　 ' Version ':1,
　　　　 ' Disable_existing_loggers ':False,
　　　　 ' formatters ': {
　　　　　　 ' Standard ': {
　　　　　　　　 ' Format ':' [% (asctime) s][% (threadname) s:% (thread) d][task_id:% (name) s][% (filename) s:% (Lineno) d] '
' [% (levelname) s][% (message) s] '
},
　　　　　　 ' Simple ': {
　　　　　　　　 ' Format ':' [% (LevelName) s][% (asctime) s][% (filename) s:% (Lineno) d]% (message) s '
},
　　　　　　 ' Collect ': {
　　　　　　　　 ' Format ':'% (message) s '
}
},
　　　　 ' Filters ': {
　　　　　　 ' Require_debug_true ': {
　　　　　　　　 ‘()‘:' Django.utils.log.RequireDebugTrue ',
},
},
　　　　 ' Handlers ': {
　　　　　　 #打印到终端的日志
　　　　　　 ' Console ': {
　　　　　　　　 ' Level ':' DEBUG ',
　　　　　　　　 ' Filters ': [' Require_debug_true '],
　　　　　　　　 ' Class ':' Logging. Streamhandler ',
　　　　　　　　 ' Formatter ':' Simple '
},
　　　　　　 #打印到文件的日志, collect logs of info and above
　　　　　　 ' Default ': {
　　　　　　　　 ' Level ':' INFO ',
　　　　　　　　 ' Class ':' Logging.handlers.RotatingFileHandler ',# Save to file, auto-cut
　　　　　　　　 ' filename ': Os.path.join (Base_log_dir,"Xxx_info.log"),# log file
　　　　　　　　 ' MaxBytes ':1024 *1024 *5,# Log Size 5M
　　　　　　　　 ' Backupcount ':3,
　　　　　　　　 ' Formatter ':' Standard ',
　　　　　　　　 ' Encoding ':' Utf-8 ',
},
　　　　　　 #打印到文件的日志: Collect errors and the logs above
　　　　　　 ' ERROR ': {
　　　　　　　　 ' Level ':' ERROR ',
　　　　　　　　 ' Class ':' Logging.handlers.RotatingFileHandler ',# Save to file, auto-cut
　　　　　　　　 ' filename ': Os.path.join (Base_log_dir,"Xxx_err.log"),# log file
　　　　　　　　 ' MaxBytes ':1024 *1024 *5,# Log Size 5M
　　　　　　　　 ' Backupcount ':5,
　　　　　　　　 ' Formatter ':' Standard ',
　　　　　　　　 ' Encoding ':' Utf-8 ',
},
　　　　　　 #打印到文件的日志
　　　　　　 ' Collect ': {
　　　　　　　　 ' Level ':' INFO ',
　　　　　　　　 ' Class ':' Logging.handlers.RotatingFileHandler ',# Save to file, auto-cut
　　　　　　　　 ' filename ': Os.path.join (Base_log_dir,"Xxx_collect.log"),
　　　　　　　　 ' MaxBytes ':1024 *1024 *5,# Log Size 5M
　　　　　　　　 ' Backupcount ':5,
　　　　　　　　 ' Formatter ':' Collect ',
　　　　　　　　 ' Encoding ':"Utf-8"
}
},
　　　　 ' Loggers ': {
　　　　　　 #logging. GetLogger (__name__) Get the logger configuration
　　　　　　 ‘‘: {
　　　　　　　　 ' Handlers ': [' Default ',' Console ', ' propagate ': true,
},
#logging GetLogger (' collect ') get the logger configuration
　　　　　　　　 ' collect ': {
handlers ': [ " Console ', ' collect '],
' level ': }
},
}

Python common module (ii)