Python Common Modules continued

First, logging

Modules for logging and thread safety

import logginglogging.basicConfig(level=logging.DEBUG,                    format=‘%(asctime)s - %(name)s - %(levelname)s -%(module)s:  %(message)s‘,                    datefmt=‘%Y-%m-%d %H:%M:%S %p‘,                    filename=‘log.txt‘,                    filemode=‘w‘                    )logging.debug("debug messages")logging.info("info messages")logging.warning("warning messages")logging.error("error messages")logging.critical("critical messages")

Log Text Results

2018-03-23 10:33:43 AM - root - DEBUG -os_module:  debug messages2018-03-23 10:33:43 AM - root - INFO -os_module:  info messages2018-03-23 10:33:43 AM - root - WARNING -os_module:  warning messages2018-03-23 10:33:43 AM - root - ERROR -os_module:  error messages2018-03-23 10:33:43 AM - root - CRITICAL -os_module:  critical messages

Log Basic configuration options:

Format	Description
FileName	Specify the log file name
FileMode	Specifies the mode of the operation log file, default is ' a '
Format	Format of logging
Datefmt	Format Date and time
Style	Style
Level	Specify the log level to log
Stream	cannot be configured with filename by using the specified stream initialization
Handlers	Specify iterable, not compatible with filename and stream

Log level:

Level

	Numeric Value
CRITICAL	50
ERROR	40
WARNING	30
INFO	20
DEBUG	10
NOTSET	0

Log format:

More log details please click here

Second, Json,pickle,shelve

One way to save data while the program is running is to write all the data in a simple text file in a formatted format, so that as long as the saved and loaded tools agree on the chosen format, we can use any custom format we want.
1, Pickle
The Pickle module converts the in-memory Python object into a serialized byte stream, which is a string of bytes that can be written to any similar file object, while the Pickle module can reconstruct the original in-memory object based on the serialized byte stream.
To convert and write to a file:

import pickledb = {‘name‘:‘Eric Jia‘,‘age‘:‘18‘,‘job‘:‘ops‘,‘pay‘:200}dbfile = open(‘pickle_db‘,‘wb‘)pickle.dump(db,dbfile)dbfile.close()

To read data from a file:

import pickledbfile = open(‘pickle_db‘,‘rb‘)pickle.load(dbfile)dbfile.close()

Convert directly, do not write to file:

import pickledb = {‘name‘:‘Eric Jia‘,‘age‘:‘18‘,‘job‘:‘ops‘,‘pay‘:200}a = pickle.dumps(db)print("a:",a)b = pickle.loads(a)print("b:",b)

a: b‘\x80\x03}q\x00(X\x04\x00\x00\x00nameq\x01X\x08\x00\x00\x00Eric Jiaq\x02X\x03\x00\x00\x00ageq\x03X\x02\x00\x00\x0018q\x04X\x03\x00\x00\x00jobq\x05X\x03\x00\x00\x00opsq\x06X\x03\x00\x00\x00payq\x07K\xc8u.‘b: {‘name‘: ‘Eric Jia‘, ‘age‘: ‘18‘, ‘job‘: ‘ops‘, ‘pay‘: 200}

2. JSON
JSON is used primarily for the conversion between Python data types and strings, and the usage is consistent with pickle

1、dump：转换为字符串存入文件import jsondb = {‘name‘:‘Eric Jia‘,‘age‘:‘18‘,‘job‘:‘ops‘,‘pay‘:200}dbfile = open(‘json_db‘,‘w‘)json.dump(db,dbfile)dbfile.close()2、load：从文件中取出并转换为Python数据对象dbfile = open(‘json_db‘,‘r‘)json.load(dbfile)dbfile.close()3、dumps，loads：直接转换，不写入文件db = {‘name‘:‘Eric Jia‘,‘age‘:‘18‘,‘job‘:‘ops‘,‘pay‘:200}a = json.dumps(db)print(a)b = json.loads(a)print(b)

3, Shalve
Pickle and JSON can persist Python data types, assuming that your filesystem can handle any number of required files, each file is a separate pickle file, which avoids the need to reload and store the entire database when a record is modified. This requires storing each pickle file name as a key to the dictionary, accessing the data through key values, and the Python standard library provides us with a module called shelves to do these things.
Create a shelve database

N1 = {‘name‘:‘Eric Jia‘,‘age‘:‘18‘,‘job‘:‘ops‘,‘pay‘:200}N2 = {‘name‘:‘Bob smith‘,‘age‘:‘28‘,‘job‘:‘java‘,‘pay‘:2000}N3 = {‘name‘:‘David‘,‘age‘:‘38‘,‘job‘:‘go‘,‘pay‘:20000}import shelvedb = shelve.open(‘make_db‘)db[‘N1‘] = N1db[‘N2‘] = N2db[‘N3‘] = N3db.close()

The shelve database is created as follows

Re-open shelve and get data by key

import shelvedb = shelve.open(‘make_db‘)for key in db:    print(key,"=>",db[key])db.close()输出：N1 => {‘name‘: ‘Eric Jia‘, ‘age‘: ‘18‘, ‘job‘: ‘ops‘, ‘pay‘: 200}N2 => {‘name‘: ‘Bob smith‘, ‘age‘: ‘28‘, ‘job‘: ‘java‘, ‘pay‘: 2000}N3 => {‘name‘: ‘David‘, ‘age‘: ‘38‘, ‘job‘: ‘go‘, ‘pay‘: 20000}

Summary:
1. Shelve internally uses pickle to serialize and deserialize records, and its interface is as simple as pickle: the same as the dictionary, with the addition of open and close calls.
2. Pickle is used to convert between Python-specific types and Python data types, which is unique to Python
3. JSON is used to convert between string and Python data types, and all program languages are available.

Python Common Modules continued