Serialization module JSON Pickle shelve

What is a module:-----------

Common scenario: A module is a file that contains Python definitions and declarations, and the file name is the suffix of the module name plus the. Py.

Module Category: 1----built-in modules

2----Expansion module (https://pypi.org/)

3----Custom modules to write their own files

Why use a module?

If you quit the Python interpreter and then re-enter, then the functions or variables you defined previously will be lost, so we usually write the program to a file so that it can be persisted and executed in Python test.py when needed, when test.py is called a scripting script.

With the development of the program, more and more functions, in order to facilitate management, we usually divide the program into a file, so that the structure of the program is clearer and easier to manage. In this case, we can not only use these files as scripts to execute, but also as a module to import into other modules, to achieve the reuse of functionality,

Serialization module:

What is serialization??

Serialization of (serialization) The process of converting the state information of an object into a form that can be stored or transmitted

The popular point is that-------the process of converting the original dictionary, list, and so on into a string is called serialization .

For example, one of the data we calculate in Python code needs to be used for another program, so how do we give it? Now we can think of a method that exists in the file, and then another Python program is read from the file. But we all know that there is no dictionary concept for a file, so we can only convert the data into a dictionary and put it in a file. You must ask, it is very simple to convert a dictionary into a string, that is, str (DIC) can do it, why do we have to learn the serialization module? Yes, the process of serialization is the process of turning from DIC to STR (DIC). Now you can convert a dictionary called dic into a string by using Str (DIC), but how do you convert a string into a dictionary? You must have thought of it. Eval (), if we pass a dictionary str_dic of a string type to eval, we get a dictionary type that is returned. The eval () function is very powerful, but what does eval do? E official demo is interpreted as: the string str as a valid expression to evaluate and return the results of the calculation. but! Powerful functions are at the cost. Security is one of its biggest drawbacks. Imagine that if we read from a file is not a data structure, but a " Delete the file " similar to the destructive statement, then the consequences are not set to imagine. The use of eval is a risk. Therefore, we do not recommend using the Eval method for deserialization (converting STR to a data structure in Python)

Why should there be serialization

Purpose of serialization

1. Persist the custom object in some form of storage; 2. Transfer objects from one place to another. 3, make the program more maintenance.

1---->>>>json

For different languages, supported data types STR INT List dict bool

ImportJSON#dic = {"Alex": (' Women ', ' women ', ' Old Woman ')}#Dic2 = {"Alex1": (' Women ', ' women ', ' Old Woman ')}#dic3 = {"Alex2": (' Women ', ' women ', ' Old Woman ')}#With open ("Json-file", mode= "W", encoding= "Utf-8") as F:#s1=json.dumps (DIC)#s2=json.dumps (DIC2)#s3=json.dumps (DIC3)#f.write (s1+ "\ n")#f.write (s2+ "\ n")#f.write (s3+ "\ n")#With open ("Json-file", mode= "R", encoding= "Utf-8") as F1:#For line in F1:#Print (Json.loads (line))

JSON file Multi-data operation

Write dumps and loads for most data

Dump can only read and write a serialized string at a time

Serialize obj to a JSON formatted str. (string representation of the JSON object) Skipkeys: The default value is False if the data in the Dict keys is not the basic type of Python (Str,unicode, Int,long,float,bool,none), when set to False, the TypeError error is reported. When set to true, this type of key ensure_ascii is skipped: when it is true, all non-ASCII characters are displayed as \uxxxx sequences, just set the ENSURE_ASCII to false at dump. In this case, the Chinese in JSON can be displayed normally. ) If Check_circular isFalse, then the circular reference check forContainer types'll be skipped andA circular reference would resultinchAn Overflowerror (orworse). If Allow_nan isFalse, then it'll be a valueerror-serialize out of range float values (Nan, INF,-inf)inchStrict compliance of the JSON specification, instead of using the JavaScript equivalents (NaN, Infinity,-Infinity). Indent: Should be a non-negative integer, if it is 0 is the Shelf branch display, if empty is the most compact display, otherwise will be wrapped and according to the value of indent displayed in front of the blank branch display, so that the printed JSON data is also called pretty-printed JSON separators: delimiter, which is actually a tuple (item_separator, dict_separator), the default is (', ', ': '), which means "," between Keys in dictionary Separated by a ":" Between key and value. Default (obj) isA function that shouldreturnA serializable version of objor RaiseTypeError. The default simply raises TypeError. Sort_keys: Sorts the data according to the value of keys. To use a custom Jsonencoder subclass (e.g. one, overrides the. Default () method to serialize additional types), Specif Y it with the CLS Kwarg; otherwise Jsonencoder isUsed.

Additional parameter Description

ImportJsondata= {'username':['Li Hua','Erlengzi'],'Sex':'male',' Age': 16}json_dic2= Json.dumps (data,sort_keys=true,indent=2,separators= (',',':'), ensure_ascii=False)Print(JSON_DIC2)

formatted output of JSON

2------>>>>pickle

Two modules for serialization

• JSON, used to convert between string and Python data types
• Pickle for conversion between Python-specific types and Python data types

The Pickle module provides four functions: dumps, dump (serialize, save), loads (deserialization, read), load (not only serializable dictionary, list ... can serialize arbitrary data types in Python )

#dic = {' Alex ': (' Women ', ' women ', ' Old Woman ')}#Dic2 = {"Alex1": (' Women ', ' women ', ' Old Woman ')}#dic3 = {"Alex2": (' Women ', ' women ', ' Old Woman ')}#with open ("Pickle_file", mode= "WB") as F:#pickle.dump (dic,f)#pickle.dump (dic2,f)#pickle.dump (dic3,f)#with open ("Pickle_file", mode= "RB") as F:#While 1:#Try:#Print (Pickle.load (f))#except Eoferror:# Break

Pickle File Multi-data operation

Note: file read/write mode Q Btyes:

3----->>>>shelve

Shelve is also a serialization tool provided to us by Python, which is simpler to use than pickle.
Shelve only provides us with an open method, which is accessed using key and is similar to a dictionary.

ImportSHELVEF= Shelve.open ('Shelve_file') f['Key'] = {'int': 10,'float': 9.5,'string':'Sample Data'}#directly to the file handle, you can deposit dataf.close ()ImportSHELVEF1= Shelve.open ('Shelve_file') Existing= f1['Key']#when you take out the data, you just need to get it directly with key, but if key doesn't exist, it will be an error.f1.close ()Print(existing)

View Code

There is a limit to this module, which does not allow multiple applications to write to the same db at the same time. So when we know that our application is only read, we can let shelve open the db by read-only mode.

Import= Shelve.open ('shelve_file', flag='r'  = f['key']f.close ()print(existing)

View Code

Since shelve does not record any modifications to the persisted object by default, we need to modify the default parameters at Shelve.open () or the object's modifications will not be saved.

ImportSHELVEF1= Shelve.open ('Shelve_file')Print(f1['Key']) f1['Key']['New_value'] ='This is not here before'f1.close () F2= Shelve.open ('Shelve_file', writeback=True)Print(f2['Key']) f2['Key']['New_value'] ='This is not here before'f2.close ()

View Code

There are pros and cons to the writeback approach. The advantage is to reduce the probability of our error, and to make the object's persistence more transparent to the user, but this method is not required in all cases, first, after using writeback, shelf in open () will increase the additional memory consumption, and when the DB in close () , each object in the cache is written to the DB, which also brings additional wait times. Because shelve has no way of knowing which objects in the cache have been modified and which objects have not been modified, all objects will be written.

Serialization module JSON Pickle shelve