In the process of running the program, all the variables are in memory, for example, to define a dict:
D = dict (name='Bob', age=20, score=88)
Variables can be modified at any time, such as name
change ‘Bill‘
, but once the program is finished, the memory used by the variables is fully recycled by the operating system. If you do not save the modified ‘Bill‘
storage to disk, the next time you rerun the program, the variable is initialized ‘Bob‘
.
The process of changing a variable from memory to a storage or transfer is called serialization, and in Python it's called pickling, which is also called serialization,marshalling,flattening in other languages, and so on.
After serialization, the serialized content can be written to disk or transferred over the network to another machine.
In turn, re-reading the variable contents from the serialized object into memory is called deserialization, i.e. unpickling.
Python provides a pickle
module for serialization.
First, we try to serialize and write an object to the file:
Import pickle>>> d = dict (name='Bob', age=20, score=88)>>> Pickle.dumps (d) B ' \x80\x03}q\x00 (x\x03\x00\x00\x00ageq\x01k\x14x\x05\x00\x00\x00scoreq\x02kxx\x04\x00\x00\x00nameq\x03x\x03\ X00\x00\x00bobq\x04u. '
pickle.dumps()
The method serializes any object into one bytes
, and then it can write to the bytes
file. Or use another method pickle.dump()
to serialize the object directly after it is written to a File-like object:
>>> f = open ('dump.txt'wb')>>> Pickle.dump (d, f)>>> F.close ()
Look at the dump.txt
files that are written, a bunch of messy stuff, all of the information inside the object that Python holds.
When we want to read the object from disk to memory, we can first read the content to one bytes
, and then deserialize the object with a pickle.loads()
method, or directly pickle.load()
from a method to deserialize the file-like Object
object directly. We open another Python command line to deserialize the object we just saved:
>>> f = open ( " dump.txt Span style= "color: #800000;" ", " rb ' >>> d = Pickle.load (f) >>> F.close () >>> d{ " age : ' score : " name : " bob }
The contents of the variable are back!
Of course, this variable and the original variable are completely irrelevant objects, they are just the same content.
The problem with Pickle is the same as for all other programming language-specific serialization problems, that is, it can only be used in Python, and may be incompatible with each other in Python, so it's okay to save only those unimportant data with pickle and not successfully deserialize it.
Json
If we are going to pass objects between different programming languages, we have to serialize the object into a standard format, such as XML, but the better way is to serialize it to JSON, because JSON represents a string that can be read by all languages, easily stored to disk, or transmitted over a network. JSON is not only a standard format, but also faster than XML, and can be read directly in the Web page, very convenient.
JSON represents objects that are standard JavaScript language objects, and JSON and Python have built-in data types that correspond to the following:
JSON type |
Python type |
{} |
Dict |
[] |
List |
"String" |
Str |
1234.56 |
int or float |
True/false |
True/false |
Null |
None |
Python's built-in json
modules provide a very sophisticated translation of Python objects into JSON format. Let's look at how to turn the Python object into a JSON:
Import JSON>>> d = dict (name='Bob', age=20, score=88)>>> Json.dumps (d) ' {"Age": +, "score": "," "Name": "Bob"} '
dumps()
method returns one str
, the content is the standard JSON. Similarly, the dump()
method can write JSON directly to one file-like Object
.
To deserialize JSON into a Python object, loads()
or a corresponding load()
method, the former JSON string is deserialized, the latter reads the string from file-like Object
and deserializes:
' {"Age": +, "score": "," "Name": "Bob"} '>>> json.loads (json_str) {'age'score ' ' name ' ' Bob '}
JSON advanced
Python dict
objects can be serialized directly into JSON {}
, but, many times, we prefer to class
represent objects, such as defining Student
classes, and then serializing:
Import JSON class Student (object): def __init__ (self, name, age, score): = name = age =Student (' Bob', 20, 88 )print(Json.dumps (s))
Run the code and relentlessly get aTypeError
Traceback (most recent): <__main__ is not JSON serializable
The reason for the error is that the Student
object is not an object that can be serialized as JSON.
If even class
instance objects cannot be serialized as JSON, this is certainly unreasonable!
Don't worry, let's take a closer look at dumps()
the parameter list of the method, you can see that the obj
dumps()
method provides a whole bunch of optional parameters in addition to the first required parameter:
Https://docs.python.org/3/library/json.html#json.dumps
These optional parameters are for us to customize JSON serialization. The previous code was unable to Student
serialize the class instance to JSON because by default the dumps()
method does not know how to Student
change the instance to a JSON {}
object.
The optional parameter default
is to turn any object into an object that can be serialized as JSON, we just need to Student
write a conversion function, and then pass in the function:
def student2dict (STD): return { 'name': std.name, 'age' : std.age, 'score': Std.score }
In this way, the Student
instance is first student2dict()
converted into a function and dict
then successfully serialized as JSON:
Print (Json.dumps (S, default=student2dict)) {"Age""name" "Bob""score": 88}
However, the next time you encounter an Teacher
instance of a class, you cannot serialize to JSON. We can steal a lazy, turn any class
instance into dict
:
Print (Json.dumps (S, default=Lambda obj:obj.__dict__))
Because class
the usual instance has a __dict__
property, it is the one dict
that stores the instance variable. There are a few exceptions, such as __slots__
the defined class.
Similarly, if we are going to deserialize JSON into an Student
object instance, the loads()
method first converts an dict
object, and then our incoming object_hook
function is responsible for dict
converting to an Student
instance:
def Dict2student (d): return Student (d['name'], d['age'), d[' score '])
' {"Age": +, "score": "," "Name": "Bob"} ' Print (Json.loads (json_str, object_hook=dict2student))<__main__. Student Object at 0x10cd3c190>
Python Learning notes serialization