JSON and pickle for python serialization

JSON (JavaScriptObjectNotation) is a lightweight data exchange format. It is based on a subset of ECMAScript. JSON uses a completely language-independent text format, but it also uses a habit similar to the C language family (including C, C ++, Java, JavaScript, Perl, Python, and so on ). These features make JSON an ideal data exchange language. Easy to read and write, and easy to parse and generate by machines (generally used to increase the network transmission rate ).

JSON module

JSON (JavaScript Object Notation) is a lightweight data exchange format. It is based on a subset of ECMAScript. JSON uses a completely language-independent text format, but it also uses a habit similar to the C language family (including C, C ++, Java, JavaScript, Perl, Python, and so on ). These features make JSON an ideal data exchange language. Easy to read and write, and easy to parse and generate by machines (generally used to increase the network transmission rate ).
JSON is composed of list and dict in python.

I. conversion of python data and JSON data formats

In pthon, the str type to JSON is converted to unicode type, and None is converted to null. dict corresponds to the object.

II. data encoding and decoding

1. simple data encoding/decoding

The simple type is the python type shown in the preceding table.

Dumps: serialize objects

# Coding: utf-8import json # Simple encoding =============================================== ========= print json. dumps (['foo', {'bar': ('Baz', None, 1.0, 2)}]) # ["foo", {"bar ": ["baz", null, 1.0, 2]}] # print json dictionary sorting. dumps ({"c": 0, "B": 0, "a": 0}, sort_keys = True) # {"a": 0, "B": 0, "c": 0} # customize the separator print json. dumps ([1, 2, 3, {'4': 5, '6': 7}], sort_keys = True, separators = (',',':')) # [1, 2, 3, {"4": 5, "6": 7}] print json. dumps ([1, 2, 3, {'4': 5 , '6': 7}], sort_keys = True, separators = ('/','-')) # [1/2/3/{"4"-5/"6"-7}] # increase indentation to enhance readability, but indent spaces will make the data larger print json. dumps ({'4': 5, '6': 7}, sort_keys = True, indent = 2, separators = (',',':')) #{# "4": 5, # "6": 7 #}# another useful dumps parameter is skipkeys. the default value is False. # When the dumps method is used to store dict objects, the key must be of the str type. if other types exist, a TypeError exception occurs. If this parameter is enabled and set to True, this key is ignored. Data = {'a': 1, (1,2): 123} print json. dumps (data, skipkeys = True) # {"a": 1}

Dump: serialize and save the object to a file

# Serialize and save the object to the Object obj = ['foo', {'bar': ('Baz', None, 1.0, 2)}]
With open (r "c: \ json.txt", "w +") as f:
Json. dump (obj, f)

Loads: deserialization of serialized strings

import jsonobj = ['foo', {'bar': ('baz', None, 1.0, 2)}]a= json.dumps(obj)print json.loads(a)# [u'foo', {u'bar': [u'baz', None, 1.0, 2]}]

Load: read and deserialize the serialized string from the file

With open (r "c: \ json.txt", "r") as f: print json. load (f)

III. coding and decoding of custom complex data types

For example, if we encounter a data type that is not supported by json by default, such as an object datetime or a custom class object, we need a custom codec function. There are two methods to implement custom codec.

1. Method 1: custom codec functions

#! /usr/bin/env python# -*- coding:utf-8 -*-# __author__ = "TKQ"import datetime,jsondt = datetime.datetime.now()def time2str(obj):    #python to json    if isinstance(obj, datetime.datetime):        json_str = {"datetime":obj.strftime("%Y-%m-%d %X")}        return json_str    return objdef str2time(json_obj):    #json to python    if "datetime" in json_obj:        date_str,time_str = json_obj["datetime"].split(' ')        date = [int(x) for x in date_str.split('-')]        time = [int(x) for x in time_str.split(':')]        dt = datetime.datetime(date[0],date[1], date[2], time[0],time[1], time[2])        return dt    return json_obja = json.dumps(dt,default=time2str)print a# {"datetime": "2016-10-27 17:38:31"}print json.loads(a,object_hook=str2time)# 2016-10-27 17:38:31

2. Method 2: inherit the JSONEncoder and JSONDecoder classes and override related methods.

#! /usr/bin/env python# -*- coding:utf-8 -*-# __author__ = "TKQ"import datetime,jsondt = datetime.datetime.now()dd = [dt,[1,2,3]]class MyEncoder(json.JSONEncoder):    def default(self,obj):        #python to json        if isinstance(obj, datetime.datetime):            json_str = {"datetime":obj.strftime("%Y-%m-%d %X")}            return json_str        return objclass MyDecoder(json.JSONDecoder):    def __init__(self):        json.JSONDecoder.__init__(self, object_hook=self.str2time)    def str2time(self,json_obj):        #json to python        if "datetime" in json_obj:            date_str,time_str = json_obj["datetime"].split(' ')            date = [int(x) for x in date_str.split('-')]            time = [int(x) for x in time_str.split(':')]            dt = datetime.datetime(date[0],date[1], date[2], time[0],time[1], time[2])            return dt        return json_obj# a = json.dumps(dt,default=time2str)a =MyEncoder().encode(dd)print a# [{"datetime": "2016-10-27 18:14:54"}, [1, 2, 3]]print MyDecoder().decode(a)# [datetime.datetime(2016, 10, 27, 18, 14, 54), [1, 2, 3]]

Pickle module

The pickle module of python implements all python data sequences and deserialization. Basically, the function usage is not much different from the JSON module, and the method is also dumps/dump and loads/load. CPickle is the C language version of the pickle module, which is relatively faster.

Different from JSON, pickle is not used for data transmission between multiple languages. it is used only for persistence of python objects or for object transfer between python programs, therefore, it supports all python data types.

The pickle deserialization object and the original object are equivalent copy objects, similar to deepcopy.

Dumps/dump serialization

from datetime import datetry:    import cPickle as pickle    #python 2except ImportError as e:    import pickle   #python 3src_dic = {"date":date.today(),"oth":([1,"a"],None,True,False),}det_str = pickle.dumps(src_dic)print det_str# (dp1# S'date'# p2# cdatetime# date# p3# (S'\x07\xe0\n\x1b'# tRp4# sS'oth'# p5# ((lp6# I1# aS'a'# aNI01# I00# tp7# s.with open(r"c:\pickle.txt","w") as f:    pickle.dump(src_dic,f)

Loads/load deserialization

from datetime import datetry:    import cPickle as pickle    #python 2except ImportError as e:    import pickle   #python 3src_dic = {"date":date.today(),"oth":([1,"a"],None,True,False),}det_str = pickle.dumps(src_dic)with open(r"c:\pickle.txt","r") as f:    print pickle.load(f)# {'date': datetime.date(2016, 10, 27), 'oth': ([1, 'a'], None, True, False)}

Differences between the JSON and pickle modules

1. JSON can only process basic data types. Pickle can process all Python data types.

2. JSON is used for character conversion between different languages. Pickle is used for Python program object persistence or object network transmission between Python programs. However, there may be differences in Python serialization in different versions.