Python3 JSON and Jsonpath

1.1 JSON Introduction

JSON is simply called objects and arrays in JavaScript, so these two structures are objects and arrays of two structures that can represent a variety of complex structures.

1. Object: Object in JS is represented as {} content, data structure is {key:value, Key:value, ...} The structure of the key-value pairs, in object-oriented language, key is the property of the object, value is the corresponding property value, so it is easy to understand that the value method is the object. Key Gets the property value, the type of the property value can be a number, a string, an array, an object.

2. Array: The array in JS is the bracketed [] content, the data structure for ["Python", "JavaScript", "C + +", ...], the value is the same as in all languages, using index get, the type of field value can be number, string, array, object several.

1.2 Four functions of the JSON module

The JSON module provides four functions: dumps, dump, loads, load, for converting between string and Python data types.

1. Json.loads ()

The json.loads () implementation converts the JSON format string decoding into a Python object from JSON to Python in a type conversion comparison as follows:

Json	Python Data Type
Json format string decoding translates into Python objects
Object	Dict
Array	List
Number (int)	Int,long
Number (real)	Float
True	True
False	False
Null	None

Example:

Import JSON

# data

Js_list = ' [1, 2, 3, 4] '

Js_dict = ' {' name ': ' Listen to the Sea ', ' age ': ' Year 18 '} '

# types prior to data conversion

Print (Type (js_list))

Print (Type (js_dict))

# json.loads () Implementation converts JSON format string decoding to Python object

List = Json.loads (js_list)

Dict = Json.loads (js_dict) # JSON data is automatically stored by Unicode

Print ('--------------------------')

# post-conversion display

Print (List)

Print (Dict)

#数据转换后的类型

Print (Type List)

Print (Type (Dict))

Operation Result:

<class ' str ' >

<class ' str ' >

--------------------------

[1, 2, 3, 4]

{' Age ': ' yearly ', ' name ': ' Listen to the Sea '}

<class ' list ' >

<class ' Dict ' >

2. Json.dumps ()

The Json.dumps () implementation converts the Python type to a JSON string, and returns a Str object that converts a Python object encoding into a JSON string, from the Python primitive type to the JSON-type conversions:

Python Data Type	Json
python type into a JSON string
Dict	Object
List	Array
int, long, float	Number
STR, Unicode	String
True	True
False	False
None	Null

Example:

Import JSON

LISTSTR = [1, 2, 3, 4]

Tuplestr = ("Python3", "selenium3", "Appium", "Java")

Dictstr = {"Name": "Listen to the Sea", "Age": "Year 18"}

# Pre-conversion format

Print (Type (LISTSTR))

Print (Type (TUPLESTR))

Print (Type (DICTSTR))

# convert the Python type to a JSON string and return a str object.

Js_strlist=json.dumps (LISTSTR)

Js_tuplestr = Json.dumps (TUPLESTR)

# json.dumps () The ASCII encoding is used by default when serializing, add parameter ensure_ascii=false disable ASCII encoding, press UTF-8 encoding.

Js_dictstr=json.dumps (Dictstr,ensure_ascii=false)

Print ("---------------")

# Print converted data display.

Print (js_strlist)

Print (JS_TUPLESTR)

Print (JS_DICTSTR)

# post-converted format

Print (Type (js_strlist))

Print (Type (JS_TUPLESTR))

Print (Type (JS_DICTSTR))

Operation Result:

<class ' list ' >

<class ' tuple ' >

<class ' Dict ' >

---------------

[1, 2, 3, 4]

["Python3", "selenium3", "Appium", "Java"]

{"Name": "Listening to the Sea", "Age": "Year 18"}

<class ' str ' >

<class ' str ' >

<class ' str ' >

Note: The ASCII encoding is used by default when Json.dumps () is serialized, and the Add parameter Ensure_ascii=false disables ASCII encoding, which is displayed by Utf-8 encoding.

3. Json.dump ()

Serializes a python built-in type into a JSON object and writes a file.

Example:

Import JSON

Liststr = [{"A1": "1"}, {"B1": "2"}]

Json.dump (LISTSTR, open ("Liststr.json", "W"), Ensure_ascii=false)

Dictstr = {"A2": "3", "B2": "4"}

Json.dump (DICTSTR, open ("Dictstr.json", "W"), Ensure_ascii=false)

Operation Result:

Generates a Liststr.json file and Dictstr.json 2 files in the current directory.

4. Json.load ().

Read the JSON-like string elements in the file into a python type.

Example:

An example of a request message in the interface document, we convert the JSON form of the message in this example into a python type.

1. Create a new file called "Interface request message. JSON" in the current directory.

Code implementation:

Import JSON

Js_t_py = json.load (Open ("./interface Request message. JSON", encoding= "Utf-8"))

Print (js_t_py)

Print (Type (js_t_py))

Operation Result:

1.3 JsonPath

JsonPath is an information extraction class library that extracts specified information from JSON documents and provides multiple language implementations, including: Javascript, Python, PHP, and Java.

JsonPath, for JSON, is equivalent to XPATH for XML.

: Https://pypi.python.org/pypi/jsonpath

Official Use note document: Http://goessner.net/articles/JsonPath

Installation of 14.3.1 Jsonpath

Installation method One: Click on the Download URL link to download Jsonpath, after extracting the Python setup.py install.

Installation method Two: Install directly using the PIP install Jsonpath command.

14.3.2 JsonPath Official Example

{"Store": {

"Book": [

{"category": "Reference",

"Author": "Nigel Rees",

"title": "Sayings of the Century",

"Price": 8.95

},

{"category": "Fiction",

"Author": "Evelyn Waugh",

"title": "Sword of Honour",

"Price": 12.99

},

{"category": "Fiction",

"Author": "Herman Melville",

"title": "Moby Dick",

"ISBN": "0-553-21311-3",

"Price": 8.99

},

{"category": "Fiction",

"Author": "J. R. R. Tolkien",

"title": "The Lord of the Rings",

"ISBN": "0-395-19395-8",

"Price": 22.99

}

],

"Bicycle": {

"Color": "Red",

"Price": 19.95

}

}

}

Comparison of 14.3.3 Jsonpath and XPath syntax

The JSON structure is clear, readable, complex, and very easy to match, and the following table corresponds to the use of XPath.

Xpath	JSONPath	Describe
/	$	Root node
.	@	Current node
/	. or[]	Take child nodes
..	Not supported	Take parent node, Jsonpath does not support
//	..	Ignore location, select all eligible conditions
*	*	Match all ELEMENT nodes
@	Not supported	JSON is not supported based on property access because JSON is a key-value recursive structure and is not required.
[]	[]	Iterator indicator (can be used to do simple iterative operations, such as array subscript, based on content selection, etc.)
|	[,]	Supports long selection in iterators.
[]	? ()	Supports filtering operations.
Not supported	()	Supports expression evaluation
()	Not supported	Grouping, Jsonpath not supported

Example syntax comparison.

Xpath	JSONPath	Example results
/store/book/author	$.store.book[*].author	The authors of all books in the store As follows: "Nigel Rees" "Evelyn Waugh" "Herman Melville" "J. R. R. Tolkien"
Author	$.. Author	All authors
/store/*	$.store.*	All things in store, which is some books and a red bicycle.
/store//price	$.store. Price	The price of everything in the store.
BOOK[3]	$.. BOOK[2]	The third book
Book[last ()]	$.. book[(@.length-1)] $.. Book[-1:]	The last book is in order.
Book[position () <3]	$.. book[0,1] $.. Book[:2]	The first and the books
BOOK[ISBN]	$.. Book[? ( @.isbn)]	Filter All books with ISBN number
BOOK[PRICE<10]	$.. Book[? ( @.price<10)]	Filter all books Cheapier than 10
//*	$.. *	All Elements in XML document. All members of the JSON structure.

1.4 Cases

Case:

Take the hook net City JSON file Http://www.lagou.com/lbs/getAllCitySearchLabels.json as an example to get all the cities.

Code implementation:

Importjsonpath

url = ' Http://www.lagou.com/lbs/getAllCitySearchLabels.json '

Headers ={"user-agent": "mozilla/5.0 (Windows NT 10.0; Win64; x64) applewebkit/537.36 (khtml, like Gecko) chrome/54.0.2840.99 safari/537.36 "}

R =requests.get (url,headers=headers)

HTML =r.text

# convert JSON format string to Python object

Jsonobj =json.loads (HTML)

# starting from the root node, matching the name node

CityList =jsonpath.jsonpath (Jsonobj, ' $... Name ')

Print (CityList)

Print (Type (citylist))

#新建一个city the. json file and set the encoding format to Utf-8

FP =open (' City.json ', ' W ', encoding= "Utf-8")

Content =json.dumps (CityList, Ensure_ascii=false)

Print (content)

Fp.write (content)

Fp.close ()

Operation Result:

Precautions:

Json.loads () is to convert the JSON format string decoding to a Python object, and if there is an error in json.loads, be aware of the encoding of the decoded JSON character.

If the encoding of the passed-in string is not UTF-8, you need to specify the character-encoded parameter encoding

Datadict =json.loads (JSONSTRGBK);

Datajsonstr is a JSON string, assuming its encoding itself is non-UTF-8 words but GBK, then the above code will cause an error, instead of the corresponding:

Datadict =json.loads (JSONSTRGBK, encoding= "GBK");

If DATAJSONSTR specifies the appropriate encoding through encoding, but it also contains other encoded characters, you need to first convert DATAJSONSTR to Unicode, and then specify the encoding format to call Json.loads ()

Datajsonstruni= Datajsonstr.decode ("GB2312");

Datadict =json.loads (Datajsonstruni, encoding= "GB2312");

Python3 JSON and Jsonpath