Discussion on the correct usage of JSON: Pyhong, MongoDB, JavaScript and Ajax_javascript techniques

About this article

This article mainly summarizes some of the problems that have been encountered in transmitting JSON data since the site was written and the solutions currently adopted. The website database uses MongoDB, the back end is Python, the front end uses "The half separates" the form the riot.js, the so-called half separates, is said that the first page data is renders directly to the HTML through the server side template engine, thus avoids the homepage two times loading question, but other dynamic content uses the Ajax Load. The whole process of data is passed through the JSON format, but in different links need to adopt different ways and encounter a number of different problems, this article is mainly to record and summarize.

1. What is JSON?

JSON (JavaScript Object notation) is a lightweight data exchange language designed by Douglas Kest Rockwell, whose predecessor XML may have been known earlier. Of course, JSON does not exist to replace XML, but it is smaller and more suitable for data delivery in Web development than XML (JSON is like XML to Lisp). As you can see from the name, the format of JSON conforms to the grammatical format of the "object" in the JavaScript language, in addition to JavaScript, many other languages have similar types, such as the dictionary in Python (dict), in addition to programming languages, some document-stored The NoSQL database also chooses JSON as its data storage format, such as MongoDB.

In general, JSON defines a markup format that makes it easy to switch between variable data and string text data in programming languages. The data structures described by JSON include the following:

Object: {Key:value}
List: [obj, obj,...]
String: "string"
Number: Numbers
Boolean value: True/false

After understanding the basic concepts of JSON, the following is a summary of several data interaction links in the above diagram.

2. Python <=> MongoDB

The interaction between Python and MongoDB is primarily supported by existing drive libraries, including Pymongo, Motor, and so on, and the interfaces provided by these drivers are very friendly, and we do not need to know any underlying implementations, as long as the Python native dictionary types are manipulated:

Import Motor 
client = Motor.motor_tornado. Motorclient () 
db = client[' Test ']
user_col = db[' user '] 
user_col.insert (dict ( 
name = ' Yu ',
is_ admin = True,
))

The only thing to note is that the index entries in MongoDB _id are stored through ObjectId ("572df0b78a83851d5f24e2c1"), and the corresponding Python object is Bson.objectid.ObjectId, Therefore, you need to use an instance of this object in the query:

From Bson.objectid import objectid 
user = Db.user.find_one (dict ( 
_id = Objectid ("572df0b78a83851d5f24e2c1")
))

3. Python <=> Ajax

The data exchange between the front end and the back end is often done through Ajax, when it encounters the first minor pit. In a previous article, I summed up a Python-coded pit, and we knew there was definitely no json/xml in the HTTP Pass, everything was binary data, but we could choose how to interpret the data in the front end by setting the content-in the Header Type, which is typically set to Content-type:application/json when you pass the JSON data, and in the latest version of Tornado, just write directly to the dictionary type:

# Handler
Async def post (self): 
user = await Self.db.user.find_one ({})
Self.write (user)

Then came the first error: Typeerror:objectid (' 572df0b58a83851d5f24e2b1 ') is not JSON serializable. Retrospective reason, although Tornado helps us simplify the operation, it is still necessary to go through a json.dumps (user) operation when writing a dictionary type like HTTP, and ObjectId type is illegal for json.dumps. So I chose the most intuitive solution:

Import JSON from 
bson.objectid import Objectid 
class Jsonencoder (JSON). Jsonencoder): 
def default (self, obj):
if Isinstance (obj, ObjectId): Return
str (obj) return
super (). Default (self, obj)
# Handler
Async def post (self): 
user = await Self.db.user.find_one ({})
Self.write (Jsonencoder.encode (user))

This time there will be no more mistakes, our own jsonencoder can deal with ObjectId, but another problem has arisen:

After jsonencoder.encode the dictionary type is converted to a string, and Content-type becomes text/html after being written to HTTP, when the front-end will assume that the received data is a string rather than the available JavaScript Object. Of course there is a further solution, that is, the front-end to another conversion:

$.post (API, {}, function (res) {
data = Json.parse (res);
Console.log (data._id);
})

The problem is temporarily resolved, and throughout the process the JSON transformation is like this:

Python ==> json.dumps ==> HTTP ==> JavaScript ==> json.parse 

As a result, the second problem comes when there are some special characters in the data, and Json.parse will get an error:

Json.parse ("{' abs ': ' \ n '}"); 
Vm536:1 uncaught syntaxerror:unexpected token ' in JSON at position 1 (...)

This is the problem in the face of the problems is only to solve the immediate errors caused by a series of changes brought about by the shortcomings. We traced up the chain above the JSON transformation to see if there was a better solution. Quite simply, by following the traditional rules, when there is a special case, change the rules of adaptation instead of changing the rules:

# Handler
Async def post (self): 
user = await Self.db.user.find_one ({})
user[' _id '] = str (user[' _id '])
Self.write (user)

Of course, if it's a list of more than one data, it needs to be further modified:

# DB
Async def get_top_users (self, n = m): 
users = []
async for user in Self.db.user.find ({}). Sort (' rank ',-1 ). Limit (n):
user[' _id '] = str (user[' _id '])
users.append (user) return to
users

4. Python <=> Html+riot.js

If the above problem can be solved by following the rules, then the next question is a rule-challenging story. In addition to the Ajax dynamic loading section, other data on the Web page is rendered through the backend template engine, which means hard-coding is HTML. They are just plain text files before the browser loads and parses the HTML file, and what we need is to directly use the data plug <script> tag directly when the browser is running JavaScript. This is not strictly a JSON application, but a direct conversion between Python's Dict and JavaScript's Object, which should be written in the usual way:

# Handler
Async def get (self): 
users = self.db.get_top_users ()
render_data = Dict (
users = Users
) C6/>self.render (' users.html ', **render_data)
<!--HTML + riot.js--> 
<app></app> 
< script> 
riot.mount (' app ', {
users: [
{% for user in users%}
{name: ' {{user[' name ']}} ', Is_admin : "{{user[' is_admin ']}}"},
{% end%}
],
})

This is true, but to solve the ObjectId () problem mentioned above still requires some extra processing (especially quotes). In addition, in order to solve the problem of ObjectId I also tried a more stupid method (before the Json.parse encountered errors):

# Handler
Async def get (self): 
users = self.db.get_top_users ()
render_data = Dict (
users = Jsonencoder.encode (users)
)
self.render (' users.html ', **render_data)
<!--HTML + riot.js--> 
<app></app> 
<script> 
riot.mount (' app ', {
users:JSON.parse (' {{users}} '),
})

In fact, as with the 3rd bar, the template engine rendering process is similar to the HTTP transport process, unlike a string variable in a template that is a pure value (without quotes), so you can completely render the variable in the form of a JavaScript script file without worrying about special characters (the following {% raw ...%} is the Tornado template used to prevent special symbols from being HTML-encoded syntax):

<!--HTML + riot.js--> 
<app></app> 
<script> 
riot.mount (' app ', {
users: {% Raw Users%})

Summarize

JSON is a very useful data format, but there are a lot of details to be aware of when switching between different locales. In addition, by following the traditional rules, when there is a special case, changing the rules of adaptation rather than trying to change the rules does not necessarily fit all the problems, but for those already recognized rules, do not easily challenge them.

The above is a small set to introduce the correct usage of JSON to discuss: Pyhong, MongoDB, JavaScript and Ajax related knowledge, hope to help everyone, if you want to learn more information please pay attention to cloud Habitat community website!