Using MongoDB in Python applications

Python is a powerful programming language for many different types of applications in the development community. Many people know that it is a flexible language that can handle almost any task. So what kind of database do you need in a Python application that is as flexible as the language itself? That's nosql, like MongoDB.

English Original: Https://realpython.com/blog/python/introduction-to-mongodb-and-python

　　

1. SQL vs NoSQL

If you are not familiar with the concept of NoSQL, MongoDB is a NoSQL database. In recent years it has been more and more popular with the whole industry. NoSQL databases provide a very different way of retrieving and storing data from a relational database.

In the decades of NoSQL, SQL database is one of the only options developers seek to build large, scalable systems. However, more and more requirements require the ability to store complex data structures. This has driven the birth of a NoSQL database, which allows developers to store heterogeneous and unstructured data.

When it comes to database schema selection, most people ask themselves the last question, "SQL or NoSQL?" ”。 Both SQL and NoSQL have their strengths and weaknesses, and you should choose one of the best suited to your application needs. Here are some of the differences between the two:

Sql

• The model is the relationship type;

• The data is stored in the table;

• Applies to cases where each record is of the same type and has the same attributes;

• The storage specification requires a predefined structure;

• Adding new attributes means that you have to change the overall architecture;

• Acid transaction support;

Nosql

• Model is non-relational;

• Can store JSON, key-value equivalence (dependent on NoSQL database type);

• Not every record should have the same structure;

• When you add data with a new attribute, it does not affect the other;

• Support for ACID transactions, depending on the NoSQL database used;

• Consistency can be changed;

• Horizontal expansion;

There are many other differences between the two types of databases, but the above mentioned are some of the more important differences. Depending on your situation, using SQL database may be preferred, whereas in other cases nosql is the more obvious choice. When selecting a database, you should carefully consider the advantages and disadvantages of each database.

One of the benefits of NoSQL is that there are many different types of databases to choose from, and each has its own use case:

• Key-value Storage: DynamoDB

• Document Storage: COUCHDB,MONGODB,RETHINKDB

• Column storage: Cassandra

• Data structure: REDIS,SSDB

There are many more, but these are some of the more common types. In recent years, SQL and NoSQL databases have even begun to merge. For example, PostgreSQL now supports storing and querying JSON data, much like MongoDB. With this, you can implement MongoDB-like functionality with Postgres, but you still don't have the other advantages of mongodb (such as horizontal scaling and simple interfaces, etc.).

2. MongoDB

Now, let's shift our gaze to the focus of this article and clarify some of the specifics of MongoDB's situation.

MongoDB is a document-oriented, open-source database program that is platform Independent. MongoDB like some other NoSQL database (but not all!) Documents that use a JSON structure store data. This is the schema that makes the data very flexible and unwanted.

Some of the more important features are:

• supports a variety of standard query types, such as matching (), comparison (,), or regular expressions;

• can store almost any type of data, whether structured, partially structured, or even polymorphic;

• To expand and process more queries, simply add more machines;

• It is highly flexible and agile, allowing you to quickly develop applications;

• Being a document-based database means that you can store all the information about your model in a single document;

• You can change the schema of the database at any time;

• The functionality of many relational databases can also be used in MongoDB (such as indexing).

In terms of operation, there are quite a few features in MongoDB that are not available in other databases:

• Whether you need a standalone server or a complete standalone server cluster, MongoDB can scale as needed;

• MongoDB also provides load balancing support by automatically moving data on individual shards;

• It has automatic failover support, if the primary server down, the new master server will automatically start and run;

• MongoDB's Management Services (MMS) can be used to monitor and back up MONGODB infrastructure services;

• Unlike relational databases, you will save quite a lot of RAM due to memory-mapped files.

Although at first MongoDB seems to be a database to solve many of our problems, it is not without drawbacks. A common drawback of mongodb is the lack of support for acid transactions, and MongoDB supports acid transactions in specific scenarios, but not in all cases. At the single document level, acid transactions are supported (this is where most transactions occur). However, because of the distributed nature of MongoDB, transactions that process multiple documents are not supported.

MongoDB also lacks support for natural join queries. In MongoDB's view: Documents are intended to be all-encompassing, which means that, in general, they do not need to refer to other documents. In the real world, this is not always effective, because the data we use is relational. Therefore, many people think that MongoDB should be used as a supplemental database for a SQL database, but when you use MongoDB Yes, you will find that this is wrong.

3, Pymongo

Now that we've described what MongoDB is, let's take a look at how to actually use it in Python. The official driver Pymongo, released by the MongoDB developer, is presented here through some examples, but you should also review the complete documentation, as we cannot cover everything.

Of course the first thing is to install, the simplest way is pip :

pip install pymongo==3.4.0

Note: For a more comprehensive guide, please review the documentation installation/upgrade page and follow the steps to set it up

When you are finished setting up, start the Python console and run the following command:

>>> import pymongo

If no exception is raised, the installation is successful.

Establish a connection

To MongoClient establish a connection using an object:

from pymongo import MongoClientclient = MongoClient()

Using the code snippet above, the connection to the default host (localhost) and port (27017) will be established. You can also specify a host and/or use a port:

client = MongoClient(‘localhost‘, 27017)

or use the Mongourl format:

client = MongoClient(‘mongodb://localhost:27017‘)

Accessing the database

Once you have an instance of the connection MongoClient , you can access any database in the MONGO server. If you want to access a database, you can access it as a property:

db = client.pymongo_test

Or you can also use the dictionary as a way to access:

db = client[‘pymongo_test‘]

If your specified database is already created, it doesn't really matter. By specifying this database name and saving the data to it, you will automatically create the database.

Insert Document

Storing data in a database is as easy as calling just two lines of code. The first line specifies which collection you will use. In MongoDB terminology, a collection is a set of documents that are stored together in a database (the equivalent of a table of SQL). Collections and documents are similar to SQL tables and rows. The second line is to use the collection to insert the data Insert_one () method:

posts = db.postspost_data = {    ‘title‘: ‘Python and MongoDB‘,    ‘content‘: ‘PyMongo is fun, you guys‘,    ‘author‘: ‘Scott‘}result = posts.insert_one(post_data)print(‘One post: {0}‘.format(result.inserted_id))

We can even use Insert_one () to insert many documents at the same time, and if you have a lot of documents to add to the database, you can use Method Insert_many (). This method accepts a list parameter:

post_1 = {    ‘title‘: ‘Python and MongoDB‘,    ‘content‘: ‘PyMongo is fun, you guys‘,    ‘author‘: ‘Scott‘}post_2 = {    ‘title‘: ‘Virtual Environments‘,    ‘content‘: ‘Use virtual environments, you guys‘,    ‘author‘: ‘Scott‘}post_3 = {    ‘title‘: ‘Learning Python‘,    ‘content‘: ‘Learn Python, it is easy‘,    ‘author‘: ‘Bill‘}new_result = posts.insert_many([post_1, post_2, post_3])print(‘Multiple posts: {0}‘.format(new_result.inserted_ids))

You should see similar output:

One post: 584d947dea542a13e9ec7ae6Multiple posts: [    ObjectId(‘584d947dea542a13e9ec7ae7‘),    ObjectId(‘584d947dea542a13e9ec7ae8‘),    ObjectId(‘584d947dea542a13e9ec7ae9‘)]

Note: Do not worry, you are not the same as shown above. They are dynamic identities that are made up of the Unix era, machine identifiers, and other unique data when inserting data.

Retrieving documents

Retrieving a document can use the Find_one () method, for example to find a record of author as Bill:

bills_post = posts.find_one({‘author‘: ‘Bill‘})print(bills_post)运行结果:{    ‘author‘: ‘Bill‘,    ‘title‘: ‘Learning Python‘,    ‘content‘: ‘Learn Python, it is easy‘,    ‘_id‘: ObjectId(‘584c4afdea542a766d254241‘)}

You may have noticed that the objectid of this article is set to _id, which is a unique identity that can be used later. If you need to query multiple records, you can use the Find () method:

scotts_posts = posts.find({‘author‘: ‘Scott‘})print(scotts_posts)结果:<pymongo.cursor.Cursor object at 0x109852f98>

His main difference is that the document data is not returned directly to us as an array. Instead, we get an instance of a cursor object. This cursor is an iterative object that contains quite a few helper methods to help you work with the data. To get each document, simply iterate through the results:

for post in scotts_posts:    print(post)

4, Mongoengine

While Pymongo is very easy to use, overall it is a great wheel, but many projects using it may be too low level. In short, you have to write a lot of your own code to persist, retrieve, and delete objects. Pymongo provides a higher abstraction of a library that is mongoengine. Mongoengine is an Object document Mapper (ODM), which is roughly equivalent to an SQL-based object-relational mapper (ORM). The abstraction provided by Mongoengine is class-based, so all the models you create are classes. While there are quite a lot of Python libraries that can help you use Mongodb,mongoengine is a better one because it has a good combination of functionality, flexibility and community support.

To install using PIP:

pip install mongoengine==0.10.7

Connection:

from mongoengine import *connect(‘mongoengine_test‘, host=‘localhost‘, port=27017)

　　
different from Pymongo. Mongoengine the database name needs to be established.

Defining documents

Before you create a document, you need to define the fields in your document that you want to hold data. Similar to many other Orm, we will do this by inheriting the document class and providing the type of data we want:

import datetimeclass Post(Document):    title = StringField(required=True, max_length=200)    content = StringField(required=True)    author = StringField(required=True, max_length=50)    published = DateTimeField(default=datetime.datetime.now)

In this simple model, we have told Mongoengine that our post instance has title, content, author, published. Now the Document object can use this information to verify the data that we provide it.

So if we try to save the post without the title then it will throw a exception and let us know. We can even make further use of this and add more restrictions:

• Required: set must;

• Default: If no other value is given using the specified defaults

• Unique: Ensure that no other document in the collection has the same value for this field

• Choices: Ensure that the value of this field is equal to one of the given values in the array

Save document

Save the document to the database and we will use the Save () method. If the database already exists in the document, all changes will be made to the existing document at the atomic level. If it does not exist, however, then it will be created.

Here is an example of creating and saving a document:

post_1 = Post(    title=‘Sample Post‘,    content=‘Some engaging content‘,    author=‘Scott‘)post_1.save()       # This will perform an insertprint(post_1.title)post_1.title = ‘A Better Post Title‘post_1.save()       # This will perform an atomic edit on "title"print(post_1.title)

There are a few things to note when calling Save ():

• Pymongo performs validation when you call. Save (), which means that it checks the data to be saved based on the pattern you declare in the class, throws an exception if the pattern (or constraint) is violated, and does not save the data;

• Because MONGO does not support true transactions, there is no way to "roll back" as in SQL database. Save () call.

When your saved data does not have a title:

post_2 = Post(content=‘Content goes here‘, author=‘Michael‘)post_2.save()raise ValidationError(message, errors=errors)mongoengine.errors.ValidationError:ValidationError (Post:None) (Field is required: [‘title‘])

Attributes to an object

Using Mongoengine is Object-oriented, you can also add a method to your subclass document. For example, the following example, where a function modifies the default query set (all objects that return a collection). By using it, we can apply a default filter to the class and get only the desired object

class Post(Document):    title = StringField()    published = BooleanField()    @queryset_manager    def live_posts(clazz, queryset):        return queryset.filter(published=True)

Associating other documents

You can also use the Referencefield object to create a reference from one document to another. Mongoengine automatically processes references on access.

class Author(Document):    name = StringField()class Post(Document):    author = ReferenceField(Author)Post.objects.first().author.name

In the above code, using the document "foreign key", we can easily find the author of the first article. There are more field classes (and parameters) than are described here, so be sure to check the document fields for more information.
　　
From all of these examples, you should be able to see that Mongoengine is ideal for managing database objects for almost any type of application. These features make it easy to create an efficient and extensible program. If you are looking for more help with Mongoengine, be sure to check out their user guide.

?

Using MongoDB in Python applications