Maintain persistent connection between django and the database

Recently I encountered a very bad problem. I wrote a background management system. Because it is a background management system, the usage frequency is not very high. When the django program is idle for a while, open the background system again, it becomes very slow, and then better. I checked many aspects, from the template engine to the request, to the django configuration, nginx, and so on, I did not find the reason. Although I have also checked whether it is the cause of the database, I did not find out why it was not in-depth enough.

Once, when dealing with permission issues, I kept tracing the source code. Because I used dbroute to control permissions, I finally located this db (specific directory: /usr/local/lib/python2.7/dist-packages/django/db/_ init __. py), the code in it has connection information, but I don't know it. After reading it, the chrysanthemum is tight. I am going! It turns out that django only connects to the database each time it queries, and immediately closes the database after the query is complete. I really don't know what the author thinks. Does it take a lot of time to create a connection for each query? The source code is as follows:

 django.conf  django.core  django.core.exceptions  django.db.utils  = (, , , , ,  settings.DATABASES  DEFAULT_DB_ALIAS   ImproperlyConfigured( %==      == load_backend(connection.settings_dict[ close_connection(**     django.db  conn                  reset_queries(** conn = _rollback_on_exception(** django.db  conn =

It is immediately closed after receiving a close signal. In fact, every django request is connected and queried once. I checked the information on the Internet. Someone solved the problem as follows:

Add the following code to the _ init _. py file in the django project:

 django.core  django.db signals.request_finished.disconnect(close_connection) 

The principle is the Signal object related to the last line of code. There is also a disconnect method, which is used to cancel the Signal Association.

I did it according to his implementation and found no. After several hours of idle, it still takes about 10 seconds to open the background. Because I use fastcgi to run django, maybe fastcgi suspends it when there is no dynamic movement of django, so it will re-establish the connection. If supervisord is used for running, it is estimated that it will not.

Since this method does not work, let's take a look at its source code and find that there are mysql, oracle, postgresql_psycopg2, sqlite3, etc. under the backends directory. So select msql to go and check the base. py file. It is found that django directly encapsulates MySQLdb. Every time a MySQLdb object is created, a connection is actually performed. Can I create multiple MySQLdb objects at a time like a thread pool? The answer is yes. Sqlalchemy has a pool to maintain a persistent connection to the Database, so you can directly change the Database in this file to the pool of sqlalchemy. Of course, we cannot modify the source code with brute force. This module is used to establish database connections, so it can be independent. In fact, it is very simple. You only need to modify the base. py file several places. The implementation method is as follows:

Copy all the files in the django/db/backends/mysql directory, put them under a libs/mysql project, and modify the base. py file. Find

 django.core.exceptions  ImproperlyConfigured( % e)

Add the following code:

 sqlalchemy = pool.manage(Database)

Basically. If you want to modify it again, find

self.connection = Database.connect(**kwargs)

Comment it out and add

            self.connection ==kwargs.get(, =kwargs.get(, 3306=kwargs[=kwargs[=kwargs[=kwargs[=

Modify the settings. py Database Configuration and change django. db. backends. mysql to libs. mysql. At this point, the world is much better. If the worm is on the brain, you can move to: Come and try again.