How to achieve Django scale: looking for bottlenecks

Offering: Zstack Cloud computing Series Tutorials

This tutorial is the first of three Django scale series. Content Introduction

Django is an excellent platform based on Python designed to build modern web applications. One of its biggest strengths is helping developers get their work done faster.

You may have built your own applications and deployed them, but then you need to load a large number of them and accept access from many users at the same time. In this case, its speed will undoubtedly be seriously affected.

This is a common problem, and fortunately, we have a variety of tools to deal with.

First, let's look at a few more obvious questions: using a real database

In the local development work, it is difficult for us to bypass the SQLite3. This database is likely to be used in virtual servers unless you are cautious.

SQLite3 is not as scalable as MySQL and PostgreSQL, such as multi-user, especially when dealing with a large number of writes (which means writing to a database if you are using a session).

If you are using a VPS with low memory capacity, such as only the droplet of MB, then I recommend that you use MySQL. If you have more memory (more than 2 GB), you should use PostgreSQL because it is more familiar to most Django developers. Disable debug mode

Debug mode is of course very important for local development, but it also slows down the production server. You can review your virtual server's settings.py file and make sure that the Debug option is set to false. Also, make sure Template_debug is also set to Fales, otherwise it will enable the debug feature. Use the Debug toolbar to troubleshoot performance issues

On the local development computer (not on the production server), enable the Django Debug toolbar to find specific issues.

To enable this toolbar, you first need to install the Django-debug-toolbar module, and then add an entry to the Middleware_classes directory:

Middleware_classes = (
# ...
' Debug_toolbar.middleware.DebugToolbarMiddleware ',
# ...
)

You also need to create a internal_ips variable and add your own IP address. If local development is in progress, our IP address may be 127.0.0.1, then add the following line to the settings.py:

Internal_ips = (' 127.0.0.1 ',)

Finally, add Debug_toolbar as the last entry Installed_apps:

Installed_apps = (
# ...
' Debug_toolbar ',
)

Please refer to the installation documentation for more details about configuration and optional recommendation.

Keep in mind that you should never apply these changes in a production environment. (unless there is a clear reason.) ）

Now you should be able to browse the site from the periphery of the page to find black panels. If you enjoy looking at the status and numbers associated with each detail, it will help.

After building a few Django apps, I recommend that you pay more attention to the SQL section in the panel, because this is often a problem-prone area.

One of the advantages/problems of Django is that when a query is executed, its associated fields often have a deferred load condition. This means that Django is often unwilling to take the initiative to perform a join. If you end up needing the relevant fields, Django will do an extra query to meet the requirements.

If you do need related fields, this may cause the number of SQL queries to become n+1. For example, suppose you want to replace a version of Twitter. We create a set of push-text patterns, and each pattern is associated with user mode. On the home page, we can list the latest 30 tweets and their creators. In this case, Django needs to do at least 31 SQL queries. 1 of them get a push list, and another 1 times for each query completed user name retrieval.

The problem with this solution is select_related. This is a very simple query modification mechanism that causes Django to perform a join when fetching data. You should use it to find any known related fields.

Simply modify the query as follows:

Entry.objects.get (id=5)

Modified to:

Entry.objects.select_related (). Get (id=5)

Please read the documentation and ensure that the mechanism is used only if necessary. Summary

In my experience, the above approach solves most site performance problems, and can only be done with a little code tuning and configuration changes.

This paper originates from Digitalocean Community. The original English: How to Scale django:finding the bottleneck by Matthew Nuzum

Translation: DIRADW