Build a higher-level query API: The right way to use Django ORM

Add by Zhj: The writer is the technical director of Dabapps, who argues that using the ORM Query method provided by Django directly in view is not good, and I disagree with that, maybe the author

This article is for Django beginners, so it's a bit verbose when it comes to explaining how the method evolves, at least method 1 is not necessary to say.

This article describes how to add method properties to the Queryset class. When the author writes this article, Django1.7 is still in development, not published. This feature is available in the Django1.7 version,

See https://docs.djangoproject.com/en/dev/releases/1.7/#calling-custom-queryset-methods-from-the-manager. Another: I have modified the translation.

English Original: http://www.dabapps.com/blog/higher-level-query-api-django-orm/

Original text: Http://www.oschina.net/translate/higher-level-query-api-django-orm

Translator: Naixjs, Fkkeee, Qing Feng Aria

Summary

In this article, I think that using Django's low-level ORM query methods (such as filter, order_by, etc.) directly in view is usually anti-pattern. As an alternative, we need to build on the model layer

Querying the API, which is not very easy to do in Django, but through a deep understanding of the inside of the ORM, I will show you some simple ways to achieve this.

Overview

When writing a Django application, we are accustomed to encapsulating the business logic and hiding the implementation details by adding a method to the model. This method seems to be very natural, and in fact it is also used in

In Django's built-in applications.

 from Import  = User.objects.get (pk=5) User.set_password ('super-sekrit') User.save ()

The Set_password here is a method defined in the model Django.contrib.auth.models.User, which hides the specific implementation of hashing the password. The corresponding code should look

Is this:

 from Import Make_password class User (models. Model):    # fields go here.    def Set_password (Self, Raw_password):         = Make_password (Raw_password)

The benefit of this is to make the code more readable, reusable, and robust.

We've done this in a separate example, and we'll use it in the example of getting database information.

To describe this approach, we used a simple app (Todo list) to illustrate. Note: This is an example because it is difficult to show a real example with a small amount of code.

Here is the models.py file:

 from  django.db import   modelspriority_choices  = [(1,  " high  ), (2, "   Low   " )]    Todo (models. Model): Content  = models. Charfield (Max_length=100 = models. Booleanfield (Default=false) owner  = models. ForeignKey ( " auth. User   " " priority  = models. Integerfield (choices=priority_choices, Default=1 

Imagine that we need to query all the incomplete, high-priority Todos of the current user. Here is the code:

def Dashboard (Request):     = Todo.objects.filter (        owner=request.user    ). Filter (        is_done=False    ). Filter (        priority =1    )    return'todos/list.html ' , {        'todos': Todos,    })

Note: This can be written as Request.user.todo_set.filter (Is_done=false, Priority=1). But remember, this is just an experiment.

Why is it so bad to write?

First, the code is lengthy. Seven lines of code to complete, the formal project, will be more complex.

Second, the disclosure implementation details. For example, we need to know that there is a Boolean field named Is_done in the model, and if you change the field type to one that has multiple allowed values, the code will not work.

Then there is the unclear intent, which is difficult to understand.

Finally, there will be duplicates in use. Example: You need to write a management command to send each user his own Todo list every week, and then you need to copy-paste the seven lines of code. It doesn't fit.

DRY (do not repeat yourself)

Let's summarize: the direct use of low-level ORM code is anti-pattern.

How to improve it?

Using Managers and Querysets

First, let's take a look at the concept first.

Django has two closely related structures associated with table-level operations: Managers and Querysets

The manager (an instance of Django.db.models.manager.Manager) is described as an interface that provides query database operations for the model. The manager is the gateway to the table-level features. Each of the model

Has a default manager, called objects.

Quesyset (Django.db.models.query.QuerySet) is a "collection of objects in a database." is essentially a lazy select query, you can also use filtering, sorting, etc. (filtered,ordered),

To restrict or modify the data being queried. Use it to create or manipulate django.db.models.sql.query.Query instances, and then convert the database back into SQL queries.

Ah? You don't get it?

As you dive into the ORM, you will understand the difference between manager and Queryset.

People are confused by the familiar manager interface, because he doesn't look that way.

The manager interface is a lie.

The Queryset method is callable by chaining. Each method that calls Queryset (for example, filter) returns a replicated queryset that waits for the next call. This is part of the smooth beauty of Django ORM.

All of the QuerySet's methods need to be re-implemented in the manager, making chaining calls possible through model.objects. These methods are only agents of the corresponding methods in the Queryset in the manager, by

Self.get_query_set (), as follows

classManager (object):#SNIP some housekeeping stuff.    defGet_query_set (self):returnQuerySet (Self.model, using=self._db)defAll (self):returnSelf.get_query_set ()defcount (self):returnSelf.get_query_set (). Count ()defFilter (self, *args, * *Kwargs):returnSelf.get_query_set (). Filter (*args, * *Kwargs)#And so on for 100+ lines ...

Let's go back to Todo list immediately and solve the query interface problem. The Django recommended method is to customize the manager subclass and add it to the models.

classIncompletetodomanager (models. Manager):defGet_query_set (self):returnSuper (Todomanager, self). Get_query_set (). Filter (is_done=False)classHighprioritytodomanager (models. Manager):defGet_query_set (self):returnSuper (Todomanager, self). Get_query_set (). Filter (Priority=1)classTodo (models. Model): Content= Models. Charfield (max_length=100)    #Other fields go here.Objects= Models. Manager ()#The default manager    #Attach our Custom managers:Incomplete =models. Incompletetodomanager () high_priority= Models. Highprioritytodomanager ()

You can also add multiple managers to the model, or redefine objects, or you can maintain a single manager and add a custom method.

Let's experiment with some of these methods:

Method 1: Multi-managers

classIncompletetodomanager (models. Manager):defGet_query_set (self):returnSuper (Todomanager, self). Get_query_set (). Filter (is_done=False)classHighprioritytodomanager (models. Manager):defGet_query_set (self):returnSuper (Todomanager, self). Get_query_set (). Filter (Priority=1)classTodo (models. Model): Content= Models. Charfield (max_length=100)    #Other fields go here.Objects= Models. Manager ()#The default manager    #Attach our Custom managers:Incomplete =models. Incompletetodomanager () high_priority= Models. Highprioritytodomanager ()

Our API looks like this:

>>> Todo.incomplete.all ()>>> Todo.high_priority.all ()

There are several problems with this method.

First, this way of implementation is rather verbose. You need to define a manager for each custom query.

Second, this will mess up your namespace. Because Django developers are accustomed to seeing model.objects as a table entry, this approach destroys the rule.

Third, non-chained invocation. Or use a low-level ORM code: Todo.incomplete.filter (Priority=1) or Todo.high_priority.filter (Is_done=false)

In conclusion, the use of multi-managers method is not the optimal choice.

Method 2:manager Method

Now, let's try another method that Django allows: multiple methods in a single custom manager

class Todomanager (models. Manager):    def  incomplete (self):        return self.filter (is_done=False)     def high_priority (self):         return self.filter (priority=1)class  Todo (models. Model):    = models. Charfield (max_length=100)    # Other fields go here.      = Todomanager ()

Our API now looks like this:

>>> Todo.objects.incomplete ()>>> Todo.objects.high_priority ()

This method is obviously better. It's not too cumbersome (there's only one manager Class) and it's easy to add more methods.

However, it is still not possible to call custom methods chained, because Todo.objects.incomplete () and Todo.objects.high_priority () all return instances of the Django Queryset class, so we

Cannot use Todo.objects.incomplete (). High_priority ().

Method 3: Customize the Queryset

Now that we're in a field that Django hasn't developed, it's not found in the Django documentation.

classTodoqueryset (models.query.QuerySet):defIncomplete (self):returnSelf.filter (is_done=False)defhigh_priority (self):returnSelf.filter (Priority=1)classTodomanager (models. Manager):defGet_query_set (self):returnTodoqueryset (Self.model, using=self._db)classTodo (models. Model): Content= Models. Charfield (max_length=100)    #Other fields go here.Objects= Todomanager ()

We can see the clues from the view code of the following calls:

>>> Todo.objects.get_query_set (). Incomplete () >>> Todo.objects.get_query_set (). High_priority ()#  (or)>>> Todo.objects.all (). Incomplete () >>> Todo.objects.all (). High_priority ()

It's almost done! And there's a lot more to it than the 2nd method, get the same benefits as the method 2, and the extra effect (a bit of drums ...). ), it finally can chain query!

>>> Todo.objects.all (). Incomplete (). High_priority ()

Yet it is not perfect enough. This custom manager is just a template, and all () has flaws, is not well-grasped in use, and more importantly incompatible, it makes our code look a bit weird.

Method 3a: Copy Django, Agent do everything

We simply redefine all queryset methods in the manager

QuerySet:

classTodoqueryset (models.query.QuerySet):defIncomplete (self):returnSelf.filter (is_done=False)defhigh_priority (self):returnSelf.filter (Priority=1)classTodomanager (models. Manager):defGet_query_set (self):returnTodoqueryset (Self.model, using=self._db)defIncomplete (self):returnSelf.get_query_set (). Incomplete ()defhigh_priority (self):returnSelf.get_query_set (). High_priority ()

This will better provide the API we want:

# yay!

But the code is redundant and does not conform to dry, every time you add a file to Queryset, or change an existing method tag, you must remember to make the same change in your manager, otherwise it may not work correctly.

Method 3b:django-model-utils

Python is a dynamic language, can we do it dry? The answer is yes, to help with a third-party application called Django-model-utils. Run pip install Django-model-utils,

And then......

 fromModel_utils.managersImportPassthroughmanagerclassTodoqueryset (models.query.QuerySet):defIncomplete (self):returnSelf.filter (is_done=False)defhigh_priority (self):returnSelf.filter (Priority=1)classTodo (models. Model): Content= Models. Charfield (max_length=100)    #Other fields go here.Objects= Passthroughmanager.for_queryset_class (Todoqueryset) ()

It's going to be much better. We just define the Queryset subclass and then append this custom Queryset subclass to our model through the Passthroughmanager class provided by Django-model-utils.

Passthroughmanager is implemented by __getattr__, and it automatically proxies them to Queryset when a method that objects does not exist is called. You need to be careful here, check to make sure we don't have a

There is no infinite recursion in some features (this is why I recommend using django-model-utils instead of writing it by myself).

How does this help?

Remember the view you defined earlier?

def Dashboard (Request):     = Todo.objects.filter (        owner=request.user    ). Filter (        is_done=False    ). Filter (        priority =1    )    return'todos/list.html ' , {        'todos': Todos,    })

Add a little change, it looks like this:

def Dashboard (Request):     = Todo.objects.for_user (        request.user    ). Incomplete (). High_priority ()    return'  todos/list.html', {        'todos': Todos,    })

I hope you can agree that the second version is easier, clearer and more readable than the first one.

Can Django help you?

The way to make this whole thing easier is already discussed in the Django Development mailing list, and here are some suggestions for Zachary Voase:

class Todomanager (models. Manager):    @models. Querymethod    def  Incomplete (query):        return Query.filter (Is_done=false)

By defining this simple decorating method, both the manager and the Queryset are able to magically make the unavailable method usable.

I personally do not fully endorse the method of using adorners. It skipped over the detailed information and felt a bit "hip-hop". The way I feel good is to add a querset subclass (not the manager subclass).

Or we'll think about it a step further. Back to the controversial review of Django's API design decisions, maybe we can get real, deeper improvements. Can you eliminate the difference between managers and Queryset?

(or at least make the distinction more pronounced)?

I'm pretty sure that, regardless of whether or not there has been such a large refactoring job before, this feature is bound to be in Django 2.0 or even later versions.

So, simply summarize:

It is not a good idea to use the source ORM query code in views and other advanced apps. Instead, use the Passthroughmanager in Django-model-utils to add our new custom Queryset API

To add to your model, this gives you the following benefits:

Verbose code is less and more robust.

Increase dry to enhance the level of abstraction.

Pushes the business logic that belongs to the model-level implementation.

Thanks for reading.