Create a more advanced query API: correct use of DjangoORM

Create a more advanced query API: correct use of DjangoORM method abstract

In this article, I will directly discuss the use of Django's low-level ORM query methods from the perspective of anti-pattern. As an alternative, we need to establish query APIs related to specific fields in the model layer that contains business logic. these APIs are not very easy to do in Django, however, I will tell you some simple ways to achieve this through in-depth understanding of the content and principles of ORM.

Overview

When writing a Django application, we are used to adding methods to the model to encapsulate business logic and hide implementation details. This method looks very natural, and is actually used in Django's built-in applications.

>>> from django.contrib.auth.models import User>>> user = User.objects.get(pk=5)>>> user.set_password('super-sekrit')>>> user.save()

Set_password is a method defined in the django. contrib. auth. models. User model. it hides the specific implementation of the hash operation on the password. The corresponding code should look like this:

   from django.contrib.auth.hashers import make_passwordclass User(models.Model):    # fields go here..    def set_password(self, raw_password):        self.password = make_password(raw_password)

We are using Django to create a general interface at the top of a specific field and a low-level ORM tool. On this basis, the abstract level is added to reduce interactive code. The advantage of this is to make the code more readable, reusable, and robust.

We have done this in a separate example. we will use it in the following example to obtain database information.

To describe this method, we use a simple app (todo list.

Note: This is an example. It is difficult to use a small amount of code to show a real example. Do not care too much about the inheritance of todo list, but focus on how to run this method.

The models. py file is as follows:

from django.db import models  PRIORITY_CHOICES = [(1, 'High'), (2, 'Low')]  class Todo(models.Model):    content = models.CharField(max_length=100)    is_done = models.BooleanField(default=False)    owner = models.ForeignKey('auth.User')    priority = models.IntegerField(choices=PRIORITY_CHOICES, default=1

Imagine that we will pass the data and create a view to present incomplete, high-priority Todos to the current user. Here is the code:

def dashboard(request):      todos = Todo.objects.filter(        owner=request.user    ).filter(        is_done=False    ).filter(        priority=1    )      return render(request, 'todos/list.html', {        'todos': todos,    })

Note: the request. user. todo_set.filter (is_done = False, priority = 1) can be written here ). But here is just an experiment.

Why is it hard to write this?

First, the code is lengthy. Seven lines of code can be completed, and the formal project will be more complex.

Second, implementation details are leaked. For example, if is_done in the code is BooleanField and its type is changed, the code cannot be used.

Then, the intention is unclear and hard to understand.

At last, there will be duplicates in use. For example, you need to write a line of command and send a todo list to all users every week through cron. at this time, you need to copy and paste seven lines of code. This does not match DRY (do not repeat yourself)

Let's make a bold guess: using the lower-level ORM Code directly is anti-pattern.

How can we improve it?

Use Managers and QuerySets

First, let's take a look at the concept.

Django has two closely related table-level operations: managers and querysets.

Manager (an instance of django. db. models. manager. Manager) is described as "a plug-in provided to Django by querying the database ". Manager is the entrance to ORM for table-level functions. Each model has a default manager called objects.

Quesyset (django. db. models. query. QuerySet) is a set of objects in the database ". It is essentially a SELECT query. you can also use filtered or ordered to restrict or modify the queried data. It is used to create or manipulate django. db. models. SQL. query. Query instances, and then query in real SQL through the database backend.

Ah? You still don't understand?

As you learn more about ORM, you will understand the difference between Manager and QuerySet.

People will be confused by the well-known Manager interface, because it does not look like that.

The Manager interface is a lie.

The QuerySet method is connectable. Each time you call the QuerySet method (such as filter), a copied queryset is returned, waiting for the next call. This is also part of the smoothness of Django ORM.

However, when Model. objects is a Manager, a problem occurs. We need to call objects as the start and link it to the QuerySet of the result.

So how does Django solve it?

All QuerySet methods are based on Manager. In this method

Let's immediately return to todo list to solve the query interface problem. The recommended method for Django is to customize the Manager subclass and add it to models.

Self. get_query_set () proxy, re-create a QuerySet. Class Manager (object): # SNIP some housekeeping stuff .. def get_query_set (self): return QuerySet (self. model, using = self. _ db) def all (self): return self. get_query_set () def count (self): return self. get_query_set (). count () def filter (self, * args, ** kwargs): return self. get_query_set (). filter (* args, ** kwargs) # and so on for 100 + lines...

You can add multiple managers in the model, or redefine objects. you can also maintain a single manager and add custom methods.

Let's experiment with these methods:

Method 1: multiple managers

class IncompleteTodoManager(models.Manager):    def get_query_set(self):        return super(TodoManager, self).get_query_set().filter(is_done=False)  class HighPriorityTodoManager(models.Manager):    def get_query_set(self):        return super(TodoManager, self).get_query_set().filter(priority=1)  class Todo(models.Model):    content = models.CharField(max_length=100)    # other fields go here..      objects = models.Manager() # the default manager      # attach our custom managers:    incomplete = models.IncompleteTodoManager()    high_priority = models.HighPriorityTodoManager()

This interface will be displayed in this way:

>>> Todo.incomplete.all()>>> Todo.high_priority.all()

This method has several problems.

First, this implementation method is cool. You need to define a class for each query custom function.

Second, this will disrupt your namespace. For Django developers, see Model. objects as the table entry. This will destroy the naming rules.

Third, it cannot be linked. In this way, you cannot combine managers to obtain incomplete and high-priority todos, or return to the lower-level ORM Code: Todo. incomplete. filter (priority = 1) or Todo. high_priority.filter (is_done = False)

To sum up, the multi-manager method is not the best choice.

Method 2: Manager method

Now, let's try other methods that Django allows: multiple methods in a single custom Manager

class TodoManager(models.Manager):    def incomplete(self):        return self.filter(is_done=False)      def high_priority(self):        return self.filter(priority=1)  class Todo(models.Model):    content = models.CharField(max_length=100)    # other fields go here..      objects = TodoManager()

Our API now looks like this:

>>> Todo.objects.incomplete()>>> Todo.objects.high_priority()

This method is obviously better. It is not too cumbersome (there is only one Manager class), and this query method reserves a namespace after an object. You can easily add more methods)

However, this is not comprehensive enough. Todo. objects. incomplete () returns a normal query, but Todo. objects. incomplete (). high_priority () cannot be used (). We are stuck in Todo. objects. incomplete (). filter (is_done = False), which is not used.

Method 3: Customize QuerySet

Now we have entered a field that is not yet open to Django. this content cannot be found in the Django document...

class TodoQuerySet(models.query.QuerySet):    def incomplete(self):        return self.filter(is_done=False)      def high_priority(self):        return self.filter(priority=1)  class TodoManager(models.Manager):    def get_query_set(self):        return TodoQuerySet(self.model, using=self._db)  class Todo(models.Model):    content = models.CharField(max_length=100)    # other fields go here..      objects = TodoManager()

We can see the clues from the View code called below:

>>> Todo.objects.get_query_set().incomplete()>>> Todo.objects.get_query_set().high_priority()>>> # (or)>>> Todo.objects.all().incomplete()

Almost done! This is more cumbersome than 2nd methods. the benefits of Method 2 are the same as the additional effects (click it to drum up...). it can finally be chained for query!

>>> Todo.objects.all().incomplete().high_priority()

However, it is not perfect. This custom Manager is just a sample, and all () is flawed. it is difficult to grasp in use, and more importantly, it is incompatible. it makes our code look a little weird.

>>> Todo.objects.all().high_priority()

Method 3a: copy Django and the proxy will do everything.

Now let's make the above "fake Manager API" discussion useful: we know how to solve this problem. We simply re-define all QuerySet methods in the Manager, and then proxy them to return our custom QuerySet:

class TodoQuerySet(models.query.QuerySet):    def incomplete(self):        return self.filter(is_done=False)      def high_priority(self):        return self.filter(priority=1)  class TodoManager(models.Manager):    def get_query_set(self):        return TodoQuerySet(self.model, using=self._db)      def incomplete(self):        return self.get_query_set().incomplete()      def high_priority(self):        return self.get_query_set().high_priority()

This can better provide the APIs we want:

Todo. objects. incomplete (). high_priority () # yay!

Except for the input and not DRY parts above, every time you add a file to QuerySet or change the existing method tag, you must remember to make the same changes in your Manager, otherwise, it may not work normally. This is a configuration issue.

Method 3b: django-model-utils

Python is a dynamic language. Can we avoid all modules? A third-party application named Django-model-utils is a little busy and a little unmanageable. Run pip install django-model-utils first, and then ......

from model_utils.managers import PassThroughManager  class TodoQuerySet(models.query.QuerySet):    def incomplete(self):        return self.filter(is_done=False)      def high_priority(self):        return self.filter(priority=1)  class Todo(models.Model):    content = models.CharField(max_length=100)    # other fields go here..      objects = PassThroughManager.for_queryset_class(TodoQuerySet)()

This is much better. We just defined the custom QuerySet subclass as before, and then added these querysets to our model through the PassThroughManager class provided by django-model-utils.

PassThroughManager is implemented by _ getattr _. it can block access to the "nonexistent methods" defined by django and automatically proxy them to QuerySet. Please be careful when checking that we do not have infinite recursion in some features (this is why I recommend using the methods provided by django-model-utils for continuous testing, instead of manually writing it again ).

How can this be done?

Do you remember the View code defined earlier?

def dashboard(request):      todos = Todo.objects.filter(        owner=request.user    ).filter(        is_done=False    ).filter(        priority=1    )      return render(request, 'todos/list.html', {        'todos': todos,    })

Let's make it look like this:

def dashboard(request):      todos = Todo.objects.for_user(        request.user    ).incomplete().high_priority()      return render(request, 'todos/list.html', {        'todos': todos,    })

I hope you can agree that the second version is easier, clearer, and more readable than the first version.

Can Django help?

The method that makes this whole thing easier has been discussed in the django development email list and a related ticket is obtained ?). Zachary Voase is recommended as follows:

class TodoManager(models.Manager):      @models.querymethod    def incomplete(query):        return query.filter(is_done=False)

With the definition of this simple decoration method, both Manager and QuerySet can magically make the unavailable method available.

I personally do not fully agree with the use of decoration-based methods. It skipped the detailed information and felt a little "Hip Hop ". I think it is better and easier to add a QuerSet subclass (instead of a Manager subclass.

Or let's think further. Return to review Django's API design decision in the dispute, maybe we can get real and deeper improvements. Can I not argue about the difference between Managers and QuerySet (at least clarify )?

I'm sure that, no matter whether there have been such a large refactoring job before, this function must be in Django 2.0 or a later version.

Therefore, a brief summary is as follows:

It is not a good idea to use the source ORM query code in views and other advanced applications. Instead, we use PassThroughManager in django-model-utils to add our newly added custom QuerySet API to your model. This gives you the following benefits:

The code is less and more robust.

Increase the DRY and abstract level.

Push the business logic to the corresponding domain model layer.