Build queries using ORM in Python's Django framework Api_python

Summary

In this article, I will discuss directly the use of Django's low-level ORM query methods in a reverse mode perspective. As an alternative, we need to build a query API that is relevant to a particular domain in the model layer that contains the business logic, which is not very easy to do in Django, but with a deep understanding of the fundamentals of ORM, I'll tell you some simple ways to achieve this.

Overview

When writing Django applications, we are used to adding methods to the model to encapsulate business logic and hide implementation details. This approach seems very natural, and in fact it is also used in Django's built-in applications.

>>> from django.contrib.auth.models import User
>>> user = User.objects.get (pk=5)
>> > User.set_password (' super-sekrit ')
>>> user.save ()

The Set_password here is a method defined in the Django.contrib.auth.models.User model that hides the concrete implementation of hashing the password. The corresponding code should look like this:

From django.contrib.auth.hashers import Make_password
 
class User (models. Model):
 
  # fields Go ...
 
  def set_password (self, Raw_password):
    Self.password = Make_password (Raw_password)

We are using Django to build a generic interface at the top of a particular domain, low level ORM tools. On this basis, increase the level of abstraction, reduce the interaction code. The advantage of this is that the code is more readable, reusable, and robust.

We've done this in a separate example, and we'll use it in the example of getting database information.

To describe this method, we used a simple app (Todo list) to illustrate.

Note: This is an example. Because it's hard to show a real example with a small amount of code. Don't take too much care todo list to inherit from him, but focus on how to make this method work.
Here is the models.py file:

From django.db import models
 
priority_choices = [(1, ' High '), (2, ' low ')]
 
class Todo (models. Model):
  content = models. Charfield (max_length=100)
  Is_done = models. Booleanfield (default=false)
  owner = models. ForeignKey (' auth. User ')
  priority = models. Integerfield (Choices=priority_choices, default=1

Imagine that we are going to pass this data and create a view to show the current user an incomplete, high-priority Todos. Here is the code:

def dashboard (Request):
 
  Todos = Todo.objects.filter (
    owner=request.user
  ). Filter (
    Is_done=false
  ). Filter (
    priority=1
  ) return
 
  render (request, ' todos/list.html ', {
    ' Todos ': Todos,
  })

Note: This can be written as Request.user.todo_set.filter (Is_done=false, Priority=1). But this is just an experiment.

Why is it so bad to write?

First, the code is verbose. Seven lines of code to complete, in a formal project, will be more complex.

Second, the disclosure of implementation details. For example, the code in the Is_done is Booleanfield, if the change of his type, the code can not be used.

And then, the intention is not clear, it is difficult to understand.

Finally, there will be duplication in use. For example: You need to write a line of commands that, through Cron, is sent every week to all users a todo list, where you need to copy-paste seven lines of code. This does not conform to dry (do not repeat yourself)

Let's make a bold guess: the direct use of low-level ORM code is in reverse mode.
How to improve it?

Using managers and Querysets
first, let's take a look at the concepts first.

Django has two closely related compositions for table-level operations: Managers and Querysets

The manager (an instance of Django.db.models.manager.Manager) is described as "supplied to Django through a query database". The manager is the table-level feature leading to the ORM gate. Each model has a default manager, called objects.
Quesyset (Django.db.models.query.QuerySet) is a "collection of objects in the database." is essentially a select query, you can also use filtering, sorting, etc. (filtered,ordered) to limit or modify the data that is queried. Used to create or manipulate django.db.models.sql.query.Query instances, and then query through the database backend in real SQL.

Ah? Don't you get it?

As you delve into ORM, you'll understand the difference between manager and Queryset.

People are confused by the familiar manager interface because he doesn't look like that.

The manager interface is a lie.

The Queryset method is link-able. Every time a method called Queryset, such as filter, returns a replicated queryset waiting for the next call. This is also part of the smooth beauty of Django ORM.

But when Model.objects is a manager, there is a problem. We need to call objects as the start, and then link to the results of the queryset up.

So how does Django solve it?

The interface lies thus exposed, and all queryset methods are based on the manager. In this method, through the proxy of Self.get_query_set (), recreate a

QuerySet.
 
class Manager (object):
 
  # snip some housekeeping stuff.
 
  def get_query_set (self): return
    QuerySet (Self.model, using=self._db)
 
  def All (self): return
    self.get_ Query_set ()
 
  def count (self): return
    Self.get_query_set (). Count ()
 
  def filter (self, *args, **kwargs): return
    Self.get_query_set (). Filter (*args, **kwargs) # and "For
 
  100+ lines ...

For more code, refer to the manager's resource file.

Let's immediately go back to the Todo list and solve the query interface problem. The Django recommendation is to customize the manager subclass and add it to the models.

You can also add multiple managers to the model, or redefine objects, or maintain a single manager, adding custom methods.

Now let's experiment with these methods:

Method 1: Multiple managers

Class Incompletetodomanager (models. Manager):
  def get_query_set (self): return
    super (Todomanager, self). Get_query_set (). Filter (Is_done=false)
 
class Highprioritytodomanager (models. Manager):
  def get_query_set (self): return
    super (Todomanager, self). Get_query_set (). Filter (Priority=1)
 
class Todo (models. Model):
  content = models. Charfield (max_length=100) # Other fields go to here
  .
 
  objects = models. Manager () # The default manager
 
  # Attach our Custom managers:
  incomplete = models. Incompletetodomanager ()
  high_priority = models. Highprioritytodomanager ()

This interface will be presented in such a way:

>>> Todo.incomplete.all ()
>>> Todo.high_priority.all ()

There are several problems with this method.

First, this way of implementation is verbose. You want to define a class for each query customization feature.

Second, this will mess up your namespaces. Django Developers bar model.objects as a table entry. Doing so will break the naming convention.

Third, not linked. This does not combine managers, get an incomplete, high-priority Todos, or go back to a low-level ORM code: Todo.incomplete.filter (Priority=1) or Todo.high_ Priority.filter (Is_done=false)
In summary, the method of using multiple managers is not the best choice.

Method 2:manager Method

Now, let's try other Django-allowed methods: multiple methods in a single custom manager

Class Todomanager (models. Manager):
  def incomplete (self): return
    self.filter (is_done=false)
 
  def high_priority (self):
    return Self.filter (Priority=1)
 
class Todo (models. Model):
  content = models. Charfield (max_length=100) # Other fields go to here
  .
 
  objects = Todomanager ()

Our API now looks like this:

>>> Todo.objects.incomplete ()
>>> Todo.objects.high_priority ()

This method is obviously better. It doesn't have a lot of baggage (only one manager Class) and this query method is a good way to reserve the namespace after the object. (To add more methods to the image and convenience)
But it's not comprehensive enough. Todo.objects.incomplete () returns a normal query, but we cannot use Todo.objects.incomplete (). High_priority (). We are stuck in Todo.objects.incomplete (). Filter (Is_done=false), not used.

Method 3: Custom Queryset

Now that we've entered a domain where Django is not yet open, this is not found in Django documents ...

Class Todoqueryset (Models.query.QuerySet):
  def incomplete (self): return
    self.filter (is_done=false)
 
  def high_priority (self): return
    self.filter (priority=1)
 
class Todomanager (models. Manager):
  def get_query_set (self): return
    todoqueryset (Self.model, using=self._db)
 
class Todo (models. Model):
  content = models. Charfield (max_length=100) # Other fields go to here
  .
 
  objects = Todomanager ()

We can see the clue in the view code from the following call:

>>> Todo.objects.get_query_set (). Incomplete ()
>>> Todo.objects.get_query_set (). High_ Priority ()
>>> # (OR)
>>> Todo.objects.all (). Incomplete ()
>>> Todo.objects.all (). High_priority ()

It's almost done! This is no more burdensome than the 2nd method, but it has the same benefits and extra effects as the Method 2 (a little drum ...). ), it finally can be chained query!

>>> Todo.objects.all (). Incomplete (). High_priority ()

But it's not perfect enough. This custom manager is just a template, and all () has flaws, is not easy to use, and more importantly is incompatible, it makes our code look a little weird.

Method 3a: Copy Django, Agent do everything

Now we've got the above "fake manager API" discussion useful: we know how to solve this problem. We simply redefine all the Queryset methods in the manager and then delegate them back to our custom Queryset:

Class Todoqueryset (Models.query.QuerySet):
  def incomplete (self): return
    self.filter (is_done=false)
 
  def high_priority (self): return
    self.filter (priority=1)
 
class Todomanager (models. Manager):
  def get_query_set (self): return
    todoqueryset (Self.model, using=self._db)
 
  def incomplete (self ): Return
    Self.get_query_set (). Incomplete ()
 
  def high_priority (self): return
    Self.get_query_set (). High_priority ()

This better provides the APIs we want:

>>> Todo.objects.incomplete () high_priority () # yay!

Every time you add a file to Queryset or change an existing method tag, you must remember to make the same changes in your manager, or it may not work correctly, except for the input section above and very dry. This is a problem with configuration
Method 3b:django-model-utils

Python is a dynamic language. We'll be able to avoid all the modules? A little bit of help from a third-party application called Django-model-utils would be a bit out of control. Run pip install django-model-utils First, then ...

From model_utils.managers import Passthroughmanager
 
class Todoqueryset (Models.query.QuerySet):
  def Incomplete (self): return
    self.filter (is_done=false)
 
  def high_priority (self): return
    Self.filter ( Priority=1)
 
class Todo (models. Model):
  content = models. Charfield (max_length=100) # Other fields go to here
  .
 
  objects = Passthroughmanager.for_queryset_class (Todoqueryset) ()

It's much better. We simply defined the custom Queryset subclass as before, and then attached these queryset to our model through the Passthroughmanager class provided by Django-model-utils.

Passthroughmanager is implemented by __GETATTR__, which prevents access to Django-defined "nonexistent methods" and automatically proxies them to Queryset. Be careful here, check to make sure we don't have infinite recursion in some of the features (this is why I recommend using the Django-model-utils method, rather than writing it by hand).

What's the help in doing this?

Remember the view code defined earlier?

def dashboard (Request):
 
  Todos = Todo.objects.filter (
    owner=request.user
  ). Filter (
    Is_done=false
  ). Filter (
    priority=1
  ) return
 
  render (request, ' todos/list.html ', {
    ' Todos ': Todos,
  })

Add a little change and we'll make it look like this:

def dashboard (Request):
 
  Todos = Todo.objects.for_user (
    request.user
  ). Incomplete (). High_priority () Return
 
  render (request, ' todos/list.html ', {
    ' Todos ': Todos,
  })

I hope you can agree. The second version is simpler, clearer, and more readable than the first.
Can Django help?

The way to make this whole thing easier, has been discussed in the Django Development mailing list, and got a related bill (associated ticket name better?). )。 Zachary Voase suggested the following:

Class Todomanager (models. Manager):
 
  @models. Querymethod
  def incomplete (query): Return
    query.filter (is_done=false)

With this simple definition of decoration, both the manager and the Queryset make the unavailable methods magically available.

I personally do not agree with the use of decorative methods. It skips over the detailed information and feels a bit "hip hop". I feel a good way to add a querset subclass (instead of a manager subclass) is a better, simpler way.
Or we think more. Back to the decision to revisit Django's API design in the controversy, maybe we can get a real deeper improvement. Can you stop arguing between managers and queryset (at least for clarification)?

I'm pretty sure that, regardless of whether you've ever had such a large refactoring job before, this feature will have to be in the Django 2.0 or even later version.

So, simply summarize:

It is not a good idea to use the source ORM query code in views and other advanced applications. Instead, use the Passthroughmanager in Django-model-utils to add our new custom Queryset API into your model, which gives you the following benefits:

• Verbose code is less and more robust.
• Increase dry, and enhance the level of abstraction.
• Pushes the business logic that belongs to the corresponding domain model layer.

Thanks for reading!