Celery+django+redis Usage Introduction

Version:

• celery==3.1.25
• django==1.8.6
• redis==2.10.6

Installation:

• Entering virtual environments (virtual environments are created differently, entering in different ways)
• Pip install celery==3.1.25 (with celery installation as an example, other installation methods are the same)

Operating Environment:

• WINDOW10 (celery4.0 does not support Windows later)
• Linux

Directory structure:

• The outermost test1 is the project name
• Inside the Test1 and the app sibling directory, there are some configurations (where celery.py is placed).
• App2 as an app in the project
• tasks.py in App2, this tasks.py is used only for APP2. If other apps also have celery tasks. You can create the same file name.

To run celery, several files are required:

• Test1 under the celery.py, The code is as follows (in the introduction):

1 from __future__ import absolute_import, unicode_literals
  2 import os
  3 from django.conf import settings
  4 from celery import Celery
  5
  6
  7 #Set up Django's configuration file
  8 os.environ.setdefault(‘DJANGO_SETTINGS_MODULE‘, ‘test.settings’)
  9 
10 # Create a celery instance
11 app = Celery(‘test1‘)
12
13 # Using a string here means the worker will not have to
14 # pickle the object when using Windows.
15 app.config_from_object(‘django.conf:settings‘)
16
17 # Search all tasks in app
18 app.autodiscover_tasks(lambda: settings.INSTALLED_APPS)
19
20 @app.task(bind=True)
21 def debug_task(self):
22 print(‘Request: {0!r}‘.format(self.request))
View Code 

View Code

• Test1 under the settings.py, The code is as follows:

1 """ original django settings content """
  2 
  3 #celery config
  4 #message middleware (using redis), message broker, for publishers to deliver messages to consumers
  5 BROKER_URL = ‘redis://127.0.0.1:6379’
  6 #Message result returns middleware (using redis) for storing task execution results
  7 CELERY_RESULT_BACKEND = ‘redis://127.0.0.1:6379’
  8 #allowed content type,
  9 CELERY_ACCEPT_CONTENT = [‘json‘]
Serialization of 10 #Tasks
11 CELERY_TASK_SERIALIZER = ‘json‘
12 #Serialization of task results
13 CELERY_RESULT_SERIALIZER = ‘json‘
14 #celery time zone, timed task use
15 CELERY_TIMEZONE = ‘Asia/Shanghai’
16 from datetime import timedelta
17 #Timed task processing, using the schedule, the task inside, writing the "path" of the task, schedule setting time, args setting parameters.
18 CELERYBEAT_SCHEDULE = {
19 ‘add_every_10_seconds‘: {
20 ‘task’: ‘app2.tasks.add’,
21 ‘schedule’: timedelta(seconds=10),
22 ‘args‘: (4,4)
twenty three },
twenty four } 

View Code

• Test1 under the __init__.py, The code is as follows:

1 from __future__ import absolute_import
2 from .celery import app as celery_app
3 # This is to ensure that celery is launched when django starts. 

View Code

• App2 under the tasks.py, this tasks file only for the APP2 use, other apps can add tasks.py files according to the actual situation, write the corresponding task code. the code is as follows:

 
 
1 # -*- coding:utf-8 -*-
2 from __future__ import absolute_import
3 from auto_model_platform.celery import app
4 
5 @app.task
6 def add(x, y):
7     time.sleep(30)
8     print("running...", x, y)
9     return x + y

View Code

Call a function within tasks.py:

• Called directly in a view function in app2.views.py, such as calling the Add (x, y) function in tasks.py. the code is as follows:

1 from app2.tasks import add
  2 
  3 class TestView(View):
  4 def get(self, request):
  5
  6 """Other logic"""
  7
  8 #celery processing other tasks (asynchronous processing), the following code, celery will process, django directly execute the other logic below
  9 r = add.delay(x,y)
10 task_id = r.id
11
12 """Other logic"""
13
14 return JsonResponse({"data":"123"}) 

View Code

• You can query the results of celery asynchronous processing through task_id

1 from test1.celery import app
  2 status = app.AsyncResult(task_id).status
  3 result = app.AsyncResult(task_id).result
  4
  5 #state has these several situations
  6 CELERY_STATUS = {
  7 ‘PENDING’: ‘Waiting to start’,
  8 ‘STARTED’: ‘Task starts’,
  9 ‘SUCCESS’: ‘Successful’,
10 ‘FAILURE’: ‘Failure’,
11 ‘RETRY’: ‘Retry’,
12 ‘REVOKED’: ‘Task Cancel’,
13 } 

View Code

The server starts the celery worker (consumer) task, and the scheduled task:

• Open the project's virtual environment and go to the project name directory, which runs under the same level directory as the app:
• Timed tasks: celery-a auto_model_platform worker-l Info--beat
• Worker task: celery-a auto_model_platform worker-l Info

Celery+django+redis Usage Introduction