Python's celery used in flask

Now continue to learn how to use celery in an integrated framework.

Using celery in flask

Integrating celery in flask requires two points:

1. The name of the instance object that created the celery must be the name of the Flask application app, or the celery startup will fail;

2. The celery must be able to load the initialization file successfully.

Celery initialization in flask

Since the running of the celery process and the running of the flask process are independent of each other, in the framework we want to use only one copy of the configuration file, which simplifies the work of the configuration.

from celery import Celeryfrom flask import Flaskapp = Flask(__name__)def make_celery(app):    celery = Celery(app.import_name)    celery.conf.update(app.config)    return celerycelery = make_celery(app)# celery的配置文件在app的setting中；

Problem: The above procedure can be initialized celery initialization, but when the use of Factory mode to create the app, celery initialization becomes difficult;

from celery import Celeryfrom flask import Flaskcelery = Nonedef make_celery(app):    celery = Celery(app.import_name)    celery.conf.update(app.config)    return celerydef create_app(config_name)    app = Flask(__name__)    config_class = config_map[config_name]    app.config.from_object(config_class)    # 初始化celery    global celery    celery = make_celery(app)

• Problem: Because the program initialization is not possible to create the App object, so celery boot must first import the App object in the tasks in order to complete the initialization, may lead to loop import errors;

• Solution: Introduce Flask-celery-helper, help us initialize celery object;

• Installing Flask-celery-helper

pip install Flask-Celery-Helper

• All additional objects that need to be initialized are separated out in a separate PY module.

# extensions.pyfrom flask_celery import Celery# 创建celery的实例celery = Celery()# __init__.pyfrom extensions import celerydef create_app(config_name)    app = Flask(__name__)    config_class = config_map[config_name]    app.config.from_object(config_class)    # 对celery进行初始化操作，可以将celery的配置写在app的配置中    celery.init_app(app=app)# tasks.pyfrom extensions import celery@celery.task()def add(x,y):    return x + y

Attention

• Flask-celery-helper official currently only support to python3.4, but landlord use py3.6 also no problem, Flask-celery-helper does not support celery4.x version, otherwise error, so need to use celery3.x version;

• Calling the Task method

# app.pyfrom tasks import add@app.route("/index")def index():    """一个测试的实例"""    print(add(3+6)) # add函数也能做普通的函数使用    add.apply_async(args=[5,7], queue=‘eegqueue‘) # 发送异步任务，指定队列    return "ok!"

Start celery

The configuration of the Flask app needs to be loaded before starting celery, so an app object needs to be created for celery use.

# run_celery.pyimport create_appflask_app = create_app("develop") # 创建app的同时，对celery完成了加载配置的工作from extensions import celery # 此时的celery对象已经在上下文中完成初始化

# 找到celery实例的位置，指定worker，指定接收某个队列的消息，如果不指定则接收所有队列的消息celery -A run_celery.celery worker -Q eegqueue --loglevel=info

• Reference:

• https://pypi.org/project/Flask-Celery-Helper/#description

Python's celery used in flask