PYTHON/MQ celery

Celery

Celery is a tool that manages the distributed task queue, which is not itself a task queue.

Celery common concepts include brokers, backend, workers, and tasks.

Brokers: The middleman is the place where celery store/fetch products, that is, the task queue, common Rabbitmq/redis/zookeeper.

Backend: Also known as result stores, store run results, common redis/memcached.

Workers:celery the worker, pulls the task out of the queue and executes it, sending the result to backend.

Tasks: Task.

Simple example

# tasks.pyfrom celery import Celeryapp = Celery(‘tasks‘,                     backend=‘redis://[host]:6379/0‘,                     broker=‘amqp://[user]:[password]@[host]:5672‘)@app.taskdef add(x, y):    return x + y# 命令行启动worker，此时broker中还没有任务celery -A tasks worker --loglevel=info# 向broker发送任务并获取结果from tasks import addresult = add.delay(2, 3)while not result.ready():    time.sleep(1)print(result.get())>>>: 5

In the example, the app is the celery object, which defines the name of the object (for command line startup), the store, and the task queue.
The Add method is wrapped with app.task to indicate that the function is a task performer.
Executes the Add.delay () method, the parameter is passed to the performer, and the result is returned, and when Result.ready () is true, the result is returned to the store, which can be obtained by Result.get ().

The difference from Pika

Pika is used to connect RABBITMQ python module, RABBITMQ itself only storage function, and no task distribution scheduling function, celery is used to do task assignment, of course, can also write a dispatch code for Pika, but there is a ready celery, as long as the use of the line.

PYTHON/MQ celery