Celery--任務調度利器

時間 2019-12-05

標籤 celery 任務調度利器简体版

原文原文鏈接

Celery文檔: http://docs.jinkan.org/docs/celery/getting-started/first-steps-with-celery.htmlhtml

安裝celery和celery-with-redis

pip install Celery
pip install celery-with-redis

開始編寫task.py

# tasks.py
import time
from celery import Celery

celery = Celery('tasks', broker='redis://localhost:6379/0')

@celery.task
def sendmail(mail):
    print('sending mail to %s...' % mail['to'])
    time.sleep(2.0)
    print('mail sent.')

啓動Celery處理任務

$ celery -A tasks worker --loglevel=info

上面的命令行實際上啓動的是Worker，若是要放到後臺運行，能夠扔給supervisor。python

發送任務

>>> from tasks import sendmail
>>> sendmail.delay(dict(to='celery@python.org'))
<AsyncResult: 1a0a9262-7858-4192-9981-b7bf0ea7483b>

能夠看到，Celery的API設計真的很是簡單。而後，在Worker裏就能夠看到任務處理的消息：redis

[2013-08-27 19:20:23,363: WARNING/MainProcess] celery@MichaeliMac.local ready.
[2013-08-27 19:20:23,367: INFO/MainProcess] consumer: Connected to redis://localhost:6379/0.
[2013-08-27 19:20:45,618: INFO/MainProcess] Got task from broker: tasks.sendmail[1a0a9262-7858-4192-9981-b7bf0ea7483b]
[2013-08-27 19:20:45,655: WARNING/PoolWorker-4] sending mail to celery@python.org...
[2013-08-27 19:20:47,657: WARNING/PoolWorker-4] mail sent.
[2013-08-27 19:20:47,658: INFO/MainProcess] Task tasks.sendmail[1a0a9262-7858-4192-9981-b7bf0ea7483b] succeeded in 2.00266814232s: None

Celery默認設置就能知足基本要求。Worker以Pool模式啓動，默認大小爲CPU核心數量，缺省序列化機制是pickle，但能夠指定爲json。因爲Python調用UNIX/Linux程序實在太容易，因此，用Celery做爲異步任務框架很是合適json