python Celery模塊應用整理

什麼是celery？我的理解

    什麼是celery？異步轉發任務至隊列處理，無阻塞。這是在後端領域提升代碼執行效率的一種手段。將本來可能須要花時間同步等待的函數或者叫task轉發至代理執行如redis server。而後執行剩下代碼，以後再請求的時候確認有沒有執行完成就可，若是完成則返回執行結果，若是任務還在後臺進行，則返回任務還在執行中。

redis 工做流程圖

這篇博客我將閱讀過的博客和官方最新文檔，進行整理，以在工做中可以快速上手爲目的。。

快速上手

首先安裝celery

pip3 install Celery

由於是選擇將redis做爲中間代理，因此還須要安裝redis server

sudo apt-get install redis-server

建立tasks.py文件

from celery import Celery
 
app = Celery('tasks',
             broker='redis://localhost',
             backend='redis://localhost')
 
@app.task
def add(x,y):
    print("running...",x,y)
    return x+y

聲明一個app的celery對象名叫tasks，add函數就是一個task用於調用執行異步任務並執行完成後返回結果

啓動task

啓動任務以前確保redis-server已經運行

執行命令

celery worker -A tasks -l debug

worker是聲明啓動一個worker，-A 是指你的app的文件名， -l 是日誌文件的等級log level，對應的等級還有 info、warning、error

啓動後就能夠在別的文件中

from tasks import add
t = add.delay(3,3) #此時worker會生成一個任務和任務id
t.get() #獲取任務執行的結果
result = t.get(propagate=False) #若是任務執行中出現異常,在client端不會異常退出
print(result)#結果爲6
is_redy = t.ready()#查看任務是否執行完畢返回bool值
print(is_redy)
t.traceback #打印異常詳細信息

配置設置

方法一：

app.conf.enable_utc = True

方法二：

app.conf.update(
    enable_utc=True,
    timezone='Europe/London',
)

方法三：

建立一個配置文件叫celeryconfig.py

from celery import Celery

app = Celery()
app.config_from_object('celeryconfig')

celeryconfig.py文件內容

enable_utc = True
timezone = 'Europe/London'

方法四：

建立一個配置參數的類

from celery import Celery

app = Celery()

class Config:
    enable_utc = True
    timezone = 'Europe/London'

app.config_from_object(Config)

設置環境變量：

import os
from celery import Celery

#: Set default configuration module name
os.environ.setdefault('CELERY_CONFIG_MODULE', 'celeryconfig')

app = Celery()
app.config_from_envvar('CELERY_CONFIG_MODULE')

建立定時計劃任務

建立一個名爲periodic_tasks.py的文件用於聲明計劃任務

from __future__ import absolute_import, unicode_literals
from .celery import app
from celery.schedules import crontab


@app.on_after_configure.connect
def setup_periodic_tasks(sender, **kwargs):
    # 每10秒執行一次test
    sender.add_periodic_task(10.0, test.s('hello'), name='add every 10')

    # 每30秒執行一次test
    sender.add_periodic_task(30.0, test.s('world'), expires=10)

    # 每週5的晚上9點42分執行test
    sender.add_periodic_task(
        crontab(hour=21, minute=42, day_of_week=5),
        test.s('Happy Friday!'),
    )

@app.task
def test(arg):
    print(arg)

在app.py文件中

from __future__ import absolute_import, unicode_literals
from celery import Celery

app = Celery('proj',
             broker='redis://localhost',
             backend='redis://localhost',
             include=['periodic_tasks'])
# include能夠導入多個模塊，好比這裏導入了計劃任務模塊，你還能夠單首創建一個task模塊專門用來放task任務，並導入

# Optional configuration, see the application user guide.
app.conf.update(
    result_expires=3600,
)

if __name__ == '__main__':
    app.start()

還有更多定時配置方式以下：

Example	Meaning
crontab()	Execute every minute.
crontab(minute=0, hour=0)	Execute daily at midnight.
crontab(minute=0, hour='*/3')	Execute every three hours: midnight, 3am, 6am, 9am, noon, 3pm, 6pm, 9pm.
crontab(minute=0, hour='0,3,6,9,12,15,18,21')	Same as previous.
crontab(minute='*/15')	Execute every 15 minutes.
crontab(day_of_week='sunday')	Execute every minute (!) at Sundays.
crontab(minute='*', hour='*',day_of_week='sun')	Same as previous.
crontab(minute='*/10', hour='3,17,22',day_of_week='thu,fri')	Execute every ten minutes, but only between 3-4 am, 5-6 pm, and 10-11 pm on Thursdays or Fridays.
crontab(minute=0,hour='*/2,*/3')	Execute every even hour, and every hour divisible by three. This means: at every hour except: 1am, 5am, 7am, 11am, 1pm, 5pm, 7pm, 11pm
crontab(minute=0, hour='*/5')	Execute hour divisible by 5. This means that it is triggered at 3pm, not 5pm (since 3pm equals the 24-hour clock value of 「15」, which is divisible by 5).
crontab(minute=0, hour='*/3,8-17')	Execute every hour divisible by 3, and every hour during office hours (8am-5pm).
crontab(0, 0,day_of_month='2')	Execute on the second day of every month.
crontab(0, 0, day_of_month='2-30/3')	Execute on every even numbered day.
crontab(0, 0, day_of_month='1-7,15-21')	Execute on the first and third weeks of the month.
crontab(0, 0,day_of_month='11', month_of_year='5')	Execute on the eleventh of May every year.
crontab(0, 0, month_of_year='*/3')	Execute on the first month of every quarter.