Celery 4.3.0 使用supervisor 配置後臺運行

參考文獻

thomassileo.name/blog/2012/0…

前言

Supervisor是一個使你的進程能夠在unix進程後臺運行的python庫。

本篇章的相關軟件版本以下：

• Celery 4.3.0
• supervisor 4.0.4

安裝

pip3 install supervisor
複製代碼

安裝完畢後，查看版本以下：

[root@server01 ~]# pip3 list | grep su
supervisor              4.0.4   
複製代碼

這時候使用pip的安裝方式其實並無設置好supervisor的環境變量，還須要查看一下supervisor安裝後的二進制可執行文件在哪裏。

• 搜索在 / 目錄下，先後模糊查詢名稱爲 supervi 的文件，以下：

[root@server01 performance]# find / -name "*supervi*" -ls | grep python3 | grep bin
405327    4 -rwxr-xr-x   1 root     root          242 Oct 12 18:42 /usr/local/python3/bin/supervisorctl
405325    4 -rwxr-xr-x   1 root     root          237 Oct 12 18:42 /usr/local/python3/bin/echo_supervisord_conf
405328    4 -rwxr-xr-x   1 root     root          240 Oct 12 18:42 /usr/local/python3/bin/supervisord

複製代碼

• 將 supervisorctl、echo_supervisord_conf 和 supervisord 添加軟鏈到執行目錄下/usr/bin

ln -s /usr/local/python3/bin/echo_supervisord_conf /usr/bin/echo_supervisord_conf
ln -s /usr/local/python3/bin/supervisord /usr/bin/supervisord
ln -s /usr/local/python3/bin/supervisorctl /usr/bin/supervisorctl
複製代碼

此時算是安裝好了。

進入django項目目錄，建立配置文件

cd /path/to/your/project # django項目目錄
echo_supervisord_conf > supervisord.conf # 建立配置文件
複製代碼

執行以下：

# 進入django項目目錄
[root@server01 ~]# cd /work/performance/ 
[root@server01 performance]# ls
apps  celery_tasks  db  docs  manage.py  performance  README.en.md  README.md  requirements.txt  start_celery.sh  static  templates  utils
[root@server01 performance]# 
# 建立配置文件
[root@server01 performance]# echo_supervisord_conf > supervisord.conf
[root@server01 performance]# 
[root@server01 performance]# cat supervisord.conf 
; Sample supervisor config file.
;
; For more information on the config file, please see:
; http://supervisord.org/configuration.html
;
; Notes:
;  - Shell expansion ("~" or "$HOME") is not supported.  Environment
;    variables can be expanded using this syntax: "%(ENV_HOME)s".
;  - Quotes around values are not supported, except in the case of
;    the environment= options as shown below.
...
複製代碼

下一步追加Celery配置到supervisord.conf文件中

示例配置以下：

[program:celeryd]
command=/home/thomas/virtualenvs/yourvenv/bin/celery worker --app=myapp -l info 
stdout_logfile=/path/to/your/logs/celeryd.log
stderr_logfile=/path/to/your/logs/celeryd.log
autostart=true
autorestart=true
startsecs=10
stopwaitsecs=600
複製代碼

command : 你要執行的 celery 命令， autostart ：自啓動

個人配置以下：

[program:celeryd]
command=/usr/bin/celery -A celery_tasks worker -l info                         
stdout_logfile=/work/performance/logs/celeryd.log
stderr_logfile=/work/performance/logs/celeryd.log
autostart=true
autorestart=true
startsecs=10
stopwaitsecs=600
複製代碼

啓動示例

只須要在django項目的路徑下，執行supervisord便可。

$ supervisord
複製代碼

執行以下：

[root@server01 performance]# supervisord
/usr/local/python3/lib/python3.7/site-packages/supervisor/options.py:471: UserWarning: Supervisord is running as root and it is searching for its configuration file in default locations (including its current working directory); you probably want to specify a "-c" argument specifying an absolute path to a configuration file for improved security.
  'Supervisord is running as root and it is searching '
[root@server01 performance]# 
複製代碼

這裏默認啓動會提示直接根據當前路徑查找配置文件supervisord.conf，固然也能夠使用-c參數指定配置文件。

supervisorctl 命令

在啓動後臺執行以後，能夠使用supervisorctl命令查看後臺的日誌信息，以及重啓服務。

$ supervisorctl tail celeryd # 查看最後的日誌
$ supervisorctl tail -f celeryd # 持續
$ supervisorctl restart celeryd
$ supervisorctl status celeryd
$ supervisorctl start celeryd
$ supervisorctl stop celeryd
複製代碼

執行以下：

• 執行supervisorctl tail celeryd 能夠看到打印一下日誌就結束了，無法持續觀察日誌

[root@server01 performance]# supervisorctl tail celeryd
/usr/local/python3/lib/python3.7/site-packages/celery/platforms.py:801: RuntimeWarning: You're running the worker with superuser privileges: this is absolutely not recommended! Please specify a different user using the --uid option. User information: uid=0 euid=0 gid=0 egid=0 uid=uid, euid=euid, gid=gid, egid=egid, [2019-10-13 22:30:20,858: INFO/MainProcess] Connected to redis://127.0.0.1:6379/8 [2019-10-13 22:30:20,871: INFO/MainProcess] mingle: searching for neighbors [2019-10-13 22:30:21,893: INFO/MainProcess] mingle: all alone [2019-10-13 22:30:21,906: WARNING/MainProcess] /usr/local/python3/lib/python3.7/site-packages/celery/fixups/django.py:202: UserWarning: Using settings.DEBUG leads to a memory leak, never use this setting in production environments! warnings.warn('Using settings.DEBUG leads to a memory leak, never ' [2019-10-13 22:30:21,907: INFO/MainProcess] celery@server01 ready. [root@server01 performance]# 複製代碼

• 增長 -f 參數，持續查看日誌信息

[root@server01 performance]# supervisorctl tail -f celeryd
==> Press Ctrl-C to exit <==
.tasks.send_pressure_report_mail
  . celery_tasks.tasks.stop_apiservers_nmon
  . celery_tasks.tasks.stop_locust

/usr/local/python3/lib/python3.7/site-packages/celery/platforms.py:801: RuntimeWarning: You're running the worker with superuser privileges: this is absolutely not recommended! Please specify a different user using the --uid option. User information: uid=0 euid=0 gid=0 egid=0 uid=uid, euid=euid, gid=gid, egid=egid, [2019-10-13 22:38:10,078: INFO/MainProcess] Connected to redis://127.0.0.1:6379/8 [2019-10-13 22:38:10,089: INFO/MainProcess] mingle: searching for neighbors [2019-10-13 22:38:11,111: INFO/MainProcess] mingle: all alone [2019-10-13 22:38:11,126: WARNING/MainProcess] /usr/local/python3/lib/python3.7/site-packages/celery/fixups/django.py:202: UserWarning: Using settings.DEBUG leads to a memory leak, never use this setting in production environments! warnings.warn('Using settings.DEBUG leads to a memory leak, never ' [2019-10-13 22:38:11,126: INFO/MainProcess] celery@server01 ready. 複製代碼

• 重啓服務

[root@server01 performance]# supervisorctl restart celeryd
celeryd: stopped
celeryd: started
複製代碼

• 查看運行狀態

[root@server01 performance]# supervisorctl status celeryd
celeryd                          RUNNING   pid 3533, uptime 0:10:05
[root@server01 performance]# 
複製代碼