linux 後臺進程管理利器supervisor

時間 2019-11-11

標籤 linux 後臺進程管理利器 supervisor 欄目 Linux 简体版

原文原文鏈接

Linux的後臺進程運行有好幾種方法，例如nohup，screen等，可是，若是是一個服務程序，要可靠地在後臺運行，咱們就須要把它作成daemon，最好還能監控進程狀態，在乎外結束時能自動重啓。

supervisor就是用Python開發的一套通用的進程管理程序，能將一個普通的命令行進程變爲後臺daemon，並監控進程狀態，異常退出時能自動重啓。

ubuntu安裝：

apt-get install supervisor

在/etc/supervisor 目錄下有supervisord.conf 文件，內容以下：

; supervisor config file

[unix_http_server]
file=/var/run/supervisor.sock   ; (the path to the socket file)
chmod=0700                       ; sockef file mode (default 0700)

[supervisord]
logfile=/var/log/supervisor/supervisord.log ; (main log file;default $CWD/supervisord.log)
pidfile=/var/run/supervisord.pid ; (supervisord pidfile;default supervisord.pid)
childlogdir=/var/log/supervisor            ; ('AUTO' child log dir, default $TEMP)

; the below section must remain in the config file for RPC
; (supervisorctl/web interface) to work, additional interfaces may be
; added by defining them in separate rpcinterface: sections
[rpcinterface:supervisor]
supervisor.rpcinterface_factory = supervisor.rpcinterface:make_main_rpcinterface

[supervisorctl]
serverurl=unix:///var/run/supervisor.sock ; use a unix:// URL  for a unix socket

; The [include] section can just contain the "files" setting.  This
; setting can list multiple files (separated by whitespace or
; newlines).  It can also contain wildcards.  The filenames are
; interpreted as relative to this file.  Included files *cannot*
; include files themselves.

[include]
files = /etc/supervisor/conf.d/*.conf

而後，給咱們本身開發的應用程序編寫一個配置文件，讓supervisor來管理它。每一個進程的配置文件均可以單獨分拆，放在 /etc/supervisor/conf.d/目錄下，以.conf做爲擴展名，若是修改了 /etc/supervisord.conf ,,須要執行 supervisorctl reload （重啓）來從新加載配置文件，不然會感受沒有生效，折騰到抓狂。。。

例如，app.conf定義了一個gunicorn的進程：

[html]

[program:app]

command=/usr/bin/gunicorn -w 1 wsgiapp:application

directory=/srv/www

user=www-data

其中，進程app定義在[program:app]中，command是命令，directory是進程的當前目錄，user是進程運行的用戶身份。

重啓supervisor，讓配置文件生效，

# supervisorctl start app

而後運行命令supervisorctl啓動進程：

中止進程：

# supervisorctl stop app

若是要在命令行中使用變量，就須要本身先編寫一個shell腳本：

[html]

#!/bin/sh

/usr/bin/gunicorn -w `grep -c ^processor /proc/cpuinfo` wsgiapp:application

而後，加上x權限，再把command指向該shell腳本便可。

先弄懂兩個命令:php

supervisord : supervisor的服務器端部分，啓動supervisor就是運行這個命令html

supervisorctl：啓動supervisor的命令行窗口。python

參考：http://www.2cto.com/os/201308/238166.htmllinux

一篇文章：使用supervisor監控進程：nginx

在linux下監控進程，可使用inittab，最近找到了supervisor，也很好用，記錄一下：
一、系統要安裝python，並安裝與之對應的setuptools，下載地址在此
二、安裝：
# sh setuptoolsxxxx.egg
三、安裝supervisor，下載地址在此，解壓縮後
# python setup.py install
這就ok了，而後執行
# echo_supervisord_conf > /etc/supervisord.conf
修改/etc/supervisord.conf文件，加入你要監控的進程，裏面的註釋很詳細，舉個簡單的例子：
這是一段要監控的進程的描述信息，添加到這個文件的末尾就行了：
[program:meta.txn.recover.on.error]
command=/cas/bin/meta.txn.recover.on.error ; 被監控的進程路徑
numprocs=1                    ; 啓動幾個進程
directory=/cas/bin                ; 執行前要不要先cd到目錄去，通常不用
autostart=true                ; 隨着supervisord的啓動而啓動
autorestart=true              ; 自動重啓。。固然要選上了
startretries=10               ; 啓動失敗時的最多重試次數
exitcodes=0                 ; 正常退出代碼（是說退出代碼是這個時就再也不重啓了嗎？待肯定）
stopsignal=KILL               ; 用來殺死進程的信號
stopwaitsecs=10               ; 發送SIGKILL前的等待時間
redirect_stderr=true          ; 重定向stderr到stdout
爲了節省空間，註釋的內容就不貼出來了。
執行
# supervisord -n
能在控制檯看到監控進程的輸出：
2010-08-17 10:26:07,467 INFO supervisord started with pid 943
2010-08-17 10:26:08,469 INFO spawned: 'meta.txn.recover.on.error' with pid 1009
2010-08-17 10:26:09,876 INFO success: meta.txn.recover.on.error entered RUNNING state, process has stayed up for > than 1 seconds (startsecs)
2010-08-17 10:26:48,442 INFO exited: meta.txn.recover.on.error (terminated by SIGKILL; not expected)
2010-08-17 10:26:49,444 INFO spawned: 'meta.txn.recover.on.error' with pid 2427
2010-08-17 10:26:50,487 INFO success: meta.txn.recover.on.error entered RUNNING state, process has stayed up for > than 1 seconds (startsecs)
黑體的地方是我用kill -9殺掉進程後出來的，看到supervisor檢測到進程退出後又再次啓動了進程。
不帶參數運行supervisord是以daemon方式運行。
把supervisord加入到開機啓動項裏就能夠完成監控進程的功能了。

【注意】：當supervisord以非daemon方式運行時，殺掉supervisord後，被監控的進程也退出了。
而以daemon方式運行，殺掉supervisord對被監控進程無影響。git

python安裝：github

easy_install supervisorweb

測試是否安裝成功：shell

echo_supervisord_confubuntu

會顯示如下內容，實際上是一個配置模版:

; Sample supervisor config file.
;
; For more information on the config file, please see:
; http://supervisord.org/configuration.html
;
; Note: shell expansion ("~" or "$HOME") is not supported. Environment
; variables can be expanded using this syntax: "%(ENV_HOME)s".

[unix_http_server]
file=/tmp/supervisor.sock ; (the path to the socket file)
;chmod=0700 ; socket file mode (default 0700)
;chown=nobody:nogroup ; socket file uid:gid owner
;username=user ; (default is no username (open server))
;password=123 ; (default is no password (open server))

;[inet_http_server] ; inet (TCP) server disabled by default
;port=127.0.0.1:9001 ; (ip_address:port specifier, *:port for all iface)
;username=user ; (default is no username (open server))
;password=123 ; (default is no password (open server))

[supervisord]
logfile=/tmp/supervisord.log ; (main log file;default $CWD/supervisord.log)
logfile_maxbytes=50MB ; (max main logfile bytes b4 rotation;default 50MB)
logfile_backups=10 ; (num of main logfile rotation backups;default 10)
loglevel=info ; (log level;default info; others: debug,warn,trace)
pidfile=/tmp/supervisord.pid ; (supervisord pidfile;default supervisord.pid)
……

; the below section must remain in the config file for RPC
; (supervisorctl/web interface) to work, additional interfaces may be
; added by defining them in separate rpcinterface: sections
[rpcinterface:supervisor]
supervisor.rpcinterface_factory = supervisor.rpcinterface:make_main_rpcinterface

[supervisorctl]
serverurl=unix:///tmp/supervisor.sock ; use a unix:// URL for a unix socket
;serverurl=http://127.0.0.1:9001 ; use an http:// url to specify an inet socket
;username=chris ; should be same as http_username if set
;password=123 ; should be same as http_password if set
;prompt=mysupervisor ; cmd line prompt (default "supervisor")
;history_file=~/.sc_history ; use readline history if available

; The below sample program section shows all possible program subsection values,
; create one or more ‘real’ program: sections to be able to control them under
; supervisor.

;[program:theprogramname]
;command=/bin/cat ; the program (relative uses PATH, can take args)
;process_name=%(program_name)s ; process_name expr (default %(program_name)s)
;numprocs=1 ; number of processes copies to start (def 1)
;directory=/tmp ; directory to cwd to before exec (def no cwd)
;umask=022 ; umask for process (default None)
;priority=999 ; the relative start priority (default 999)
;autostart=true ; start at supervisord start (default: true)
;autorestart=unexpected ; whether/when to restart (default: unexpected)
;startsecs=1 ; number of secs prog must stay running (def. 1)
;startretries=3 ; max # of serial start failures (default 3)
;exitcodes=0,2 ; ‘expected’ exit codes for process (default 0,2)
;stopsignal=QUIT ; signal used to kill process (default TERM)
;stopwaitsecs=10 ; max num secs to wait b4 SIGKILL (default 10)
……

; The [include] section can just contain the "files" setting. This
; setting can list multiple files (separated by whitespace or
; newlines). It can also contain wildcards. The filenames are
; interpreted as relative to this file. Included files *cannot*
; include files themselves.

;[include]
;files = relative/directory/*.ini

建立配置文件：

echo_supervisord_conf > /etc/supervisord.conf

(easy_install安裝的方式好像沒有/etc/supervisord.conf,因此這裏咱們手動建立）

修改配置文件：

在supervisord.conf最後增長(分號後邊的表示註釋，能夠不寫)：

[program:open_search]
command=/home/s/www/fukun/gopath/src/open_search/open_search ;須要執行的命令
directory=/home/s/www/fukun/gopath/src/open_search/ ; directory to cwd to before exec (def no cwd)
autostart=true ; start at supervisord start (default: true)
autorestart=true ; whether/when to restart (default: unexpected)
startsecs=3 ; number of secs prog must stay running (def. 1)
redirect_stderr=true ; redirect proc stderr to stdout (default false) 錯誤輸出重定向
stdout_logfile=/tmp/open_search_gorun.log ; stdout log path, NONE for none; default AUTO, log輸出

（更多配置說明請參考：http://supervisord.org/configuration.html）

supervisor管理gearman

參考官方文檔：

http://supervisord.org/running.html

Running supervisord automatically on startup

If you are using a distribution-packaged version of Supervisor, it should already be integrated into the service management infrastructure of your distribution.

There are user-contributed scripts for various operating systems at:https://github.com/Supervisor/initscripts

There are some answers at Serverfault in case you get stuck: How to automatically start supervisord on Linux (Ubuntu)

github的腳本下載下來：

#!/bin/bash
#
# supervisord   This scripts turns supervisord on
#
# Author:       Mike McGrath <mmcgrath@redhat.com> (based off yumupdatesd)
#               Jason Koppe <jkoppe@indeed.com> adjusted to read sysconfig,
#                   use supervisord tools to start/stop, conditionally wait
#                   for child processes to shutdown, and startup later
#               Mikhail Mingalev <mingalevme@gmail.com> Merged
#                   redhat-init-jkoppe and redhat-sysconfig-jkoppe, and
#                   made the script "simple customizable".
#               Brendan Maguire <maguire.brendan@gmail.com> Added OPTIONS to
#                   SUPERVISORCTL status call
#
# chkconfig:    345 83 04
#
# description:  supervisor is a process control utility.  It has a web based
#               xmlrpc interface as well as a few other nifty features.
#               Script was originally written by Jason Koppe <jkoppe@indeed.com>.
#

# source function library
. /etc/rc.d/init.d/functions

set -a

PREFIX=/usr

SUPERVISORD=$PREFIX/bin/supervisord
SUPERVISORCTL=$PREFIX/bin/supervisorctl

PIDFILE=/var/run/supervisord.pid
LOCKFILE=/var/lock/subsys/supervisord

OPTIONS="-c /etc/supervisord.conf"

# unset this variable if you don't care to wait for child processes to shutdown before removing the $LOCKFILE-lock
WAIT_FOR_SUBPROCESSES=yes

# remove this if you manage number of open files in some other fashion
ulimit -n 96000

RETVAL=0


running_pid()
{
    # Check if a given process pid's cmdline matches a given name
    pid=$1
    name=$2
    [ -z "$pid" ] && return 1
    [ ! -d /proc/$pid ] && return 1
    (cat /proc/$pid/cmdline | tr "\000" "\n"|grep -q $name) || return 1
    return 0
}

running()
{
# Check if the process is running looking at /proc
# (works for all users)

    # No pidfile, probably no daemon present
    [ ! -f "$PIDFILE" ] && return 1
    # Obtain the pid and check it against the binary name
    pid=`cat $PIDFILE`
    running_pid $pid $SUPERVISORD || return 1
    return 0
}

start() {
        echo "Starting supervisord: "
    
        if [ -e $PIDFILE ]; then 
        echo "ALREADY STARTED"
        return 1
    fi

    # start supervisord with options from sysconfig (stuff like -c)
        $SUPERVISORD $OPTIONS
    
    # show initial startup status
    $SUPERVISORCTL $OPTIONS status
    
    # only create the subsyslock if we created the PIDFILE
        [ -e $PIDFILE ] && touch $LOCKFILE
}

stop() {
        echo -n "Stopping supervisord: "
        $SUPERVISORCTL $OPTIONS shutdown
    if [ -n "$WAIT_FOR_SUBPROCESSES" ]; then 
            echo "Waiting roughly 60 seconds for $PIDFILE to be removed after child processes exit"
            for sleep in  2 2 2 2 4 4 4 4 8 8 8 8 last; do
                if [ ! -e $PIDFILE ] ; then
                    echo "Supervisord exited as expected in under $total_sleep seconds"
                    break
                else
                    if [[ $sleep -eq "last" ]] ; then
                        echo "Supervisord still working on shutting down. We've waited roughly 60 seconds, we'll let it do its thing from here"
                        return 1
                    else
                        sleep $sleep
                        total_sleep=$(( $total_sleep + $sleep ))
                    fi

                fi
            done
        fi

        # always remove the subsys. We might have waited a while, but just remove it at this point.
        rm -f $LOCKFILE
}

restart() {
        stop
        start
}

case "$1" in
    start)
        start
        RETVAL=$?
        ;;
    stop)
        stop
        RETVAL=$?
        ;;
    restart|force-reload)
        restart
        RETVAL=$?
        ;;
    reload)
        $SUPERVISORCTL $OPTIONS reload
        RETVAL=$?
        ;;
    condrestart)
        [ -f $LOCKFILE ] && restart
        RETVAL=$?
        ;;
    status)
        $SUPERVISORCTL $OPTIONS status
        if running ; then
            RETVAL=0
        else
            RETVAL=1
        fi
        ;;
    *)
        echo $"Usage: $0 {start|stop|status|restart|reload|force-reload|condrestart}"
        exit 1
esac

exit $RETVAL

vim /etc/init.d/supervisord , 修改文件屬性爲可執行chmod 777 /etc/init.d/supervisordmy

加入gearman的任務，下面是百度雲推送的任務

vim /etc/supervisord.conf

[program:APNS_PUSH]
command=/usr/bin/php /data2/www/web/apns/worker_origin.php
process_name=APNS_PUSH_%(process_num)s
numprocs=2
autostart=true
autorestart=true
user=nginx
stdout_logfile=/data2/log/push/push_%(process_num)s.log
stderr_logfile=/data2/log/push/push.error.log

下面先啓動gearman，啓動supervisord，ps -ef |grep xxx 查看一下任務有沒有執行

參考：http://www.cnblogs.com/trying/p/4038285.html

Supervisord是用Python實現的一款很是實用的進程管理工具，在批量服務化管理時特別有效。

一、安裝
安裝環境 centos python2.4
安裝 wget http://pypi.python.org/packages/source/s/supervisor/supervisor-3.0a10.tar.gz#md5=99c6fbd45bade87301296b7a597fb68e
tar zxvf supervisor-3.0a10.tar.gz
cd supervisor-3.0a10
python setup.py install
若是上述遇到問題，請按以下操做。
依賴setuptools安裝 wget http://pypi.python.org/packages/2.4/s/setuptools/setuptools-0.6c11-py2.4.egg#md5=bd639f9b0eac4c42497034dec2ec0c2b
sh setuptools-0.6c11-py2.4.egg

二、配置
產生基礎配置文件 echo_supervisord_conf > /etc/supervisord.conf
配置 [inet_http_server]前面的分號去掉; 定義port=192.168.1.1:8888 啓動 supervisord -c /etc/supervisord.conf 查看瀏覽器打開 http://192.168.10.129:9001
深刻配置 [program:service]
command=sh /opt/soft/service/run.sh
目前咱們用supervisord 監控 memcached 進程
[program:memcached] ###定義服務
command=memcached -m 2000 -u nobody -l 0.0.0.0 -c 10240 -p 11211  ###服務運行時的顯示的command
numprocs=1                                                                                       ###控制memcached啓動的進程數
user=root                                                                                            ###程序運行的用戶
autostart=true                                                                                     ###supervisord啓動memcached啓動
autorestart=true                                                                                  ###supervisord重啓memcache自動重啓
stdout_logfile=/var/log/memcached.stdout.log                                       ###監控memcached日誌文件
redirect_stderr=true                                                                            ###將stderr重定向到stdout（或者建立個err.log）
stopsignal=QUIT

這個程序最好不能是daemon後臺守護程序。supervisord會做爲這個程序的父來啓動它

三、啓動
supervisord (以daemon方式啓動)
或 supervisord -c /etc/supervisord.conf （非daemon）

supervisord，初始啓動Supervisord，啓動、管理配置中設置的進程。
supervisorctl 簡單的後臺進程控制工具

supervisorctl stop programxxx，中止某一個進程(programxxx)，programxxx爲[program:chatdemon]裏配置的值
supervisorctl start programxxx，啓動某個進程
supervisorctl restart programxxx，重啓某個進程
supervisorctl stop all，中止所有進程，注：start、restart、stop都不會載入最新的配置文件。
supervisorctl reload，載入最新的配置文件，並按新的配置啓動、管理全部進程。
supervisorctl reread，當一個服務由自動啓動修改成手動啓動時執行一下就ok

不帶參數運行supervisord是以daemon方式運行
當supervisord以非daemon方式運行時，殺掉supervisord後，被監控的進程也退出了。
而以daemon方式運行，殺掉supervisord對被監控進程無影響