後臺處理日誌通常採用ELK的架構,把日誌打到Elasticsearch。有些小流量的應用能夠採用直接上報的形式,省去了搭建ELK平臺的繁瑣。配合釘釘能快速感知。php
實現思路:利用sed命令截取當前週期內產生的日誌文件,經過釘釘羣機器人上報。nginx
這裏咱們用Nginx、php產生的日誌作講解。 截取一段Nginx的錯誤日誌:shell
2019/11/08 10:28:38 [error] 5136#0: *386932 FastCGI sent in stderr: "PHP message: PHP Deprecated: Automatically populating $HTTP_RAW_POST_DATA is deprecated and will be removed in a future version. To avoid this warning set 'always_populate_raw_post_data' to '-1' in php.ini and use the php://input stream instead. in Unknown on line 0" while reading response header from upstream"
複製代碼
利用sed命令截取,代碼以下:json
sed -n '/2019\/11\/08\s10:[0-9][0-9]:[0-9][0-9]\s\[error\]/ p' nginx_error.log
複製代碼
截取php-fpm.log的日誌bash
[15-Oct-2019 15:15:10] NOTICE: child 2054 stopped for tracing
[15-Oct-2019 15:15:10] NOTICE: about to trace 2054
[15-Oct-2019 15:15:10] ERROR: failed to ptrace(PEEKDATA) pid 2054: Input/output error (5)
[15-Oct-2019 15:15:10] NOTICE: finished trace of 2054
[15-Oct-2019 15:17:00] WARNING: [pool www] child 2051,
複製代碼
利用sed命令截取,代碼以下:服務器
sed -n '/15-Oct-2019\s15:[0-9][0-9]:[0-9][0-9]\]\s[ERROR|WARNING]/ p' php-fpm.log
複製代碼
總體shell腳本以下 log_report.sh:架構
#!/bin/bash
PHP_LOG_PATH='/var/log/php-fpm.log'
NGINX_LOG_PATH='/var/log/nginx.error.log'
DING_TALK_URL='ding_talk_url'
php_log_process(){
# 由於服務器改了語言環境,因此這裏臨時更換了下語言環境
DATE_TIME=`env LANG=en_US.UTF-8 date +"%d-%b-%Y %H"`
message=`sed -n '/'${DATE_TIME}':[0-9][0-9]:[0-9][0-9]\]\s[ERROR|WARNING]/ p' ${PHP_LOG_PATH}`
echo $message
}
nginx_log_process(){
DATE_TIME=`env LANG=en_US.UTF-8 date +"%Y\/%m\/%d\s%H"`
message=`sed -n '/'${DATE_TIME}':[0-9][0-9]:[0-9][0-9]\s\[error\]/ p' ${NGINX_LOG_PATH}`
echo $message
}
main(){
res=`php_log_process`
if test -z "$res";then
echo 'The php log is empty'
else
push ${res//\"/-}
fi
temp=`nginx_log_process`
if test -z "$temp";then
echo 'The nginx log is empty'
else
push ${temp//\"/-}
fi
}
test1(){
# message=`sed -n '/2019\/11\/06\s16:[0-9][0-9]:[0-9][0-9]\s\[error\]/ p' $NGINX_LOG_PATH`
# message=`sed -n '/15-Oct-2019\s15:[0-9][0-9]:[0-9][0-9]\]\s[ERROR|WARNING]/ p' $PHP_LOG_PATH`
DATE_TIME=`env LANG=en_US.UTF-8 date +"%Y\/%m\/%d\s%H"`
message=`sed -n '/'${DATE_TIME}':[0-9][0-9]:[0-9][0-9]\s\[error\]/ p' ${NGINX_LOG_PATH}`
# 判斷是否有日誌內容
if test -z "$message";then
echo 'is empty'
else
echo ${message//\"/-}
push ${message//\"/-}
fi
}
push(){
curl $DING_TALK_URL \
-H 'Content-Type: application/json' \
-d '{"msgtype": "text",
"text": {
"content": "'"$*"'"
}
}'
}
if [[ $1 == 'test' ]];then
test1
else
main
fi
複製代碼
還需配合crontab 任務,這裏有個策略就是把任務放在了第59分鐘執行,這樣就能把以前1小時內產生的日誌抓取出來了app
# log report
59 * * * * /bin/sh log_report.sh
複製代碼
釘釘添加羣聊自定義機器人過程略過。 效果圖以下: curl