Nginx高級配置-自定義json格式日誌

　　　　　　　　　　　　　　Nginx高級配置-自定義json格式日誌

　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　做者：尹正傑

版權聲明：原創做品，謝絕轉載！不然將追究法律責任。

 

 

　　在大數據運維工做中，咱們常常會使用flume，filebeat相關日誌收集工具取收集日誌，但這些日誌在收集前都日誌基本上都是json格式的，經過flume收集日誌到hdfs集羣，開發人員就直接使用java，scala語言取處理日誌，有的時候會使用到spark,fink等框架去處理日誌。所以nginx配置爲json格式仍是很是有必要的。

　　訪問日誌是記錄客戶端即用戶的具體請求內容信息，全局配置模塊中的error_log是記錄nginx服務器運行時的日誌保存路徑和記錄日誌的level，所以有着本質的區別，並且Nginx的錯誤日誌通常只有一個，可是訪問日誌能夠在不一樣server中定義多個，定義一個日誌須要使用access_log指定日誌的保存路徑，使用log_format指定日誌的格式，格式中定義要保存的具體日誌內容。

 

一.自定義默認格式日誌

若是是要保留日誌的源格式，只是添加相應的日誌內容，則配置以下。 關於nginx日誌使用的變量名稱含義，博主推薦閱讀: 　　https://www.cnblogs.com/yinzhengjie/p/12046613.html

1>.編寫主配置文件

[root@node101.yinzhengjie.org.cn ~]# cat /yinzhengjie/softwares/nginx/conf/nginx.conf worker_processes 4; worker_cpu_affinity 00000001 00000010 00000100 00001000; events { worker_connections 100000; use epoll; accept_mutex on; multi_accept on; } http { include mime.types; default_type text/html; charset utf-8; log_format my_default_format '$remote_addr - $remote_user [$time_local] "$request"' '$status $body_bytes_sent "$http_referer"' '"$http_user_agent"' '"$http_x_forwarded_ for"' '$server_name:$server_port'; access_log logs/access.log my_default_format; include /yinzhengjie/softwares/nginx/conf.d/*.conf; } [root@node101.yinzhengjie.org.cn ~]# [root@node101.yinzhengjie.org.cn ~]# nginx -t nginx: the configuration file /yinzhengjie/softwares/nginx/conf/nginx.conf syntax is ok nginx: configuration file /yinzhengjie/softwares/nginx/conf/nginx.conf test is successful [root@node101.yinzhengjie.org.cn ~]# 

2>.編寫子配置文件

[root@node101.yinzhengjie.org.cn ~]# cat /yinzhengjie/softwares/nginx/conf.d/share.conf server { listen 80; server_name node101.yinzhengjie.org.cn; location / { root /yinzhengjie/data/web/nginx/static; index index.html; } location /nginx_status { stub_status; allow 172.30.1.108; deny all; } location /main { index index.html; default_type text/html; set $name jason; set $nginx_name $server_name; echo "姓名: $name"; echo "************"; echo "Nginx服務器名稱: $nginx_name"; } } [root@node101.yinzhengjie.org.cn ~]# [root@node101.yinzhengjie.org.cn ~]# nginx -t nginx: the configuration file /yinzhengjie/softwares/nginx/conf/nginx.conf syntax is ok nginx: configuration file /yinzhengjie/softwares/nginx/conf/nginx.conf test is successful [root@node101.yinzhengjie.org.cn ~]# 

3>.從新加載nginx的配置文件

[root@node101.yinzhengjie.org.cn ~]# ps -ef | grep nginx | grep -v grep root 9297 1 0 Dec17 ? 00:00:00 nginx: master process nginx nginx 11823 9297 0 12:50 ? 00:00:00 nginx: worker process nginx 11824 9297 0 12:50 ? 00:00:00 nginx: worker process nginx 11825 9297 0 12:50 ? 00:00:00 nginx: worker process nginx 11826 9297 0 12:50 ? 00:00:00 nginx: worker process [root@node101.yinzhengjie.org.cn ~]# [root@node101.yinzhengjie.org.cn ~]# nginx -s reload [root@node101.yinzhengjie.org.cn ~]# [root@node101.yinzhengjie.org.cn ~]# ps -ef | grep nginx | grep -v grep root 9297 1 0 Dec17 ? 00:00:00 nginx: master process nginx nginx 11890 9297 1 12:57 ? 00:00:00 nginx: worker process nginx 11891 9297 1 12:57 ? 00:00:00 nginx: worker process nginx 11892 9297 1 12:57 ? 00:00:00 nginx: worker process nginx 11893 9297 1 12:57 ? 00:00:00 nginx: worker process [root@node101.yinzhengjie.org.cn ~]# 

4>.瀏覽器訪問"http://node101.yinzhengjie.org.cn/main"並查看日誌格式,以下圖所示。

 

二.自定義json格式日誌

　　Nginx的默認訪問日誌記錄內容相對比較單一，默認的格式也不方便後期作日誌統計分析，生產環境中一般將nginx日誌轉換爲json日誌，而後配合使用ELK作日誌收集-統計-分析。

1>.編輯主配置文件

[root@node101.yinzhengjie.org.cn ~]# cat -n /yinzhengjie/softwares/nginx/conf/nginx.conf 1 worker_processes 4; 2 worker_cpu_affinity 00000001 00000010 00000100 00001000; 3 4 events { 5 worker_connections 100000; 6 use epoll; 7 accept_mutex on; 8 multi_accept on; 9 } 10 11 http { 12 include mime.types; 13 14 default_type text/html; 15 16 charset utf-8; 17 18 log_format my_access_json '{"@timestamp":"$time_iso8601",' '"host":"$server_addr",' '"clientip":"$remote_addr",' '"size":$body_bytes_sent,' '"responsetime":$request_time,' '"upstreamtime":"$upstream_response_time",' '"upstreamhost":"$upstream_addr",' '"http_host":"$host",' '"uri":"$uri",' '"domain":"$host",' '"xff":"$http_x_forwarded_for",' '"referer":"$http_referer",' '"tcp_xff":"$proxy_protocol_addr",' '"http_user_agent":"$http_user_agent",' '"status":"$status"}'; 
　　 19 20 21 access_log logs/access_json.log my_access_json; 22 23 include /yinzhengjie/softwares/nginx/conf.d/*.conf; 24 } 25 [root@node101.yinzhengjie.org.cn ~]# [root@node101.yinzhengjie.org.cn ~]# nginx -t nginx: the configuration file /yinzhengjie/softwares/nginx/conf/nginx.conf syntax is ok nginx: configuration file /yinzhengjie/softwares/nginx/conf/nginx.conf test is successful [root@node101.yinzhengjie.org.cn ~]# 

2>.編輯子配置文件

[root@node101.yinzhengjie.org.cn ~]# cat -n /yinzhengjie/softwares/nginx/conf.d/share.conf 1 server { 2 listen 80; 3 server_name node101.yinzhengjie.org.cn; 4 5 location / { 6 root /yinzhengjie/data/web/nginx/static; 7 index index.html; 8 } 9 10 location /nginx_status { 11 stub_status; 12 allow 172.30.1.108; 13 deny all; 14 } 15 16 location /main { 17 index index.html; 18 default_type text/html; 19 set $name jason; 20 set $nginx_name $server_name; 21 echo "姓名: $name"; 22 echo "************"; 23 echo "Nginx服務器名稱: $nginx_name"; 24 } 25 26 } [root@node101.yinzhengjie.org.cn ~]# [root@node101.yinzhengjie.org.cn ~]# nginx -t nginx: the configuration file /yinzhengjie/softwares/nginx/conf/nginx.conf syntax is ok nginx: configuration file /yinzhengjie/softwares/nginx/conf/nginx.conf test is successful [root@node101.yinzhengjie.org.cn ~]# [root@node101.yinzhengjie.org.cn ~]# 

3>.從新加載nginx配置文件

[root@node101.yinzhengjie.org.cn ~]# ps -ef | grep nginx | grep -v grep root 9297 1 0 Dec17 ? 00:00:00 nginx: master process nginx nginx 11890 9297 0 12:57 ? 00:00:00 nginx: worker process nginx 11891 9297 0 12:57 ? 00:00:00 nginx: worker process nginx 11892 9297 0 12:57 ? 00:00:00 nginx: worker process nginx 11893 9297 0 12:57 ? 00:00:00 nginx: worker process [root@node101.yinzhengjie.org.cn ~]# [root@node101.yinzhengjie.org.cn ~]# nginx -s reload [root@node101.yinzhengjie.org.cn ~]# [root@node101.yinzhengjie.org.cn ~]# ps -ef | grep nginx | grep -v grep root 9297 1 0 Dec17 ? 00:00:00 nginx: master process nginx nginx 11946 9297 1 13:25 ? 00:00:00 nginx: worker process nginx 11947 9297 1 13:25 ? 00:00:00 nginx: worker process nginx 11948 9297 0 13:25 ? 00:00:00 nginx: worker process nginx 11949 9297 1 13:25 ? 00:00:00 nginx: worker process [root@node101.yinzhengjie.org.cn ~]# 

4>.瀏覽器訪問"http://node101.yinzhengjie.org.cn/main"並查看日誌格式,以下圖所示。