解決nginx負載均衡高可用keepalived只針對物理機的問題

時間 2019-12-11

標籤解決 nginx 負載均衡可用 keepalived 針對物理問題欄目 Nginx 简体版

原文原文鏈接

在高可用keepalived軟件，在默認的狀況下僅僅在對方機器宕機或keepalived停掉的時候纔會接管業務。nginx

可是在實際工做過程當中，例如在nginx負載均衡工做實例中，nginx服務已中止，而keepalived服務還在shell

工做的狀況，這會致使用戶訪問VIP，而找不到對應的服務。bash

例如：app

主機IP：10.0.0.5/24負載均衡

備機IP：10.0.0.6/24ssh

VIP:10.0.0.100/24tcp

方法一：直接在主機上寫守護腳本，來對nginx服務進行監控oop

一、腳本內容以下：測試

 1 [root@lb01 scripts]# cat check_nginx.sh 
 2 #!/bin/bash
 3 
 4 #-------------CopyRight-------------  
 5 #   Name:check nginx server  
 6 #   Version Number:1.00  
 7 #   Type:sh  
 8 #   Language:bash shell  
 9 #   Date:2018-06-020  
10 #   Author:sandy
11 #   QQ:442656067
12 #   Email:eeexu123@163.com  
13 #   Blog:https://www.cnblogs.com/eeexu123/
14 
15 while true
16 do
17     if [ `netstat -tulnp|grep nginx|wc -l` -ne 1 ];then
18         /etc/init.d/keepalived stop
19     fi
20     sleep 5
21 done

二、而後，直接運行此腳本，每五秒進行一次檢測nginx服務是否在線。spa

1 [root@lb01 scripts]# sh check_nginx.sh &
2 [1] 15259
3 [root@lb01 scripts]# ps -ef |grep check_nginx
4 root      15259  14594  0 10:03 pts/0    00:00:00 sh check_nginx.sh
5 root      15266  14594  0 10:03 pts/0    00:00:00 grep check_nginx

三、測試

1）查看主機VIP是否在線

 1 [root@lb01 ~]# ifconfig
 2 eth0      Link encap:Ethernet  HWaddr 00:0C:29:2C:00:BB  
 3           inet addr:10.0.0.5  Bcast:10.0.0.255  Mask:255.255.255.0
 4           inet6 addr: fe80::20c:29ff:fe2c:bb/64 Scope:Link
 5           UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
 6           RX packets:41469 errors:0 dropped:0 overruns:0 frame:0
 7           TX packets:26310 errors:0 dropped:0 overruns:0 carrier:0
 8           collisions:0 txqueuelen:1000 
 9           RX bytes:33793651 (32.2 MiB)  TX bytes:5162495 (4.9 MiB)
10 
11 eth0:1    Link encap:Ethernet  HWaddr 00:0C:29:2C:00:BB  
12           inet addr:10.0.0.100  Bcast:0.0.0.0  Mask:255.255.255.0
13           UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
14 
15 eth1      Link encap:Ethernet  HWaddr 00:0C:29:2C:00:C5  
16           inet addr:172.16.1.5  Bcast:172.16.1.255  Mask:255.255.255.0
17           inet6 addr: fe80::20c:29ff:fe2c:c5/64 Scope:Link
18           UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
19           RX packets:8830 errors:0 dropped:0 overruns:0 frame:0
20           TX packets:311 errors:0 dropped:0 overruns:0 carrier:0
21           collisions:0 txqueuelen:1000 
22           RX bytes:809854 (790.8 KiB)  TX bytes:38876 (37.9 KiB)
23 
24 lo        Link encap:Local Loopback  
25           inet addr:127.0.0.1  Mask:255.0.0.0
26           inet6 addr: ::1/128 Scope:Host
27           UP LOOPBACK RUNNING  MTU:65536  Metric:1
28           RX packets:0 errors:0 dropped:0 overruns:0 frame:0
29           TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
30           collisions:0 txqueuelen:0 
31           RX bytes:0 (0.0 b)  TX bytes:0 (0.0 b)

2）中止nginx服務

1 [root@lb01 ~]# /application/nginx/sbin/nginx -s stop
2 [root@lb01 ~]# netstat -tulnp
3 Active Internet connections (only servers)
4 Proto Recv-Q Send-Q Local Address               Foreign Address             State       PID/Program name   
5 tcp        0      0 0.0.0.0:22                  0.0.0.0:*                   LISTEN      1613/sshd           
6 tcp        0      0 127.0.0.1:25                0.0.0.0:*                   LISTEN      1692/master         
7 tcp        0      0 :::22                       :::*                        LISTEN      1613/sshd           
8 tcp        0      0 ::1:25                      :::*                        LISTEN      1692/master

3）查看VIP是否漂移

主機IP：

 1 [root@lb01 ~]# ifconfig
 2 eth0      Link encap:Ethernet  HWaddr 00:0C:29:2C:00:BB 3 inet addr:10.0.0.5 Bcast:10.0.0.255 Mask:255.255.255.0 4 inet6 addr: fe80::20c:29ff:fe2c:bb/64 Scope:Link 5 UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1 6 RX packets:41555 errors:0 dropped:0 overruns:0 frame:0 7 TX packets:26526 errors:0 dropped:0 overruns:0 carrier:0 8 collisions:0 txqueuelen:1000 9 RX bytes:33803135 (32.2 MiB) TX bytes:5182305 (4.9 MiB) 10 11 eth1 Link encap:Ethernet HWaddr 00:0C:29:2C:00:C5 12 inet addr:172.16.1.5 Bcast:172.16.1.255 Mask:255.255.255.0 13 inet6 addr: fe80::20c:29ff:fe2c:c5/64 Scope:Link 14 UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1 15 RX packets:8854 errors:0 dropped:0 overruns:0 frame:0 16 TX packets:311 errors:0 dropped:0 overruns:0 carrier:0 17 collisions:0 txqueuelen:1000 18 RX bytes:811478 (792.4 KiB) TX bytes:38876 (37.9 KiB) 19 20 lo Link encap:Local Loopback 21 inet addr:127.0.0.1 Mask:255.0.0.0 22 inet6 addr: ::1/128 Scope:Host 23 UP LOOPBACK RUNNING MTU:65536 Metric:1 24 RX packets:0 errors:0 dropped:0 overruns:0 frame:0 25 TX packets:0 errors:0 dropped:0 overruns:0 carrier:0 26 collisions:0 txqueuelen:0 27 RX bytes:0 (0.0 b) TX bytes:0 (0.0 b)

備機IP

 1 [root@lb02 ~]# ifconfig
 2 eth0      Link encap:Ethernet  HWaddr 00:0C:29:93:72:8D 3 inet addr:10.0.0.6 Bcast:10.0.0.255 Mask:255.255.255.0 4 inet6 addr: fe80::20c:29ff:fe93:728d/64 Scope:Link 5 UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1 6 RX packets:40535 errors:0 dropped:0 overruns:0 frame:0 7 TX packets:23552 errors:0 dropped:0 overruns:0 carrier:0 8 collisions:0 txqueuelen:1000 9 RX bytes:33425626 (31.8 MiB) TX bytes:4173830 (3.9 MiB) 10 11 eth0:1 Link encap:Ethernet HWaddr 00:0C:29:93:72:8D 12 inet addr:10.0.0.100 Bcast:0.0.0.0 Mask:255.255.255.0 13 UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1 14 15 eth1 Link encap:Ethernet HWaddr 00:0C:29:93:72:97 16 inet addr:172.16.1.6 Bcast:172.16.1.255 Mask:255.255.255.0 17 inet6 addr: fe80::20c:29ff:fe93:7297/64 Scope:Link 18 UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1 19 RX packets:8608 errors:0 dropped:0 overruns:0 frame:0 20 TX packets:56 errors:0 dropped:0 overruns:0 carrier:0 21 collisions:0 txqueuelen:1000 22 RX bytes:781127 (762.8 KiB) TX bytes:6646 (6.4 KiB) 23 24 lo Link encap:Local Loopback 25 inet addr:127.0.0.1 Mask:255.0.0.0 26 inet6 addr: ::1/128 Scope:Host 27 UP LOOPBACK RUNNING MTU:65536 Metric:1 28 RX packets:0 errors:0 dropped:0 overruns:0 frame:0 29 TX packets:0 errors:0 dropped:0 overruns:0 carrier:0 30 collisions:0 txqueuelen:0 31 RX bytes:0 (0.0 b) TX bytes:0 (0.0 b)

由上述可知，VIP發生漂移。keepalived觸發腳本執行成功。

方法二：能夠使用keepalived的配置文件來觸發已寫好的腳本

一、腳本內容以下：

 1 [root@lb01 scripts]# cat keep_nginx.sh 
 2 #!/bin/bash
 3 
 4 #-------------CopyRight-------------  
 5 #   Name:check nginx server  
 6 #   Version Number:1.00  
 7 #   Type:sh  
 8 #   Language:bash shell  
 9 #   Date:2018-06-020  
10 #   Author:sandy
11 #   QQ:442656067
12 #   Email:eeexu123@163.com  
13 #   Blog:https://www.cnblogs.com/eeexu123/
14 
15 if [ `netstat -tulnp|grep nginx|wc -l` -ne 1 ];then
16     /etc/init.d/keepalived stop
17 fi

二、將腳本加上可執行權限

[root@lb01 scripts]# chmod +x keep_nginx.sh

1 [root@lb01 scripts]# ll
2 總用量 8
3 -rw-r--r--. 1 root root 401 7月   4 10:11 check_nginx.sh
4 -rwxr-xr-x. 1 root root 365 7月   4 10:11 keep_nginx.sh

三、在主機的keepalived的配置文件上，作以下修改

　　/etc/keepalived/keepalived.conf配置文件以下：

　　綠色部份內容是新添加觸發腳本

 1 ! Configuration File for keepalived
 2 
 3 global_defs {
 4    notification_email {
 5      eeexu123@163.com
 6    }
 7    notification_email_from Alexandre.Cassen@firewall.loc
 8    smtp_server 127.0.0.1
 9    smtp_connect_timeout 30
10    router_id lb01
11 }
12 
13 vrrp_script chk_nginx_proxy { 14     script "/server/scripts/keep_nginx.sh"
15 
16     interval 2
17     weight 2
18 } 19 vrrp_instance VI_1 {
20     state MASTER
21     interface eth0
22     virtual_router_id 51
23     priority 150
24     advert_int 1
25     authentication {
26         auth_type PASS
27         auth_pass 1111
28     }
29     virtual_ipaddress {
30         10.0.0.100/24 dev eth0 label eth0:1
31     }
32  track_script { 33  chk_nginx_proxy 34  } 35 }

四、說明：

vrrp_script chk_nginx_proxy {                    #定義vrrp腳本
    script "/server/scripts/keep_nginx.sh"       #執行keep_nginx.sh腳本，當nginx服務有問題時，就中止keepalived服務

    interval 2                                   #間隔兩秒檢查一次
    weight 2
}

track_script { #觸發檢查
chk_nginx_proxy
}

五、測試

1）查看主機中的VIP是否在線

 1 [root@lb01 ~]# ifconfig
 2 eth0      Link encap:Ethernet  HWaddr 00:0C:29:2C:00:BB  
 3           inet addr:10.0.0.5  Bcast:10.0.0.255  Mask:255.255.255.0
 4           inet6 addr: fe80::20c:29ff:fe2c:bb/64 Scope:Link
 5           UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
 6           RX packets:41469 errors:0 dropped:0 overruns:0 frame:0
 7           TX packets:26310 errors:0 dropped:0 overruns:0 carrier:0
 8           collisions:0 txqueuelen:1000 
 9           RX bytes:33793651 (32.2 MiB)  TX bytes:5162495 (4.9 MiB)
10 
11 eth0:1    Link encap:Ethernet  HWaddr 00:0C:29:2C:00:BB  
12           inet addr:10.0.0.100  Bcast:0.0.0.0  Mask:255.255.255.0
13           UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
14 
15 eth1      Link encap:Ethernet  HWaddr 00:0C:29:2C:00:C5  
16           inet addr:172.16.1.5  Bcast:172.16.1.255  Mask:255.255.255.0
17           inet6 addr: fe80::20c:29ff:fe2c:c5/64 Scope:Link
18           UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
19           RX packets:8830 errors:0 dropped:0 overruns:0 frame:0
20           TX packets:311 errors:0 dropped:0 overruns:0 carrier:0
21           collisions:0 txqueuelen:1000 
22           RX bytes:809854 (790.8 KiB)  TX bytes:38876 (37.9 KiB)
23 
24 lo        Link encap:Local Loopback  
25           inet addr:127.0.0.1  Mask:255.0.0.0
26           inet6 addr: ::1/128 Scope:Host
27           UP LOOPBACK RUNNING  MTU:65536  Metric:1
28           RX packets:0 errors:0 dropped:0 overruns:0 frame:0
29           TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
30           collisions:0 txqueuelen:0 
31           RX bytes:0 (0.0 b)  TX bytes:0 (0.0 b)

2）中止nginx服務

1 [root@lb01 ~]# /application/nginx/sbin/nginx -s stop
2 [root@lb01 ~]# netstat -tulnp
3 Active Internet connections (only servers)
4 Proto Recv-Q Send-Q Local Address               Foreign Address             State       PID/Program name   
5 tcp        0      0 0.0.0.0:22                  0.0.0.0:*                   LISTEN      1613/sshd           
6 tcp        0      0 127.0.0.1:25                0.0.0.0:*                   LISTEN      1692/master         
7 tcp        0      0 :::22                       :::*                        LISTEN      1613/sshd           
8 tcp        0      0 ::1:25                      :::*                        LISTEN      1692/master

3）查看VIP是否漂移

主機IP：

 1 [root@lb01 ~]# ifconfig
 2 eth0      Link encap:Ethernet  HWaddr 00:0C:29:2C:00:BB  
 3           inet addr:10.0.0.5  Bcast:10.0.0.255  Mask:255.255.255.0
 4           inet6 addr: fe80::20c:29ff:fe2c:bb/64 Scope:Link
 5           UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
 6           RX packets:41555 errors:0 dropped:0 overruns:0 frame:0
 7           TX packets:26526 errors:0 dropped:0 overruns:0 carrier:0
 8           collisions:0 txqueuelen:1000 
 9           RX bytes:33803135 (32.2 MiB)  TX bytes:5182305 (4.9 MiB)
10 
11 eth1      Link encap:Ethernet  HWaddr 00:0C:29:2C:00:C5  
12           inet addr:172.16.1.5  Bcast:172.16.1.255  Mask:255.255.255.0
13           inet6 addr: fe80::20c:29ff:fe2c:c5/64 Scope:Link
14           UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
15           RX packets:8854 errors:0 dropped:0 overruns:0 frame:0
16           TX packets:311 errors:0 dropped:0 overruns:0 carrier:0
17           collisions:0 txqueuelen:1000 
18           RX bytes:811478 (792.4 KiB)  TX bytes:38876 (37.9 KiB)
19 
20 lo        Link encap:Local Loopback  
21           inet addr:127.0.0.1  Mask:255.0.0.0
22           inet6 addr: ::1/128 Scope:Host
23           UP LOOPBACK RUNNING  MTU:65536  Metric:1
24           RX packets:0 errors:0 dropped:0 overruns:0 frame:0
25           TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
26           collisions:0 txqueuelen:0 
27           RX bytes:0 (0.0 b)  TX bytes:0 (0.0 b)

備機IP

 1 [root@lb02 ~]# ifconfig
 2 eth0      Link encap:Ethernet  HWaddr 00:0C:29:93:72:8D  
 3           inet addr:10.0.0.6  Bcast:10.0.0.255  Mask:255.255.255.0
 4           inet6 addr: fe80::20c:29ff:fe93:728d/64 Scope:Link
 5           UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
 6           RX packets:40535 errors:0 dropped:0 overruns:0 frame:0
 7           TX packets:23552 errors:0 dropped:0 overruns:0 carrier:0
 8           collisions:0 txqueuelen:1000 
 9           RX bytes:33425626 (31.8 MiB)  TX bytes:4173830 (3.9 MiB)
10 
11 eth0:1    Link encap:Ethernet  HWaddr 00:0C:29:93:72:8D  
12           inet addr:10.0.0.100  Bcast:0.0.0.0  Mask:255.255.255.0
13           UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
14 
15 eth1      Link encap:Ethernet  HWaddr 00:0C:29:93:72:97  
16           inet addr:172.16.1.6  Bcast:172.16.1.255  Mask:255.255.255.0
17           inet6 addr: fe80::20c:29ff:fe93:7297/64 Scope:Link
18           UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
19           RX packets:8608 errors:0 dropped:0 overruns:0 frame:0
20           TX packets:56 errors:0 dropped:0 overruns:0 carrier:0
21           collisions:0 txqueuelen:1000 
22           RX bytes:781127 (762.8 KiB)  TX bytes:6646 (6.4 KiB)
23 
24 lo        Link encap:Local Loopback  
25           inet addr:127.0.0.1  Mask:255.0.0.0
26           inet6 addr: ::1/128 Scope:Host
27           UP LOOPBACK RUNNING  MTU:65536  Metric:1
28           RX packets:0 errors:0 dropped:0 overruns:0 frame:0
29           TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
30           collisions:0 txqueuelen:0 
31           RX bytes:0 (0.0 b)  TX bytes:0 (0.0 b)