生產環境中,不少企業把Nginx做爲負載均衡器來用,它的重要性很高,一旦宕機會致使整個站點不能訪問,因此有必要再準備一臺備用Nginx,Keepalived用在這種場景下很是合適。html
VIP的英文名字是「Virtual IP",即「虛擬IP",也有人把它叫做「浮動IP」,由於這個IP是由Keepalived給服務器配置上的,服務器靠這個VIP對外提供服務,當master機器宕機,VIP被分配到backup上,這樣用戶看來是無感知的。linux
rpm -ivh http://nginx.org/packages/centos/7/noarch/RPMS/nginx-release-centos-7-0.el7.ngx.noarch.rpm
[root@gary-tao ~]# ls /etc/keepalived/keepalived.conf /etc/keepalived/keepalived.conf [root@gary-tao ~]# vi /etc/keepalived/keepalived.conf [root@gary-tao ~]# > !$ //清空配置文件內容 [root@gary-tao ~]# viM /etc/keepalived/keepalived.conf 增長以下配置內容,按需求更改部分配置: global_defs { //全局定義參數 notification_email { aming@aminglinux.com //定義接收告警的人 } notification_email_from root@aminglinux.com //定義發郵件地址(實際上沒用) smtp_server 127.0.0.1 //定義發郵件地址,若爲127.0.0.1則使用本機自帶郵件服務器發送 smtp_connect_timeout 30 router_id LVS_DEVEL } vrrp_script chk_nginx { //chk_nginx爲自定義名字,後面還會用到它 script "/usr/local/sbin/check_ng.sh" //自定義腳本,該腳本爲監控nginx服務的腳本 interval 3 //每隔3S執行一次該腳本 } vrrp_instance VI_1 { state MASTER //角色爲master interface ens33 //針對哪一個網卡監聽VIP virtual_router_id 51 priority 100 //權重爲100,master要比backup大 advert_int 1 authentication { auth_type PASS auth_pass aminglinux>com //定義密碼,這個密碼自定義 } virtual_ipaddress { 172.16.111.150 //定義VIP } track_script { chk_nginx //定義監控腳本,這裏和上面vrr_script後面的字符串保持一致 } }
[root@gary-tao ~]# vim /usr/local/sbin/check_ng.sh //腳本名字是自定義的,與keepalived配置文件要一致 編輯腳本增長以下內容: #!/bin/bash #時間變量,用於記錄日誌 d=`date --date today +%Y%m%d_%H:%M:%S` #計算nginx進程數量 n=`ps -C nginx --no-heading|wc -l` #若是進程爲0,則啓動nginx,而且再次檢測nginx進程數量, #若是還爲0,說明nginx沒法啓動,此時須要關閉keepalived if [ $n -eq "0" ]; then /etc/init.d/nginx start n2=`ps -C nginx --no-heading|wc -l` if [ $n2 -eq "0" ]; then echo "$d nginx down,keepalived will stop" >> /var/log/check_ng.log systemctl stop keepalived fi fi [root@gary-tao ~]# chmod 755 /usr/local/sbin/check_ng.sh //須要給它權限,不然沒法被keepalived調用加載 [root@gary-tao ~]# systemctl start keepalived //啓動master上的keepalived,若是nginx服務沒有啓動,它會自動拉起來,並監聽VIP [root@gary-tao ~]# ip addr //master上已經自動配置了172.16.111.159這個IP 1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN qlen 1 link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 inet 127.0.0.1/8 scope host lo valid_lft forever preferred_lft forever inet6 ::1/128 scope host valid_lft forever preferred_lft forever 2: ens33: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen 1000 link/ether 00:0c:29:09:e5:58 brd ff:ff:ff:ff:ff:ff inet 172.16.111.100/16 brd 172.16.255.255 scope global ens33 valid_lft forever preferred_lft forever inet 172.16.111.150/32 scope global ens33 valid_lft forever preferred_lft forever inet6 fe80::1ffb:cde1:5f3e:5778/64 scope link valid_lft forever preferred_lft forever 3: ens37: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen 1000 link/ether 00:0c:29:09:e5:62 brd ff:ff:ff:ff:ff:ff inet 172.16.111.131/24 brd 172.16.111.255 scope global dynamic ens37 valid_lft 1057sec preferred_lft 1057sec inet6 fe80::888c:a1d7:871b:8971/64 scope link valid_lft forever preferred_lft forever [root@gary-tao ~]# ps aux |grep keep root 9467 0.0 0.1 120720 1400 ? Ss 19:24 0:00 /usr/sbin/keepalived -D root 9468 0.0 0.3 122792 3104 ? S 19:24 0:00 /usr/sbin/keepalived -D root 9469 0.0 0.2 127116 2836 ? S 19:24 0:00 /usr/sbin/keepalived -D root 9641 0.0 0.0 112680 976 pts/0 R+ 19:26 0:00 grep --color=auto keep [root@gary-tao ~]# ps aux |grep nginx root 3132 0.0 0.2 46860 2892 ? Ss 1月23 0:00 nginx: master process /usr/local/nginx/sbin/nginx -c /usr/local/nginx/conf/nginx.conf nobody 6471 0.0 0.4 48784 4172 ? S 00:00 0:00 nginx: worker process nobody 6472 0.0 0.4 48784 4172 ? S 00:00 0:00 nginx: worker process root 9655 0.0 0.0 112680 976 pts/0 R+ 19:26 0:00 grep --color=auto nginx [root@gary-tao ~]# less /var/log/messages //查看日誌
[root@gary-tao ~]# setenforce 0 //臨時關閉SELinux //開機關閉SELinux 編輯/etc/selinux/config文件,將SELINUX的值設置爲disabled [root@gary-tao ~]# getenforce //查看SElinux是否關閉 Disabled [root@gary-tao ~]# systemctl stop firewalld.service //關閉防火牆 [root@gary-tao ~]# iptables -nvL //查看防火牆 Chain INPUT (policy ACCEPT 0 packets, 0 bytes) pkts bytes target prot opt in out source destination Chain FORWARD (policy ACCEPT 0 packets, 0 bytes) pkts bytes target prot opt in out source destination Chain OUTPUT (policy ACCEPT 0 packets, 0 bytes) pkts bytes target prot opt in out source destination
[root@gary ~]# ls /etc/keepalived/keepalived.conf /etc/keepalived/keepalived.conf [root@gary ~]# > !$ //清空配置 > /etc/keepalived/keepalived.conf [root@gary ~]# vim /etc/keepalived/keepalived.conf 編輯增長以下配置內容: global_defs { notification_email { aming@aminglinux.com } notification_email_from root@aminglinux.com smtp_server 127.0.0.1 smtp_connect_timeout 30 router_id LVS_DEVEL } vrrp_script chk_nginx { script "/usr/local/sbin/check_ng.sh" //檢測腳本 interval 3 } vrrp_instance VI_1 { state BACKUP //這個須要改,說明是從的狀態 interface ens33 virtual_router_id 51 priority 90 //這個權重比master少 advert_int 1 authentication { auth_type PASS auth_pass aminglinux>com } virtual_ipaddress { 172.16.111.150 //這個跟master同樣 } track_script { chk_nginx } }
[root@gary ~]# vim /usr/local/sbin/check_ng.sh 編輯增長以下配置內容: #時間變量,用於記錄日誌 d=`date --date today +%Y%m%d_%H:%M:%S` #計算nginx進程數量 n=`ps -C nginx --no-heading|wc -l` #若是進程爲0,則啓動nginx,而且再次檢測nginx進程數量, #若是還爲0,說明nginx沒法啓動,此時須要關閉keepalived if [ $n -eq "0" ]; then systemctl start nginx n2=`ps -C nginx --no-heading|wc -l` if [ $n2 -eq "0" ]; then echo "$d nginx down,keepalived will stop" >> /var/log/check_ng.log systemctl stop keepalived fi fi [root@gary ~]# chmod 755 /usr/local/sbin/check_ng.sh //更改權限 [root@gary ~]# systemctl start keepalived //啓動服務 [root@gary ~]# ps aux |grep keep root 16039 0.0 0.1 120720 1400 ? Rs 19:51 0:00 /usr/sbin/keepalived -D root 16040 0.0 0.3 122792 3104 ? S 19:51 0:00 /usr/sbin/keepalived -D root 16041 0.1 0.2 127116 2656 ? S 19:51 0:00 /usr/sbin/keepalived -D root 16059 0.0 0.0 112676 976 pts/0 S+ 19:51 0:00 grep --color=auto keep [root@gary ~]# ps aux |grep nginx root 15771 0.0 0.0 46308 948 ? Ss 16:56 0:00 nginx: master process /usr/sbin/nginx -c /etc/nginx/nginx.conf nginx 15772 0.0 0.2 46692 2148 ? S 16:56 0:00 nginx: worker process root 16077 0.0 0.0 112676 972 pts/0 S+ 19:51 0:00 grep --color=auto nginx
##master機器 [root@gary-tao ~]# curl -I 172.16.111.100 HTTP/1.1 200 OK Server: nginx/1.12.1 Date: Fri, 26 Jan 2018 05:52:53 GMT Content-Type: text/html Content-Length: 26 Last-Modified: Wed, 03 Jan 2018 11:33:54 GMT Connection: keep-alive ETag: "5a4cbfa2-1a" Accept-Ranges: bytes ##backup機器 [root@gary ~]# curl -I 172.16.111.110 HTTP/1.1 200 OK Server: nginx/1.12.2 Date: Fri, 26 Jan 2018 05:54:39 GMT Content-Type: text/html Content-Length: 612 Last-Modified: Tue, 17 Oct 2017 13:25:49 GMT Connection: keep-alive ETag: "59e604dd-264" Accept-Ranges: bytes
[root@gary-tao ~]# ps aux |grep nginx root 9975 0.0 0.2 46852 2956 ? Ss 1月25 0:00 nginx: master process /usr/local/nginx/sbin/nginx -c /usr/local/nginx/conf/nginx.conf root 15423 0.0 0.0 112680 976 pts/1 R+ 14:11 0:00 grep --color=auto nginx nobody 43196 0.0 0.4 48776 4700 ? S 00:10 0:00 nginx: worker process nobody 43197 0.0 0.4 48776 4704 ? S 00:10 0:00 nginx: worker process [root@gary-tao ~]# /etc/init.d/nginx stop //關閉服務 Stopping nginx (via systemctl): [ 肯定 ] [root@gary-tao ~]# ps aux |grep nginx //服務自動起來 root 15492 0.0 0.1 45992 1300 ? Ss 14:11 0:00 nginx: master process /usr/local/nginx/sbin/nginx -c /usr/local/nginx/conf/nginx.conf nobody 15496 0.0 0.3 48480 3948 ? S 14:11 0:00 nginx: worker process nobody 15497 0.0 0.3 48480 3948 ? S 14:11 0:00 nginx: worker process root 15511 0.0 0.0 112680 972 pts/1 R+ 14:11 0:00 grep --color=auto nginx
[root@gary-tao ~]# iptables -I OUTPUT -p vrrp -j DROP //把主上VRRP協議出去的包封掉 [root@gary-tao ~]# iptables -nvL //查看防火牆 Chain INPUT (policy ACCEPT 22 packets, 1608 bytes) pkts bytes target prot opt in out source destination Chain FORWARD (policy ACCEPT 0 packets, 0 bytes) pkts bytes target prot opt in out source destination Chain OUTPUT (policy ACCEPT 15 packets, 1428 bytes) pkts bytes target prot opt in out source destination 19 760 DROP 112 -- * * 0.0.0.0/0 0.0.0.0/0 在backup上查看 [root@gary ~]# ip add 1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN qlen 1 link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 inet 127.0.0.1/8 scope host lo valid_lft forever preferred_lft forever inet6 ::1/128 scope host valid_lft forever preferred_lft forever 2: ens33: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen 1000 link/ether 00:0c:29:2c:5f:75 brd ff:ff:ff:ff:ff:ff inet 172.16.111.110/16 brd 172.16.255.255 scope global ens33 valid_lft forever preferred_lft forever inet 172.16.111.150/32 scope global ens33 valid_lft forever preferred_lft forever inet6 fe80::b6dc:6aed:f1d0:2f43/64 scope link valid_lft forever preferred_lft forever [root@gary ~]# tail /var/log/messages //查看日誌 Jan 26 14:16:15 gary Keepalived_vrrp[2837]: Sending gratuitous ARP on ens33 for 172.16.111.150 Jan 26 14:16:15 gary Keepalived_vrrp[2837]: Sending gratuitous ARP on ens33 for 172.16.111.150 Jan 26 14:16:20 gary Keepalived_vrrp[2837]: Sending gratuitous ARP on ens33 for 172.16.111.150 Jan 26 14:16:20 gary Keepalived_vrrp[2837]: VRRP_Instance(VI_1) Sending/queueing gratuitous ARPs on ens33 for 172.16.111.150 Jan 26 14:16:20 gary Keepalived_vrrp[2837]: Sending gratuitous ARP on ens33 for 172.16.111.150 Jan 26 14:16:20 gary Keepalived_vrrp[2837]: Sending gratuitous ARP on ens33 for 172.16.111.150 Jan 26 14:16:20 gary Keepalived_vrrp[2837]: Sending gratuitous ARP on ens33 for 172.16.111.150 Jan 26 14:16:20 gary Keepalived_vrrp[2837]: Sending gratuitous ARP on ens33 for 172.16.111.150 Jan 26 14:20:01 gary systemd: Started Session 33 of user root. Jan 26 14:20:01 gary systemd: Starting Session 33 of user root.
瀏覽器地址查看:nginx
[root@gary-tao ~]# iptables -F //在master恢復防火牆 //而後backup上查看ip及日誌 [root@gary ~]# ip add 1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN qlen 1 link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 inet 127.0.0.1/8 scope host lo valid_lft forever preferred_lft forever inet6 ::1/128 scope host valid_lft forever preferred_lft forever 2: ens33: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen 1000 link/ether 00:0c:29:2c:5f:75 brd ff:ff:ff:ff:ff:ff inet 172.16.111.110/16 brd 172.16.255.255 scope global ens33 valid_lft forever preferred_lft forever inet6 fe80::b6dc:6aed:f1d0:2f43/64 scope link valid_lft forever preferred_lft forever [root@gary ~]# tail /var/log/messages Jan 26 14:16:20 gary Keepalived_vrrp[2837]: Sending gratuitous ARP on ens33 for 172.16.111.150 Jan 26 14:16:20 gary Keepalived_vrrp[2837]: Sending gratuitous ARP on ens33 for 172.16.111.150 Jan 26 14:16:20 gary Keepalived_vrrp[2837]: Sending gratuitous ARP on ens33 for 172.16.111.150 Jan 26 14:20:01 gary systemd: Started Session 33 of user root. Jan 26 14:20:01 gary systemd: Starting Session 33 of user root. Jan 26 14:30:01 gary systemd: Started Session 34 of user root. Jan 26 14:30:01 gary systemd: Starting Session 34 of user root. Jan 26 14:35:28 gary Keepalived_vrrp[2837]: VRRP_Instance(VI_1) Received advert with higher priority 100, ours 90 Jan 26 14:35:28 gary Keepalived_vrrp[2837]: VRRP_Instance(VI_1) Entering BACKUP STATE Jan 26 14:35:28 gary Keepalived_vrrp[2837]: VRRP_Instance(VI_1) removing protocol VIPs.
瀏覽器地址查看:vim
//關閉master上的keepalived服務 [root@gary-tao ~]# systemctl stop keepalived [root@gary-tao ~]# ip add 1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN qlen 1 link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 inet 127.0.0.1/8 scope host lo valid_lft forever preferred_lft forever inet6 ::1/128 scope host valid_lft forever preferred_lft forever 2: ens33: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen 1000 link/ether 00:0c:29:09:e5:58 brd ff:ff:ff:ff:ff:ff inet 172.16.111.100/16 brd 172.16.255.255 scope global ens33 valid_lft forever preferred_lft forever inet6 fe80::1ffb:cde1:5f3e:5778/64 scope link valid_lft forever preferred_lft forever 3: ens37: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen 1000 link/ether 00:0c:29:09:e5:62 brd ff:ff:ff:ff:ff:ff inet 172.16.111.131/24 brd 172.16.111.255 scope global dynamic ens37 valid_lft 1345sec preferred_lft 1345sec inet6 fe80::888c:a1d7:871b:8971/64 scope link valid_lft forever preferred_lft forever [root@gary-tao ~]# ps aux |grep keep root 20375 0.0 0.0 112680 976 pts/1 R+ 14:51 0:00 grep --color=auto keep [root@gary ~]# ip addr //在backup上VIP立刻就起來了 1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN qlen 1 link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 inet 127.0.0.1/8 scope host lo valid_lft forever preferred_lft forever inet6 ::1/128 scope host valid_lft forever preferred_lft forever 2: ens33: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen 1000 link/ether 00:0c:29:2c:5f:75 brd ff:ff:ff:ff:ff:ff inet 172.16.111.110/16 brd 172.16.255.255 scope global ens33 valid_lft forever preferred_lft forever inet 172.16.111.150/32 scope global ens33 valid_lft forever preferred_lft forever inet6 fe80::b6dc:6aed:f1d0:2f43/64 scope link valid_lft forever preferred_lft forever [root@gary ~]# tail /var/log/messages Jan 26 14:51:01 gary Keepalived_vrrp[2837]: Sending gratuitous ARP on ens33 for 172.16.111.150 Jan 26 14:51:01 gary Keepalived_vrrp[2837]: Sending gratuitous ARP on ens33 for 172.16.111.150 Jan 26 14:51:01 gary Keepalived_vrrp[2837]: Sending gratuitous ARP on ens33 for 172.16.111.150 Jan 26 14:51:01 gary Keepalived_vrrp[2837]: Sending gratuitous ARP on ens33 for 172.16.111.150 Jan 26 14:51:06 gary Keepalived_vrrp[2837]: Sending gratuitous ARP on ens33 for 172.16.111.150 Jan 26 14:51:06 gary Keepalived_vrrp[2837]: VRRP_Instance(VI_1) Sending/queueing gratuitous ARPs on ens33 for 172.16.111.150 Jan 26 14:51:06 gary Keepalived_vrrp[2837]: Sending gratuitous ARP on ens33 for 172.16.111.150 Jan 26 14:51:06 gary Keepalived_vrrp[2837]: Sending gratuitous ARP on ens33 for 172.16.111.150 Jan 26 14:51:06 gary Keepalived_vrrp[2837]: Sending gratuitous ARP on ens33 for 172.16.111.150 Jan 26 14:51:06 gary Keepalived_vrrp[2837]: Sending gratuitous ARP on ens33 for 172.16.111.150
瀏覽器地址示例圖也改變:後端
//啓動master上的keepalived服務 [root@gary-tao ~]# systemctl start keepalived [root@gary-tao ~]# ip add 1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN qlen 1 link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 inet 127.0.0.1/8 scope host lo valid_lft forever preferred_lft forever inet6 ::1/128 scope host valid_lft forever preferred_lft forever 2: ens33: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen 1000 link/ether 00:0c:29:09:e5:58 brd ff:ff:ff:ff:ff:ff inet 172.16.111.100/16 brd 172.16.255.255 scope global ens33 valid_lft forever preferred_lft forever inet 172.16.111.150/32 scope global ens33 valid_lft forever preferred_lft forever inet6 fe80::1ffb:cde1:5f3e:5778/64 scope link valid_lft forever preferred_lft forever 3: ens37: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen 1000 link/ether 00:0c:29:09:e5:62 brd ff:ff:ff:ff:ff:ff inet 172.16.111.131/24 brd 172.16.111.255 scope global dynamic ens37 valid_lft 1084sec preferred_lft 1084sec inet6 fe80::888c:a1d7:871b:8971/64 scope link valid_lft forever preferred_lft forever [root@gary-tao ~]# ps aux |grep keep root 20384 0.0 0.1 120720 1400 ? Ss 14:54 0:00 /usr/sbin/keepalived -D root 20385 0.0 0.3 122792 3100 ? S 14:54 0:00 /usr/sbin/keepalived -D root 20386 0.0 0.2 127116 2836 ? S 14:54 0:00 /usr/sbin/keepalived -D root 20456 0.0 0.0 112680 976 pts/1 R+ 14:55 0:00 grep --color=auto keep [root@gary ~]# ip addr //在backup上VIP立刻斷掉了 1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN qlen 1 link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 inet 127.0.0.1/8 scope host lo valid_lft forever preferred_lft forever inet6 ::1/128 scope host valid_lft forever preferred_lft forever 2: ens33: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen 1000 link/ether 00:0c:29:2c:5f:75 brd ff:ff:ff:ff:ff:ff inet 172.16.111.110/16 brd 172.16.255.255 scope global ens33 valid_lft forever preferred_lft forever inet6 fe80::b6dc:6aed:f1d0:2f43/64 scope link valid_lft forever preferred_lft forever [root@gary ~]# tail /var/log/messages //查看日誌顯示 Jan 26 14:51:01 gary Keepalived_vrrp[2837]: Sending gratuitous ARP on ens33 for 172.16.111.150 Jan 26 14:51:06 gary Keepalived_vrrp[2837]: Sending gratuitous ARP on ens33 for 172.16.111.150 Jan 26 14:51:06 gary Keepalived_vrrp[2837]: VRRP_Instance(VI_1) Sending/queueing gratuitous ARPs on ens33 for 172.16.111.150 Jan 26 14:51:06 gary Keepalived_vrrp[2837]: Sending gratuitous ARP on ens33 for 172.16.111.150 Jan 26 14:51:06 gary Keepalived_vrrp[2837]: Sending gratuitous ARP on ens33 for 172.16.111.150 Jan 26 14:51:06 gary Keepalived_vrrp[2837]: Sending gratuitous ARP on ens33 for 172.16.111.150 Jan 26 14:51:06 gary Keepalived_vrrp[2837]: Sending gratuitous ARP on ens33 for 172.16.111.150 Jan 26 14:55:12 gary Keepalived_vrrp[2837]: VRRP_Instance(VI_1) Received advert with higher priority 100, ours 90 Jan 26 14:55:12 gary Keepalived_vrrp[2837]: VRRP_Instance(VI_1) Entering BACKUP STATE Jan 26 14:55:12 gary Keepalived_vrrp[2837]: VRRP_Instance(VI_1) removing protocol VIPs. [root@gary ~]#
瀏覽器地址示例圖也改變:centos