用keepalived配置高可用集羣

一 集羣介紹

一、概述

  • 根據功能劃分爲兩大類:高可用和負載均衡
  • 高可用集羣一般爲兩臺服務器,一臺工做,另一臺做爲冗餘,當提供服務的機器宕機,冗餘將接替繼續提供服務
  • 實現高可用的開源軟件有:heartbeat、keepalived
  • 負載均衡集羣,須要有一臺服務器做爲分發器,它負責把用戶的請求分發給後端的服務器處理,在這個集羣裏,除了分發器外,就是給用戶提供服務的服務器了,這些服務器數量至少爲2
  • 實現負載均衡的開源軟件有LVS、keepalived、haproxy、nginx,商業的有F五、Netscaler
    在阿里、騰訊這些大公司,是不容許服務不可用的,所以在一些核心業務採用的是高可用;heartbeat在centos6中有不少bug,並且好久沒有更新了;keepalived不只有高可用的功能,還有負載均衡的功能;商業性質的負載均衡軟件優點在於有很高的併發量,即便訪問量很大,它也能支撐,而且它的穩定性很好;而使用開源軟件來搭建負載均衡的穩定性就取決於服務器的穩定性

二、keepalived介紹

  • 在這裏咱們使用keepalived來實現高可用集羣,由於heartbeat在centos6上有一些問題,影響實驗效果
  • keepalived經過VRRP(Virtual Router Redundancy Protocl)來實現高可用。
  • 在這個協議裏會將多臺功能相同的路由器組成一個小組,這個小組裏會有1個master角色和N(N>=1)個backup角色。
  • master會經過組播的形式向各個backup發送VRRP協議的數據包,當backup收不到master發來的VRRP數據包時,就會認爲master宕機了。此時就須要根據各個backup的優先級來決定誰成爲新的mater。
  • Keepalived要有三個模塊,分別是core、check和vrrp。其中core模塊爲keepalived的核心,負責主進程的啓動、維護以及全局配置文件的加載和解析,check模塊負責健康檢查,vrrp模塊是來實現VRRP協議的。 VRRP叫作虛擬路由冗餘協議

三、用keepalived配置高可用集羣

  1. 準備兩臺機器130和136,136做爲master,130做爲backup
  2. 兩臺機器都執行yum install -y keepalived
  3. 兩臺機器都安裝nginx,其中136上已經編譯安裝過nginx,130上須要yum安裝nginx: yum install -y nginx
    用keepalived實現高可用,實際上是讓服務器上的某個服務高可用,在這裏咱們是讓nginx實現高可用,用nginx的緣由是不少企業使用nginx做爲負載均衡器,在這裏咱們能夠用如下命令來查看下130上是否已經安裝niginx
# rpm -qa |grep nginx         //查看是否存在nginx的rpm包
# sudo yum install epel-release         //若是系統中找不到nginx的rpm包,則須要執行這兩行命令
# yum update
# yum install -y nginx
  1. 設定虛擬IP 即vip爲100
  2. 編輯136上keepalived配置文件,
    實際上/etc/keepalived/目錄下是存在着keepalived.conf配置文件的,咱們須要先清空keepalived.conf文件原來的內容,使用下面命令來清空:
# > /etc/keepalived/keepalived.conf

結果以下:html

[root@aminglinux ~]# > !$
> /etc/keepalived/keepalived.conf
[root@aminglinux ~]# cat /etc/keepalived/keepalived.conf
[root@aminglinux ~]#

在這裏咱們須要用另外一個份配置文件模板,將下面的內容寫入到/etc/keepalived/keepalived.conf中去mysql

global_defs { //這裏面會定義一些全局參數
notification_email { //若是出現問題,須要給下面郵箱發送郵件 aming@aminglinux.com }
notification_email_from root@aminglinux.com //定義由哪一個郵箱發出郵件
smtp_server 127.0.0.1
smtp_connect_timeout 30
router_id LVS_DEVEL
}
vrrp_script chk_nginx { //檢測服務是否正常
script "/usr/local/sbin/check_ng.sh" //使用這個腳原本檢測,檢測到若是不正常,須要啓動起來
interval 3 //檢測的間斷時間爲3秒
}
vrrp_instance VI_1 { //用於定義master相關的一些東西
state MASTER //定義這臺機器的角色爲master,若是爲從的話,值就爲backup
interface ens33 //定義經過vrrp發廣播的網卡
virtual_router_id 51 //定義路由器的id,與從上這個值同樣
priority 100 //權重,從上的值不同
advert_int 1
authentication { //認證相關的信息
auth_type PASS //定義認證類型爲密碼認證
auth_pass aminglinux>com //定義密碼認證的密碼
}
virtual_ipaddress { //定義一個公有的虛擬ip,做爲域名解析的ip,一旦主宕機,從當即綁定這個ip
192.168.75.100
}
track_script { //加載服務
chk_nginx
}
}linux

  1. 136編輯監控腳本,內容以下

#!/bin/bash
#時間變量,用於記錄日誌
d=`date --date today +%Y%m%d_%H:%M:%S`
#計算nginx進程數量
n=`ps -C nginx --no-heading|wc -l`
#若是進程爲0,則啓動nginx,而且再次檢測nginx進程數量,
#若是還爲0,說明nginx沒法啓動,此時須要關閉keepalived
if [ $n -eq "0" ]; then
/etc/init.d/nginx start
n2=`ps -C nginx --no-heading|wc -l`
if [ $n2 -eq "0" ]; then
echo "$d nginx down,keepalived will stop" >> /var/log/check_ng.log
systemctl stop keepalived
fi
finginx

  1. 給腳本755權限,若是腳本沒有這個權限的話,keepalived服務就不能正常啓動
[root@aminglinux ~]# chmod 755 !$    
chmod 755 /usr/local/sbin/check_ng.sh
  1. systemctl start keepalived 136啓動服務
[root@aminglinux ~]# systemctl start keepalived   
[root@aminglinux ~]# ps aux |grep keepalived  
root      1740  0.0  0.0 120740  1404 ?        Ss   01:31   0:00 /usr/sbin/keepalived -D   
root      1741  0.0  0.1 127476  3272 ?        S    01:31   0:00 /usr/sbin/keepalived -D   
root      1744  0.0  0.1 131780  3116 ?        S    01:31   0:00 /usr/sbin/keepalived -D   
root      1878  0.0  0.0 112676   988 pts/0    S+   01:32   0:00 grep --color=auto keepalived   
[root@aminglinux ~]# ps aux |grep nginx    
root      2365  0.0  0.0  20616   712 ?        Ss   01:33   0:00 nginx: master process /usr/local/nginx/sbin/nginx -c /usr/local/nginx/conf/nginx.conf   
nobody    2366  0.2  0.1  23060  3292 ?        S    01:33   0:00 nginx: worker process   
nobody    2367  0.2  0.1  23060  3296 ?        S    01:33   0:00 nginx: worker process   
root      2381  0.0  0.0 112676   984 pts/0    S+   01:33   0:00 grep --color=auto nginx

這時候若是咱們將nginx停掉,keepalived的腳本會自動重啓nginxsql

[root@aminglinux ~]# /etc/init.d/nginx stop
Stopping nginx (via systemctl):                            [  肯定  ]
[root@aminglinux ~]# ps aux |grep nginx
root     15301  0.0  0.0  20616   708 ?        Ss   02:41   0:00 nginx: master process /usr/local/nginx/sbin/nginx -c /usr/local/nginx/conf/nginx.conf
nobody   15302  0.0  0.1  23060  3292 ?        S    02:41   0:00 nginx: worker process
nobody   15303  0.0  0.1  23060  3280 ?        S    02:41   0:00 nginx: worker process
root     15308  0.0  0.0 112676   984 pts/0    S+   02:41   0:00 grep --color=auto nginx
[root@aminglinux ~]#

咱們也可使用ip addr命令來查看到ens33這個網卡上多綁定了一個IP 192.168.75.100,這個IP就是在keepalived.conf中定義的IP後端

[root@aminglinux ~]# ip addr
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN qlen 1
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
    inet6 ::1/128 scope host 
       valid_lft forever preferred_lft forever
2: ens33: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen 1000
    link/ether 00:0c:29:21:5e:c0 brd ff:ff:ff:ff:ff:ff
    inet 192.168.75.136/24 brd 192.168.75.255 scope global ens33
       valid_lft forever preferred_lft forever
    inet 192.168.75.100/32 scope global ens33
       valid_lft forever preferred_lft forever
    inet 192.168.75.150/24 brd 192.168.75.255 scope global secondary ens33:0
       valid_lft forever preferred_lft forever
    inet6 fe80::d652:b567:6190:8f28/64 scope link 
       valid_lft forever preferred_lft forever
3: ens37: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen 1000
    link/ether 00:0c:29:21:5e:ca brd ff:ff:ff:ff:ff:ff
[root@aminglinux ~]#

下面咱們再來配置從,配置從以前須要先檢查下主和從上是否配置了防火牆或者selinux,須要讓他們關閉centos

  1. 130上編輯配置文件,內容以下

global_defs {
notification_email {
aming@aminglinux.com
} notification_email_from root@aminglinux.com
smtp_server 127.0.0.1
smtp_connect_timeout 30
router_id LVS_DEVEL
}
vrrp_script chk_nginx {
script "/usr/local/sbin/check_ng.sh"
interval 3
}
vrrp_instance VI_1 {
state BACKUP
interface ens33
virtual_router_id 51
priority 90
advert_int 1
authentication {
auth_type PASS
auth_pass aminglinux>com
}
virtual_ipaddress {
192.168.188.100
}
track_script {
chk_nginx
}
}瀏覽器

  1. 130上編輯監控腳本/usr/local/sbin/check_ng.sh,內容以下

#時間變量,用於記錄日誌
d=`date --date today +%Y%m%d_%H:%M:%S`
#計算nginx進程數量
n=`ps -C nginx --no-heading|wc -l`
#若是進程爲0,則啓動nginx,而且再次檢測nginx進程數量,
#若是還爲0,說明nginx沒法啓動,此時須要關閉keepalived
if [ $n -eq "0" ]; then
systemctl start nginx
n2=`ps -C nginx --no-headig|wc -l`
if [ $n2 -eq "0" ]; then
echo "$d nginx down,keepalived will stop" >> /var/log/check_ ng.log
systemctl stop keepalived
fi
fibash

  1. 給腳本755權限
[root@localhost ~]# chmod 755 !$
chmod 755 /usr/local/sbin/check_ng.sh
[root@localhost ~]#
  1. 130上也啓動服務 systemctl start keepalived
[root@localhost ~]# systemctl start keepalived
[root@localhost ~]# ps aux |grep keep
root     10928  0.2  0.0 120740  1404 ?        Ss   02:20   0:00 /usr/sbin/keepalived -D
root     10929  0.2  0.1 120740  2756 ?        S    02:20   0:00 /usr/sbin/keepalived -D
root     10930  0.2  0.1 125104  2840 ?        S    02:20   0:00 /usr/sbin/keepalived -D
root     10945  0.0  0.0 112676   984 pts/1    S+   02:20   0:00 grep --color=auto keep
[root@localhost ~]#

配置完畢,咱們來經過瀏覽器輸入IP 192.168.75.136地址訪問下主服務器,此時訪問到的是/data/wwwroot/default/index.html文件,再來192.168.75.130訪問從服務器,訪問到的默認頁面是/usr/share/nginx/html/index.html;咱們再來使用vip 192.168.75.100訪問,結果訪問到的是主服務器的頁面服務器

四、測試高可用

  1. 先肯定好兩臺機器上nginx差別,好比能夠經過curl -I 來查看nginx版本
  2. 測試1:關閉master上的nginx服務
    在主或從上關閉nginx,keepalived都能自動重啓nginx,這是依賴於咱們寫的監測腳本
  3. 測試2:在master上增長iptabls規則
  4. iptables -I OUTPUT -p vrrp -j DROP
[root@aminglinux ~]# iptables -I OUTPUT -p vrrp -j DROP
[root@aminglinux ~]# iptables -nvL
Chain INPUT (policy ACCEPT 165 packets, 12440 bytes)
 pkts bytes target     prot opt in     out     source               destination         
Chain FORWARD (policy ACCEPT 0 packets, 0 bytes)
 pkts bytes target     prot opt in     out     source               destination         
Chain OUTPUT (policy ACCEPT 105 packets, 11068 bytes)
 pkts bytes target     prot opt in     out     source               destination         
   81  3240 DROP       112  --  *      *       0.0.0.0/0            0.0.0.0/0

這裏咱們將主上經過vrrp協議出去的包丟棄,結果是不能達到切換主從的目的

  1. 測試3:關閉master上的keepalived服務
[root@aminglinux ~]# iptables -F
[root@aminglinux ~]# iptables -nvL
Chain INPUT (policy ACCEPT 12 packets, 840 bytes)
 pkts bytes target     prot opt in     out     source               destination         
Chain FORWARD (policy ACCEPT 0 packets, 0 bytes)
 pkts bytes target     prot opt in     out     source               destination         
Chain OUTPUT (policy ACCEPT 11 packets, 948 bytes)
 pkts bytes target     prot opt in     out     source               destination         
[root@aminglinux ~]# ip addr
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN qlen 1
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
    inet6 ::1/128 scope host 
       valid_lft forever preferred_lft forever
2: ens33: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen 1000
    link/ether 00:0c:29:21:5e:c0 brd ff:ff:ff:ff:ff:ff
    inet 192.168.75.136/24 brd 192.168.75.255 scope global ens33
       valid_lft forever preferred_lft forever
    inet 192.168.75.100/32 scope global ens33
       valid_lft forever preferred_lft forever
    inet 192.168.75.150/24 brd 192.168.75.255 scope global secondary ens33:0
       valid_lft forever preferred_lft forever
    inet6 fe80::d652:b567:6190:8f28/64 scope link 
       valid_lft forever preferred_lft forever
3: ens37: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen 1000
    link/ether 00:0c:29:21:5e:ca brd ff:ff:ff:ff:ff:ff
[root@aminglinux ~]# systemctl stop keepalived
[root@aminglinux ~]# ip addr
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN qlen 1
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
    inet6 ::1/128 scope host 
       valid_lft forever preferred_lft forever
2: ens33: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen 1000
    link/ether 00:0c:29:21:5e:c0 brd ff:ff:ff:ff:ff:ff
    inet 192.168.75.136/24 brd 192.168.75.255 scope global ens33
       valid_lft forever preferred_lft forever
    inet 192.168.75.150/24 brd 192.168.75.255 scope global secondary ens33:0
       valid_lft forever preferred_lft forever
    inet6 fe80::d652:b567:6190:8f28/64 scope link 
       valid_lft forever preferred_lft forever
3: ens37: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen 1000
    link/ether 00:0c:29:21:5e:ca brd ff:ff:ff:ff:ff:ff
[root@aminglinux ~]#

咱們能夠看到,關閉主上的keepalived服務,相似主機宕機,主機上綁定的192.168.75.100這個IP被解綁,而下面咱們到從上查看ip addr,能夠看到從上已經綁定了192.168.75.100這個IP,用瀏覽器訪問這個IP,訪問到的頁面爲從機頁面,說明主從已經切換

[root@localhost ~]# ip addr
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN qlen 1
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
    inet6 ::1/128 scope host 
       valid_lft forever preferred_lft forever
2: ens33: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen 1000
    link/ether 00:0c:29:0c:20:c9 brd ff:ff:ff:ff:ff:ff
    inet 192.168.75.130/24 brd 192.168.75.255 scope global ens33
       valid_lft forever preferred_lft forever
    inet 192.168.75.100/32 scope global ens33
       valid_lft forever preferred_lft forever
    inet6 fe80::b44e:aca4:f738:7833/64 scope link 
       valid_lft forever preferred_lft forever
[root@localhost ~]# tail /var/log/messages
Apr 10 03:04:15 localhost Keepalived_vrrp[12554]: Sending gratuitous ARP on ens33 for 192.168.75.100
Apr 10 03:04:15 localhost Keepalived_vrrp[12554]: Sending gratuitous ARP on ens33 for 192.168.75.100
Apr 10 03:04:15 localhost Keepalived_vrrp[12554]: Sending gratuitous ARP on ens33 for 192.168.75.100
Apr 10 03:04:15 localhost Keepalived_vrrp[12554]: Sending gratuitous ARP on ens33 for 192.168.75.100
Apr 10 03:04:20 localhost Keepalived_vrrp[12554]: Sending gratuitous ARP on ens33 for 192.168.75.100
Apr 10 03:04:20 localhost Keepalived_vrrp[12554]: VRRP_Instance(VI_1) Sending/queueing gratuitous ARPs on ens33 for 192.168.75.100
Apr 10 03:04:20 localhost Keepalived_vrrp[12554]: Sending gratuitous ARP on ens33 for 192.168.75.100
Apr 10 03:04:20 localhost Keepalived_vrrp[12554]: Sending gratuitous ARP on ens33 for 192.168.75.100
Apr 10 03:04:20 localhost Keepalived_vrrp[12554]: Sending gratuitous ARP on ens33 for 192.168.75.100
Apr 10 03:04:20 localhost Keepalived_vrrp[12554]: Sending gratuitous ARP on ens33 for 192.168.75.100
[root@localhost ~]#
  1. 測試4:啓動master上的keepalived服務
[root@aminglinux ~]# systemctl start keepalived
[root@aminglinux ~]# ip addr
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN qlen 1
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
    inet6 ::1/128 scope host 
       valid_lft forever preferred_lft forever
2: ens33: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen 1000
    link/ether 00:0c:29:21:5e:c0 brd ff:ff:ff:ff:ff:ff
    inet 192.168.75.136/24 brd 192.168.75.255 scope global ens33
       valid_lft forever preferred_lft forever
    inet 192.168.75.100/32 scope global ens33
       valid_lft forever preferred_lft forever
    inet 192.168.75.150/24 brd 192.168.75.255 scope global secondary ens33:0
       valid_lft forever preferred_lft forever
    inet6 fe80::d652:b567:6190:8f28/64 scope link 
       valid_lft forever preferred_lft forever
3: ens37: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen 1000
    link/ether 00:0c:29:21:5e:ca brd ff:ff:ff:ff:ff:ff
[root@aminglinux ~]#

而後咱們再到從機上查看綁定IP

[root@localhost ~]# ip addr
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN qlen 1
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
    inet6 ::1/128 scope host 
       valid_lft forever preferred_lft forever
2: ens33: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen 1000
    link/ether 00:0c:29:0c:20:c9 brd ff:ff:ff:ff:ff:ff
    inet 192.168.75.130/24 brd 192.168.75.255 scope global ens33
       valid_lft forever preferred_lft forever
    inet6 fe80::b44e:aca4:f738:7833/64 scope link 
       valid_lft forever preferred_lft forever
[root@localhost ~]#

在實際生產環境中,可能會一主多從,這時咱們能夠在keepalived.conf中給每一個從配置不一樣的權重priority,權值越高,則優先級越高 除了nginx的高可用,咱們還能夠作mysql的高可用,若是要作mysql高可用的話,必定要保證主從數據一致

相關文章
相關標籤/搜索