Lvs+Keepalived實現MySQL高可用

時間  2019-11-17
標籤 lvs+keepalived lvs keepalived 實現 mysql 可用 欄目 負載均衡 简体版
原文   原文鏈接

LVS+Keepalived+MySQL高可用配置php

本文全部配置前提是已實現MySQL雙主備份(MySQL雙主)html

 

安裝前的準備:mysql

VIP:192.168.0.201linux

Keepalived:算法

　　Keepalived-Master:192.168.0.241sql

　　Keepalived-Backup:192.168.0.277服務器

RealServer:併發

　　MySQL1:192.168.0.225負載均衡

　　MySQL2:192.168.0.226工具

 

kernel-devel各版本下載地址:http://rpmfind.net/linux/rpm2html/search.php?query=kernel-devel

keepalived下載地址:http://www.keepalived.org/download.html

 

kernel-devel.2.6.32-431.el6.x86_64.rpm (根據系統實際內核版本)

keepalived-1.2.7.tar.gz

ipvsadm-1.26.tar.gz

依賴包 :

yum install openssl-devle popt-devel popt-static libnl*

 

1、軟件安裝

一、安裝kernel-devel:

rpm -ivh kernel-devel-2.6.32-431.29.2.el6.x86_64.rpm

二、安裝lvs管理工具ipsadm:

tar zxvf ipvsadm-1.26.tar.gz
cd ipvsadm
make 
make install

三、安裝keepalived:

tar zxvf keepalived-1.2.7.tar.gz
cd keepalived
./configure --prefix=/usr/local/keepalived
make
make install

四、將keepalived添加到系統服務:

cp /usr/local/keepalived/sbin/keepalived /usr/sbin/
cp /usr/local/keepalived/etc/sysconfig/keepalived /etc/sysconfig/
cp /usr/local/keepalived/etc/rc.d/init.d/keepalived /etc/init.d/
mkdir /etc/keepalived
cp /usr/local/keepalived/etc/keepalived/keepalived.conf /etc/keepalived/

 

2、配置 

 一、MySQL服務器中的配置 :

將realserver腳本放在/etc/init.d/目錄下，執行便可:

#!/bin/sh
VIP=192.168.0.201

sh /etc/rc.d/init.d/functions

case "$1" in
# 禁用本地的ARP請求、綁定本地迴環地址
start)
    /sbin/ifconfig lo down
    /sbin/ifconfig lo up
    echo "1" >/proc/sys/net/ipv4/conf/lo/arp_ignore
    echo "2" >/proc/sys/net/ipv4/conf/lo/arp_announce
    echo "1" >/proc/sys/net/ipv4/conf/all/arp_ignore
    echo "2" >/proc/sys/net/ipv4/conf/all/arp_announce
    /sbin/sysctl -p >/dev/null 2>&1
    /sbin/ifconfig lo:0 $VIP netmask 255.255.255.255 up #在迴環地址上綁定VIP，設定掩碼，與Direct Server（自身）上的IP保持通訊
    /sbin/route add -host $VIP dev lo:0
    echo "LVS-DR real server starts successfully.\n"
    ;;
stop)
    /sbin/ifconfig lo:0 down
    /sbin/route del $VIP >/dev/null 2>&1
    echo "1" >/proc/sys/net/ipv4/conf/lo/arp_ignore
    echo "2" >/proc/sys/net/ipv4/conf/lo/arp_announce
    echo "1" >/proc/sys/net/ipv4/conf/all/arp_ignore
    echo "2" >/proc/sys/net/ipv4/conf/all/arp_announce
echo "LVS-DR real server stopped.\n"
    ;;
status)
    isLoOn=`/sbin/ifconfig lo:0 | grep "$VIP"`
    isRoOn=`/bin/netstat -rn | grep "$VIP"`
    if [ "$isLoON" == "" -a "$isRoOn" == "" ]; then
        echo "LVS-DR real server has run yet."
    else
        echo "LVS-DR real server is running."
    fi
    exit 3
    ;;
*)
    echo "Usage: $0 {start|stop|status}"
    exit 1
esac
exit 0

而後使用 ip addr 命令查看VIP是否綁定:

1: lo: <LOOPBACK,UP,LOWER_UP> mtu 16436 qdisc noqueue state UNKNOWN 
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
    inet 192.168.0.201/32 brd 192.168.0.201 scope global lo:0
    inet6 ::1/128 scope host 
       valid_lft forever preferred_lft forever
2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen 1000
    link/ether 52:54:00:59:f1:3f brd ff:ff:ff:ff:ff:ff
    inet 192.168.0.225/24 brd 192.168.0.255 scope global eth0
    inet6 fe80::5054:ff:fe59:f13f/64 scope link 
       valid_lft forever preferred_lft forever

能夠看到vip已經綁定在了lo上。兩臺msyql服務器執行一樣的操做。

二、Keepalived的配置:

2.一、首先關閉系統防火牆iptables和selinux

2.二、開啓路由轉發及非本機ip綁定支持功能:

echo 1 >/proc/sys/net/ipv4/ip_forward

echo 1 >/proc/sys/net/ipv4/ip_nonlocal_bind

2.三、修改Keepalived的配置文件/etc/keepalived/keepalived.conf:

! Configuration File for keepalived

global_defs {
   notification_email {
     123@sina.com
   }
   notification_email_from 321@sina.com
   smtp_server 192.168.0.241
   smtp_connect_timeout 30
   router_id LVS1
}

vrrp_instance VI_1 {
    state BACKUP              ＃角色,若都爲BACKUP，則表示當其中一臺掛掉時，以另外一臺爲MASTER，(即不會主動搶VIP)
    interface eth0            ＃主備之間健康檢查的網卡
    virtual_router_id 51      ＃主備id相同
    priority 100
    advert_int 1
    authentication {
        auth_type PASS        ＃主備之間通訊認證
        auth_pass 0711
    }
    virtual_ipaddress {
        192.168.0.201
    }
}

virtual_server 192.168.0.201 3306 {
    delay_loop 6
    lb_algo rr
    lb_kind DR
    nat_mask 255.255.255.0
    persistence_timeout 50
    protocol TCP

    real_server 192.168.0.225 3306 {
        weight 1
        TCP_CHECK {
            connect_timeout 3
            nb_get_retry 3
            delay_before_retry 3
        connect_prot 3306
        }
    }
    real_server 192.168.0.226 3306 {
        weight 1
        TCP_CHECK {
            connect_timeout 3
            nb_get_retry 3
            delay_before_retry 3
        connect_prot 3306
        }
    }
}

keepalived的Master和Backup配置文件不一樣的是，

Master: state爲MASTER，priority的值比BACKUP大。

BACKUP:state爲BACKUP，priority的值比MASTER小。

2.四、啓動Keepalived:

service keepalived start

而後使用ipvsadm工具查看集羣列表:

主、備結果同樣。

[root@localhost ~]# ipvsadm -L -n
IP Virtual Server version 1.2.1 (size=4096)
Prot LocalAddress:Port Scheduler Flags
  -> RemoteAddress:Port           Forward Weight ActiveConn InActConn
TCP  192.168.0.201:3306 rr persistent 50
  -> 192.168.0.225:3306           Route   1      0          0         
  -> 192.168.0.226:3306           Route   1      0          0

查看ip地址狀況:

MASTER:

[root@localhost ~]# ip addr
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 16436 qdisc noqueue state UNKNOWN 
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
    inet6 ::1/128 scope host 
       valid_lft forever preferred_lft forever
2: p4p1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen 1000
    link/ether 64:00:6a:34:c3:e2 brd ff:ff:ff:ff:ff:ff
    inet 192.168.0.241/24 brd 192.168.0.255 scope global p4p1
    inet 192.168.0.201/32 scope global p4p1
    inet6 fe80::6600:6aff:fe34:c3e2/64 scope link 
       valid_lft forever preferred_lft forever

BACKUP：

[root@localhost ~]# ip addr
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 16436 qdisc noqueue state UNKNOWN 
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
    inet6 ::1/128 scope host 
       valid_lft forever preferred_lft forever
2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen 1000
    link/ether 52:54:00:b3:34:e6 brd ff:ff:ff:ff:ff:ff
    inet 192.168.0.227/24 brd 192.168.0.255 scope global eth0
    inet6 fe80::5054:ff:feb3:34e6/64 scope link 
       valid_lft forever preferred_lft forever

能夠看到，MASTER上綁定了VIP，而BACKUP沒有。

三、測試:

3.1keepalived主從測試:

把MASTER的keepalived服務停掉，service keepalived stop ，而後再查看ip:

[root@localhost ~]# service keepalived stop
Stopping keepalived:                                       [  OK  ]

[root@localhost ~]# ip addr
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 16436 qdisc noqueue state UNKNOWN 
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
    inet6 ::1/128 scope host 
       valid_lft forever preferred_lft forever
2: p4p1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen 1000
    link/ether 64:00:6a:34:c3:e2 brd ff:ff:ff:ff:ff:ff
    inet 192.168.0.241/24 brd 192.168.0.255 scope global p4p1
    inet6 fe80::6600:6aff:fe34:c3e2/64 scope link 
       valid_lft forever preferred_lft forever

發現VIP沒了。此時再查看BACKUP的ip:

[root@localhost ~]# ip addr
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 16436 qdisc noqueue state UNKNOWN 
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
    inet6 ::1/128 scope host 
       valid_lft forever preferred_lft forever
2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen 1000
    link/ether 52:54:00:b3:34:e6 brd ff:ff:ff:ff:ff:ff
    inet 192.168.0.227/24 brd 192.168.0.255 scope global eth0
    inet 192.168.0.201/32 scope global eth0
    inet6 fe80::5054:ff:feb3:34e6/64 scope link 
       valid_lft forever preferred_lft forever

VIP成功漂移到了BACKUP上。

同時，經過keepalived的日誌也能夠看出，當master停掉後，backup主動成爲master:

cat /var/log/messages

Jan 25 16:18:28 localhost Keepalived_healthcheckers[1199]: SMTP connection ERROR to [192.168.0.227]:25.
Jan 25 16:42:41 localhost Keepalived_vrrp[1200]: VRRP_Instance(VI_1) Transition to MASTER STATE
Jan 25 16:42:42 localhost Keepalived_vrrp[1200]: VRRP_Instance(VI_1) Entering MASTER STATE Jan 25 16:42:42 localhost Keepalived_vrrp[1200]: VRRP_Instance(VI_1) setting protocol VIPs.
Jan 25 16:42:42 localhost Keepalived_vrrp[1200]: VRRP_Instance(VI_1) Sending gratuitous ARPs on eth0 for 192.168.0.201
Jan 25 16:42:42 localhost Keepalived_healthcheckers[1199]: Netlink reflector reports IP 192.168.0.201 added
Jan 25 16:42:47 localhost Keepalived_vrrp[1200]: VRRP_Instance(VI_1) Sending gratuitous ARPs on eth0 for 192.168.0.201

 3.2MySQL高可用測試:

經過VIP鏈接mysql:

localhost:~ ahaii$ mysql -h192.168.0.201 -uroot -p
Enter password: 
Welcome to the MySQL monitor.  Commands end with ; or \g.
Your MySQL connection id is 643
Server version: 5.5.54-log Source distribution

Copyright (c) 2000, 2015, Oracle and/or its affiliates. All rights reserved.

Oracle is a registered trademark of Oracle Corporation and/or its
affiliates. Other names may be trademarks of their respective
owners.

Type 'help;' or '\h' for help. Type '\c' to clear the current input statement.

mysql>

鏈接成功。

同時在keepalived-master上查看lvs分配狀況:

[root@localhost ~]# ipvsadm -L -n
IP Virtual Server version 1.2.1 (size=4096)
Prot LocalAddress:Port Scheduler Flags
  -> RemoteAddress:Port           Forward Weight ActiveConn InActConn
TCP  192.168.0.201:3306 rr persistent 50
  -> 192.168.0.225:3306           Route   1      0          0         
  -> 192.168.0.226:3306           Route   1      1          0         
[root@localhost ~]#

顯示如今鏈接的是226的那臺mysql服務器。

此時，將226服務器上的mysql服務停掉，而後再經過vip鏈接mysql，並查看lvs分配狀況:

依然能經過vip鏈接mysql:

localhost:~ ahaii$ mysql -h192.168.0.201 -uroot -p
Enter password: 
Welcome to the MySQL monitor.  Commands end with ; or \g.
Your MySQL connection id is 37014
Server version: 5.5.54-log Source distribution

Copyright (c) 2000, 2015, Oracle and/or its affiliates. All rights reserved.

Oracle is a registered trademark of Oracle Corporation and/or its
affiliates. Other names may be trademarks of their respective
owners.

Type 'help;' or '\h' for help. Type '\c' to clear the current input statement.

mysql>

此時，226已被keepalived集羣剔除掉，而且客戶端鏈接的是225服務器 ：

[root@localhost ~]# ipvsadm -L -n
IP Virtual Server version 1.2.1 (size=4096)
Prot LocalAddress:Port Scheduler Flags
  -> RemoteAddress:Port           Forward Weight ActiveConn InActConn
TCP  192.168.0.201:3306 rr persistent 50
  -> 192.168.0.225:3306           Route   1      1          0

當把226服務器的mysql服務從新啓動時，keepalived又將其加入到集羣中:

[root@localhost ~]# ipvsadm -L -n
IP Virtual Server version 1.2.1 (size=4096)
Prot LocalAddress:Port Scheduler Flags
  -> RemoteAddress:Port           Forward Weight ActiveConn InActConn
TCP  192.168.0.201:3306 rr persistent 50
  -> 192.168.0.225:3306           Route   1      1          0         
  -> 192.168.0.226:3306           Route   1      0          0

keepalived剔除、添加226服務器的日誌記錄以下:

Jan 25 17:00:27 localhost Keepalived_healthcheckers[1429]: TCP connection to [192.168.0.226]:3306 failed !!!
Jan 25 17:00:27 localhost Keepalived_healthcheckers[1429]: Removing service [192.168.0.226]:3306 from VS [192.168.0.201]:3306
Jan 25 17:03:27 localhost Keepalived_healthcheckers[1429]: TCP connection to [192.168.0.226]:3306 success.
Jan 25 17:03:27 localhost Keepalived_healthcheckers[1429]: Adding service [192.168.0.226]:3306 to VS [192.168.0.201]:3306

 

疑惑:

一、客戶點請求VIP時，因爲調度器和realserver都有VIP，如何保證是調度器接收請求，而不是realserver？

爲了保證是調度器接收客戶端的請求，在realserver上都作了以下配置:

 

echo "1" >/proc/sys/net/ipv4/conf/lo/arp_ignore
echo "2" >/proc/sys/net/ipv4/conf/lo/arp_announce
echo "1" >/proc/sys/net/ipv4/conf/all/arp_ignore
echo "2" >/proc/sys/net/ipv4/conf/all/arp_announce

 

用arp_ignore參數設定當請求的目標地址是本機IP時，對其ARP的查詢做出不一樣級別的響應。

 1:表示只回答目標IP地址是本地網卡的arp請求

 2:表示只回答目標IP地址是本地網卡的arp請求，而且來訪IP與本機IP在同一網段內。

用arp_announce參數設定本機向外宣告自身IP-MAC時的級別。

 1:表示儘可能避免響應ARP請求中MAC不是本網卡的。如一個主機有多塊網卡，其中一塊網卡接收到了ARP請求，發現所請求的MAC是本機另外一塊網卡的，這個時候接收到ARP請求的這塊網卡就儘可能避免響應。

 2:表示老是使用最合適的網卡來響應。一個主機有多塊網卡，其中一塊網卡接收到了ARP請求，發現所請求的MAC是本機另外一塊網卡的，這個時候接收到ARP請求的這塊網卡就必定不響應，只有發現請求的MAC是本身的纔給與響應。

以上設置，就是爲了接收客戶點請求的是調度器的VIP，而不是realserver的VIP。

 

 

二、爲何負載均衡器和realserver上都有VIP呢？

當均衡器收到客戶端請求後，根據配置的算法選取一臺realserver，將客戶端請求報文的目標MAC地址修改成選取的realserver的MAC地址後從新封裝該數據幀併發送給選取的realserver。realserver收到該數據幀後檢查IP報文發現該報文的目標IP(VIP)是本身，而後對該請求報文作出響應，將響應結果直接發送給客戶端(整個過程當中源地址不變)。

 

以上是整個安裝配置和測試的過程。

相關文章
相關標籤/搜索
每日一句
    每一个你不满意的现在，都有一个你没有努力的曾经。
本站公眾號
   歡迎關注本站公眾號,獲取更多信息
聯繫我們 最近搜索 最新文章 沪ICP备13005482号-10 MyBatis教程 SQL 教程 MySQL教程 Java 教程 Thymeleaf 教程 Hibernate教程 Spring教程 Redis教程