MySQL主主複製+LVS+Keepalived實現MySQL高可用性

時間 2019-12-08

標籤 mysql 複製 lvs+keepalived lvs keepalived 實現用性欄目 MySQL 简体版

原文原文鏈接

MySQL複製可以保證數據的冗餘的同時能夠作讀寫分離來分擔系統壓力，若是是主主複製還能夠很好的避免主節點的單點故障。可是MySQL主主複製存在一些問題沒法知足咱們的實際須要：未提供統一訪問入口來實現負載均衡，若是其中master宕掉的話須要手動切換到另一個master，而不能自動進行切換。html

這篇文章下面要介紹如何經過LVS+Keepalived的方式來是實現MySQL的高可用性，同時解決以上問題。前端

Keepalived和LVS介紹

Keepalived是一個基於VRRP（虛擬路由冗餘協議）可用來實現服務高可用性的軟件方案，避免出現單點故障。Keepalived通常用來實現輕量級高可用性，且不須要共享存儲，通常用於兩個節點之間，常見有LVS+Keepalived、Nginx+Keepalived組合。mysql

LVS(Linux Virtual Server)是一個高可用性虛擬的服務器集羣系統。本項目在1998年5月由章文嵩博士成立，是中國國內最先出現的自由軟件項目之一。
LVS主要用於多服務器的負載均衡，做用於網絡層。LVS構建的服務器集羣系統中，前端的負載均衡層被稱爲Director Server；後端提供服務的服務器組層被稱爲Real Server。經過下圖能夠大體瞭解LVS的基礎架構。
linux

LVS有三種工做模式，分別是DR（Direct Routing 直接路由）、TUN(Tunneling IP隧道）、NAT（Network Address Translation 網絡地址轉換）。其中TUN模式可以支持更多的Real Server，但須要全部服務器支持IP隧道協議；DR也能夠支持至關的Real Server，但須要保證Director Server虛擬網卡與物理網卡在同一網段；NAT擴展性有限，沒法支持更多的Real Server，由於全部的請求包和應答包都須要Director Server進行解析再生，影響效率。同時，LVS負載均衡有10中調度算法，分別是rr、wrr、lc、wlc、lblc、lblcr、dh、sh、sed、nq算法

詳細的LVS說明請參見傳送門sql

本文中將利用LVS實現MySQL的讀寫負載均衡，Keepalived避免節點出現單點故障。vim

LVS+Keepalived配置

環境準備

LVS1：192.168.1.2後端

LVS2：192.168.1.11服務器

MySQL Server1：192.168.1.5網絡

MySQL Server2：192.168.1.6

VIP：192.168.1.100

OS: CentOS 6.4

Keepalive安裝

keepalived下載地址

須要安裝如下軟件包

# yum install -y kernel-devel openssl openssl-devel

解壓keepalived到/usr/local/並進入目錄執行配置編譯

# ./configure --prefix=/usr/local/keepalived --with-kernel-dir=/usr/src/kernels/2.6.32-431.5.1.el6.x86_64/
Keepalived configuration
------------------------
Keepalived version       : 1.2.13
Compiler                 : gcc
Compiler flags           : -g -O2
Extra Lib                : -lssl -lcrypto -lcrypt 
Use IPVS Framework       : Yes
IPVS sync daemon support : Yes
IPVS use libnl           : No
fwmark socket support    : Yes
Use VRRP Framework       : Yes
Use VRRP VMAC            : Yes
SNMP support             : No
SHA1 support             : No
Use Debug flags          : No

# make && make install

默認狀況下keepalived啓動時會去/etc/keepalived目錄下找配置文件，將須要的配置文件拷貝到指定位置

# cp /usr/local/keepalived/etc/rc.d/init.d/keepalived /etc/rc.d/init.d/
# cp /usr/local/keepalived/etc/sysconfig/keepalived /etc/sysconfig/
# cp /usr/local/keepalived/etc/keepalived/keepalived.conf /etc/keepalived/
# cp /usr/local/keepalived/sbin/keepalived /usr/sbin/
# chkconfig mysqld on
# chkconfig keepalived on

LVS安裝

ipvsadm下載地址

須要安裝如下軟件包

# yum install -y libnl* popt*

查看是否加載lvs模塊

# modprobe -l |grep ipvs

解壓安裝

# ln -s /usr/src/kernels/2.6.32-431.5.1.el6.x86_64/ /usr/src/linux
# tar -zxvf ipvsadm-1.26.tar.gz
# make && make install

LVS安裝完成，查看當前LVS集羣

# ipvsadm -L -n
IP Virtual Server version 1.2.1 (size=4096)
Prot LocalAddress:Port Scheduler Flags
-> RemoteAddress:Port           Forward Weight ActiveConn InActConn

LVS+Keepalived配置

搭建MySQL主主複製

這裏再也不贅述，請參考MySQL複製

配置Keepalived

下面是LVS1節點（Keepalived主節點）上的Keepalived配置，LVS2相似

# vim /etc/keepalived/keepalived.conf    

! Configuration File for keepalived

global_defs {
   router_id LVS1
}

vrrp_instance VI_1 {
    state MASTER #指定instance初始狀態，實際根據優先級決定.backup節點不同
    interface eth0 #虛擬IP所在網
    virtual_router_id 51 #VRID，相同VRID爲一個組，決定多播MAC地址
    priority 100 #優先級，另外一臺改成90.backup節點不同
    advert_int 1  #檢查間隔
    authentication {
        auth_type PASS  #認證方式，能夠是pass或ha
        auth_pass 1111  #認證密碼
    }
    virtual_ipaddress {
        192.168.1.100  #VIP
    }
}

virtual_server 192.168.1.100 3306 {
    delay_loop 6  #服務輪詢的時間間隔
    lb_algo wrr  #加權輪詢調度，LVS調度算法 rr|wrr|lc|wlc|lblc|sh|sh
    lb_kind DR   #LVS集羣模式 NAT|DR|TUN，其中DR模式要求負載均衡器網卡必須有一塊與物理網卡在同一個網段
    #nat_mask 255.255.255.0
    persistence_timeout 50  #會話保持時間
    protocol TCP  #健康檢查協議

     ## Real Server設置，3306就是MySQL鏈接端口
    real_server 192.168.1.5 3306 {
        weight 3  ##權重
        TCP_CHECK {
            connect_timeout 3
            nb_get_retry 3
            delay_before_retry 3
            connect_port 3306
        }
    }
    real_server 192.168.1.6 3306 {
        weight 3
        TCP_CHECK {
            connect_timeout 3
            nb_get_retry 3
            delay_before_retry 3
            connect_port 3306
        }
    }
}

配置LVS

編寫LVS啓動腳本/etc/init.d/realserver

#!/bin/sh
VIP=192.168.1.100
. /etc/rc.d/init.d/functions

case "$1" in
# 禁用本地的ARP請求、綁定本地迴環地址
start)
    /sbin/ifconfig lo down
    /sbin/ifconfig lo up
    echo "1" >/proc/sys/net/ipv4/conf/lo/arp_ignore
    echo "2" >/proc/sys/net/ipv4/conf/lo/arp_announce
    echo "1" >/proc/sys/net/ipv4/conf/all/arp_ignore
    echo "2" >/proc/sys/net/ipv4/conf/all/arp_announce
    /sbin/sysctl -p >/dev/null 2>&1
    /sbin/ifconfig lo:0 $VIP netmask 255.255.255.255 up #在迴環地址上綁定VIP，設定掩碼，與Direct Server（自身）上的IP保持通訊
    /sbin/route add -host $VIP dev lo:0
    echo "LVS-DR real server starts successfully.\n"
    ;;
stop)
    /sbin/ifconfig lo:0 down
    /sbin/route del $VIP >/dev/null 2>&1
    echo "1" >/proc/sys/net/ipv4/conf/lo/arp_ignore
    echo "2" >/proc/sys/net/ipv4/conf/lo/arp_announce
    echo "1" >/proc/sys/net/ipv4/conf/all/arp_ignore
    echo "2" >/proc/sys/net/ipv4/conf/all/arp_announce
echo "LVS-DR real server stopped.\n"
    ;;
status)
    isLoOn=`/sbin/ifconfig lo:0 | grep "$VIP"`
    isRoOn=`/bin/netstat -rn | grep "$VIP"`
    if [ "$isLoON" == "" -a "$isRoOn" == "" ]; then
        echo "LVS-DR real server has run yet."
    else
        echo "LVS-DR real server is running."
    fi
    exit 3
    ;;
*)
    echo "Usage: $0 {start|stop|status}"
    exit 1
esac
exit 0

將lvs腳本加入開機自啓動

# chmod +x /etc/init.d/realserver
# echo "/etc/init.d/realserver" >> /etc/rc.d/rc.local

分別啓動LVS和keepalived

# service realserver start
# service keepalived start

注意此時網卡的變化，能夠看到虛擬網卡已經分配到了realserver上。

此時查看LVS集羣狀態，能夠看到集羣下有兩個Real Server，調度算法，權重等信息。ActiveConn表明當前Real Server的活躍鏈接數

# ipvsadm -ln
IP Virtual Server version 1.2.1 (size=4096)
Prot LocalAddress:Port Scheduler Flags
  -> RemoteAddress:Port           Forward Weight ActiveConn InActConn
TCP  192.168.1.100:3306 wrr persistent 50
  -> 192.168.1.5:3306             Route   3      4          1         
  -> 192.168.1.6:3306             Route   3      0          2

此時LVS+Keepalived+MySQL主主複製已經搭建完成。

測試驗證

功能性驗證

關閉MySQL Server2

# service mysqld stop

在LVS1查看/var/log/messages中關於keepalived日誌，LVS1檢測到了MySQL Server2宕機，同時LVS集羣自動剔除了故障節點

Sep  9 13:50:53 192.168.1.2 Keepalived_healthcheckers[18797]: TCP connection to [192.168.1.6]:3306 failed !!!
Sep  9 13:50:53 192.168.1.2 Keepalived_healthcheckers[18797]: Removing service [192.168.1.6]:3306 from VS [192.168.1.100]:3306

重新啓動MySQL Server2後自動將故障節點自動加入LVS集羣

Sep  9 13:51:41 192.168.1.2 Keepalived_healthcheckers[18797]: TCP connection to [192.168.1.6]:3306 success.
Sep  9 13:51:41 192.168.1.2 Keepalived_healthcheckers[18797]: Adding service [192.168.1.6]:3306 to VS [192.168.1.100]:3306

關閉LVS1上的Keepalived（模擬宕機操做），查看LVS1上的日誌，能夠看到Keepalived移出了LVS1上的VIP

Sep  9 14:01:27 192.168.1.2 Keepalived[18796]: Stopping Keepalived v1.2.13 (09/09,2014)
Sep  9 14:01:27 192.168.1.2 Keepalived_healthcheckers[18797]: Removing service [192.168.1.5]:3306 from VS [192.168.1.100]:3306
Sep  9 14:01:27 192.168.1.2 Keepalived_healthcheckers[18797]: Removing service [192.168.1.6]:3306 from VS [192.168.1.100]:3306
Sep  9 14:01:27 192.168.1.2 Keepalived_vrrp[18799]: VRRP_Instance(VI_1) sending 0 priority
Sep  9 14:01:27 192.168.1.2 Keepalived_vrrp[18799]: VRRP_Instance(VI_1) removing protocol VIPs.

同時查看LVS2上日誌，能夠看到LVS2成爲了Master，並接管了VIP

Sep  9 14:11:24 192.168.1.11 Keepalived_vrrp[7457]: VRRP_Instance(VI_1) Transition to MASTER STATE
Sep  9 14:11:25 192.168.1.11 Keepalived_vrrp[7457]: VRRP_Instance(VI_1) Entering MASTER STATE
Sep  9 14:11:25 192.168.1.11 Keepalived_vrrp[7457]: VRRP_Instance(VI_1) setting protocol VIPs.
Sep  9 14:11:25 192.168.1.11 Keepalived_vrrp[7457]: VRRP_Instance(VI_1) Sending gratuitous ARPs on eth0 for 192.168.1.100
Sep  9 14:11:25 192.168.1.11 Keepalived_healthcheckers[7456]: Netlink reflector reports IP 192.168.1.100 added
Sep  9 14:11:25 192.168.1.11 avahi-daemon[1407]: Registering new address record for 192.168.1.100 on eth0.IPv4.
Sep  9 14:11:30 192.168.1.11 Keepalived_vrrp[7457]: VRRP_Instance(VI_1) Sending gratuitous ARPs on eth0 for 192.168.1.100

在LVS2上查看LVS集羣狀態，一切正常。

# ipvsadm -ln
IP Virtual Server version 1.2.1 (size=4096)
Prot LocalAddress:Port Scheduler Flags
  -> RemoteAddress:Port           Forward Weight ActiveConn InActConn
TCP  192.168.1.100:3306 wrr persistent 50
  -> 192.168.1.5:3306             Route   3      2          0         
  -> 192.168.1.6:3306             Route   3      1          0