Keepalived與MySQL互爲主從自動切換配置

時間 2019-11-06

標籤 keepalived mysql 互爲主從自動切換配置欄目負載均衡简体版

原文原文鏈接

爲解決Mysql數據庫單點問題，實現兩臺MySQL數據庫互爲主備，雙向replication。當一Master出現問題，則將Slave切換爲Master繼續工做.mysql

環境說明sql

系統版本：CentOS Linux release 7.6.1810 (Core)數據庫

MySQL版本：mysql Ver 14.14 Distrib 5.7.27vim

keepalived版本：Keepalived v1.2.13安全

序號　　服務器IP 　　　　用途服務器

1 　　　　192.168.158.10 Master
2 　　　　192.168.158.20 Slave
3 　　　　192.168.158.30 VIPsession

1、MySQL互爲主從配置ssh

1.> 兩臺安裝相同版本的MySQL數據庫.
2.> 主備機NTP時鐘同步
3.> 雙機互信配置ssh免密認證
4.> 數據庫配置(Master的配置和Slave的配置server-id不能一致,別的均可以同樣)
4.1> 修改Master主機上MySQL數據庫的配置文件，而後新啓動MySQLsocket

#vim /ect/my.cnf
[mysqld]
log-bin=mysql-bin
server-id=100
expire_logs_days = 10
datadir=/var/lib/mysql
socket=/var/lib/mysql/mysql.sock
symbolic-links=0
log-error=/var/log/mysqld.log
pid-file=/var/run/mysqld/mysqld.pid
validate_password=off         #關閉密碼安全策略
default_password_lifetime=0     #設置密碼不過時
log_bin=/var/log/mysql/mysql-bin

4.2> 修改Slave主機上MySQL數據庫的配置文件，而後新啓動MySQLui

#vim /ect/my.cnf

[mysqld]
log-bin=mysql-bin
server-id=101
expire_logs_days = 10
datadir=/var/lib/mysql
socket=/var/lib/mysql/mysql.sock
symbolic-links=0
log-error=/var/log/mysqld.log
pid-file=/var/run/mysqld/mysqld.pid
validate_password=off           
default_password_lifetime=0     
log_bin=/var/log/mysql/mysql-bin

5.> 啓動MySQL服務

# systemctl start mysqld

6.> 查詢相關狀態，以Master主機爲例，以下

mysql> show master status;
+------------------+----------+--------------+------------------+-------------------+
| File             | Position | Binlog_Do_DB | Binlog_Ignore_DB | Executed_Gtid_Set |
+------------------+----------+--------------+------------------+-------------------+
| mysql-bin.000001 |      154 |              |                  |                   |
+------------------+----------+--------------+------------------+-------------------+
1 row in set (0.00 sec)

7.> 建立複製帳號並同步
7.1> 在Master庫和Slave庫分別執行，建立數據同步複製帳號.

mysql> GRANT REPLICATION SLAVE,REPLICATION CLIENT ON *.* TO replication@'%' IDENTIFIED BY 'replication';
mysql> flush  privileges;

7.2> 7.2> 在Master主機上,執行同步操做(注意master_host參數主備機相互指向),以下：

mysql> change master to master_host='192.168.158.20',master_port=3306,master_user='replication',master_password='replication',master_log_file='mysql-bin.000001',master_log_pos=154;
mysql> start slave;

7.3> 在Slave主機上,執行同步操做(注意master_host參數主備機相互指向),以下：

mysql> change master to master_host='192.168.158.10',master_port=3306,master_user='replication',master_password='replication',master_log_file='mysql-bin.000001',master_log_pos=154;
mysql> start slave;

7.4> 在Master、Slave主機上，查詢同步狀態「show slave status\G」，檢查結果中Slave_IO_Running: Yes和Slave_SQL_Running: Yes，不然有異常。

8.> 配置密文命令訪問(兩臺主機都配置)
Mysql數據庫使用mysql或mysqldump等相關命令時，須要在命令行界面輸入密碼，當使用腳本時，在腳本里填寫密碼顯然不太安全，所以能夠設置Mysql的密文文件。

# mysql_config_editor set --login-path=local --user=root --port=3306 --password
# mysql_config_editor print --all

9.> 建立切換腳本

切換腳本規劃，如本次是mysql切換，所以在該目錄下建立mysql目錄,將全部切換腳本放在/home/mysql目錄下,本次相關腳本說明以下：

進入/home/mysql目錄，以下文件：

Logs 　　 //存儲日誌的文件目錄

mybackup.sh //清空slave配置，從新獲取遠程日誌文件及Pos,並開啓同步

mycheck.sh //檢查mysql運行狀態，若是運行正常，退出。若是運行不正常調用pkill keepalived

mymaster.sh //先判斷同步複製是否執行完成，若是未執行完成等待1分鐘後，中止同步（stop slave;），而且記錄切換後的日誌和pos

.mysqlenv //腳本運行環境文件

mystop.sh //設置參數保證數據不丟失，最後檢查看是否還有寫操做，最後1分鐘退出

syncposfile //每次切換後，Master最後一次File值和Position值。

10.環境文件

10.1> Master主機端的環境文件

[root@localhost mysql]# vim .mysqlenv

MYSQL=/usr/bin/mysql
MYSQL_CMD="--login-path=local"
#遠端主機的IP地址
REMOTE_IP=192.168.158.20
export mysql="$MYSQL $MYSQL_CMD "

10.2> Slave主機端的環境文件

[root@localhost mysql]# vim .mysqlenv
MYSQL=/usr/bin/mysql
MYSQL_CMD="--login-path=local"
#遠端主機的IP地址
REMOTE_IP=192.168.158.10
export mysql="$MYSQL $MYSQL_CMD"

11.> 服務檢查腳本

11.1> mycheck.sh

[root@localhost mysql]# vim mycheck.sh 
#!/bin/sh

##################################################
#File Name  : mycheck.sh
#Description: mysql is working MYSQL_OK is 1
#             mysql is down MYSQL_OK is 0
##################################################

BASEPATH=/home/mysql
LOGSPATH=$BASEPATH/logs
source $BASEPATH/.mysqlenv

CHECK_TIME=3
MYSQL_OK=1
##################################################################
function check_mysql_helth (){
  $mysql -e "show status;" >/dev/null 2>&1
  if [ $? == 0 ] 
  then 
    MYSQL_OK=1
  else
    MYSQL_OK=0
    #systemctl status keepalived
 fi
 return $MYSQL_OK
}

#check_mysql_helth
while [ $CHECK_TIME -ne 0 ] #不等於
do
    let "CHECK_TIME -= 1"
    check_mysql_helth
    if [ $MYSQL_OK = 1 ] ; then
    CHECK_TIME=0
    echo "$(date "+%Y-%m-%d %H:%M:%S") The scripts mycheck.sh is running ..." >> $LOGSPATH/mysql_switch.log
    exit 0
    fi
    if [ $MYSQL_OK -eq 0 ] && [ $CHECK_TIME -eq 0 ] #等於
    then
    systemctl stop keepalived
    echo "$(date "+%Y-%m-%d %H:%M:%S") The mycheck.sh, mysql is down, after switch..." >> $LOGSPATH/mysql_switch.log
    exit 1
    fi
    sleep 1　　
done
[root@localhost mysql]#

11.2> 切換腳本

[root@localhost mysql]# vim mymaster.sh 
#!/bin/sh

##################################################
#File Name  : mymaster.sh
#Description: First determine whether synchronous
#             replication is performed, and if no
#             execution is completed, wait for 1
#             minutes. Log logs and POS after
#             switching, and record files synchronously.
##################################################

BASEPATH=/home/mysql
LOGSPATH=$BASEPATH/logs
source $BASEPATH/.mysqlenv

$mysql -e "show slave status\G" > $LOGSPATH/mysqlslave.states
Master_Log_File=`cat $LOGSPATH/mysqlslave.states | grep -w Master_Log_File | awk -F": " '{print $2}'`
Relay_Master_Log_File=`cat $LOGSPATH/mysqlslave.states | grep -w Relay_Master_Log_File | awk -F": " '{print $2}'`
Read_Master_Log_Pos=`cat $LOGSPATH/mysqlslave.states | grep -w Read_Master_Log_Pos | awk -F": " '{print $2}'`
Exec_Master_Log_Pos=`cat $LOGSPATH/mysqlslave.states | grep -w Exec_Master_Log_Pos | awk -F": " '{print $2}'`
i=1

while true
do
    if [ $Master_Log_File = $Relay_Master_Log_File ] && [ $Read_Master_Log_Pos -eq $Exec_Master_Log_Pos ];then
        echo "$(date "+%Y-%m-%d %H:%M:%S") The mymaster.sh, slave sync ok... " >> $LOGSPATH/mysql_switch.log
        break
    else
        sleep 1
        if [ $i -gt 60 ];then
            break
        fi
        continue
        let i++
    fi
done

$mysql -e "stop slave;"
$mysql -e "set global innodb_support_xa=0;"
$mysql -e "set global sync_binlog=0;"
$mysql -e "set global innodb_flush_log_at_trx_commit=0;"
$mysql -e "flush logs;GRANT ALL PRIVILEGES ON *.* TO 'replication'@'%' IDENTIFIED BY 'replication';flush privileges;"
$mysql -e "show master status;" > $LOGSPATH/master_status_$(date "+%y%m%d-%H%M").txt

# sync pos file
/usr/bin/scp $LOGSPATH/master_status_$(date "+%y%m%d-%H%M").txt root@$REMOTE_IP:$BASEPATH/syncposfile/backup_master.status
echo "$(date "+%Y-%m-%d %H:%M:%S") The mymaster.sh, Sync pos file sucess." >> $LOGSPATH/mysql_switch.log
[root@localhost mysql]#

11.3> 回切腳本

[root@localhost mysql]# vim mybackup.sh 
#!/bin/sh

##################################################
#File Name  : mybackup.sh
#Description: Empty the slave configuration, retrieve
#             the remote log file and Pos, and open
#             the synchronization
##################################################

BASEPATH=/home/mysql
LOGSPATH=$BASEPATH/logs
source $BASEPATH/.mysqlenv

$mysql -e "GRANT ALL PRIVILEGES ON *.* TO 'replication'@'%' IDENTIFIED BY 'replication';flush privileges;"
$mysql -e "set global innodb_support_xa=0;"
$mysql -e "set global sync_binlog=0;"
$mysql -e "set global innodb_flush_log_at_trx_commit=0;"
$mysql -e "flush logs;"
$mysql -e "reset slave all;"

if [ -f $BASEPATH/syncposfile/backup_master.status ];then
        New_ReM_File=`cat $BASEPATH/syncposfile/backup_master.status | grep -v File |awk '{print $1}'`
        New_ReM_Position=`cat $BASEPATH/syncposfile/backup_master.status | grep -v File |awk '{print $2}'`
        echo "$(date "+%Y-%m-%d %H:%M:%S") This mybackup.sh, New_ReM_File:$New_ReM_File,New_ReM_Position:$New_ReM_Position" >> $LOGSPATH/mysql_switch.log
        $mysql -e "change master to master_host='$REMOTE_IP',master_port=3306,master_user='replication',master_password='replication',master_log_file='$New_ReM_File',master_log_pos=$New_ReM_Position;"
        $mysql -e "start slave;"
        $mysql -e "show slave status\G;" > $LOGSPATH/slave_status_$(date "+%y%m%d-%H%M").txt
        cat $LOGSPATH/slave_status_$(date "+%y%m%d-%H%M").txt >> $LOGSPATH/mysql_switch.log
        rm -f $BASEPATH/syncposfile/backup_master.status
else
    echo "$(date "+%Y-%m-%d %H:%M:%S") The scripts mybackup.sh running error..." >> $LOGSPATH/mysql_switch.log
fi
[root@localhost mysql]#

11.4> 中止腳本

[root@localhost mysql]# vim mystop.sh 
#!/bin/sh

##################################################
#File Name  : mystop.sh
#Description: Set parameters to ensure that the data
#             is not lost, and finally check to see
#             if there are still write operations,
#             the last 1 minutes to exit

##################################################

BASEPATH=/home/mysql
LOGSPATH=$BASEPATH/logs
source $BASEPATH/.mysqlenv

$mysql -e "GRANT ALL PRIVILEGES ON *.* TO 'replication'@'%' IDENTIFIED BY 'replication';flush privileges;"
$mysql -e "set global innodb_support_xa=1;"
$mysql -e "set global sync_binlog=1;"
$mysql -e "set global innodb_flush_log_at_trx_commit=1;"
$mysql -e "show master status\G" > $LOGSPATH/mysqlmaster0.states
M_File1=`cat $LOGSPATH/mysqlmaster0.states | awk -F': ' '/File/{print $2}'`
M_Position1=`cat $LOGSPATH/mysqlmaster0.states | awk -F': ' '/Position/{print $2}'`
sleep 2
$mysql -e "show master status\G" > $LOGSPATH/mysqlmaster1.states
M_File2=`cat $LOGSPATH/mysqlmaster1.states | awk -F': ' '/File/{print $2}'`
M_Position2=`cat $LOGSPATH/mysqlmaster1.states | awk -F': ' '/Position/{print $2}'`

i=1

while true
do
    if [ $M_File1 = $M_File2 ] && [ $M_Position1 -eq $M_Position2 ];then
        echo "$(date "+%Y-%m-%d %H:%M:%S") The mystop.sh, master sync ok.." >> $LOGSPATH/mysql_switch.log
        exit 0
    else
        sleep 1
        if [$i -gt 60 ];then
            break
        fi
        continue
        let i++
    fi
done
echo "$(date "+%Y-%m-%d %H:%M:%S") The mystop.sh, master sync exceed one minutes..." >> $LOGSPATH/mysql_switch.log
[root@localhost mysql]#

2、Keepalived安裝與配置

1.兩臺都安裝Keepalived(略)

2.切換原理

Keepalived可實現將虛擬IP地址在實體物理機上來回漂移。Keepalived在轉換狀態時會依照狀態來呼叫配置文件中內置的定義。
當進入Master狀態時會呼叫notify_master定義的腳本
當進入Backup狀態時會呼叫notify_backup定義的腳本
當keepalived程序終止時呼叫notify_stop定義的腳本
當發現異常狀況時進入Fault狀態呼叫notify_fault定義的腳本

切換的過程以下：

1.>在Master主機上keepalived運行時執行mycheck.sh腳本不停的檢查mysql的運行狀態，當發現mysql中止後將keepalived進程殺掉。
2.>此時Slave主機上會接管虛擬IP地址，並調用notify_master定義的腳本
3.>當原Master主機上的mysql和keepalived進程恢復正常後，會調用notify_backup定義的腳本，此時數據庫的主端還在Savle主機上。
4.>回切，關閉Slave端的keepavlied進程，會調用notify_stop腳本，同時Master主機上會調用notify_master定義的腳本。此時數據庫的主端在Master主機上
5.>啓動Slave端的keepavlied進程，會調用notify_backup腳本，此時完成數據同步。

3.Keepalived的配置
在Master端和Savle端均安裝好keepalived後，進行配置，修改/etc/keepalived/keepalived.conf文件.
3.1> Master端配置

[root@localhost keepalived]# cat keepalived.conf
global_defs {
   router_id MySQL-HA
} 

vrrp_script check_run {
script "/home/mysql/mycheck.sh"
interval 10
}

vrrp_sync_group VG1 {
group {
VI_1
}
}

vrrp_instance VI_1 {
    state MASTER
    #state BACKUP
    interface enp0s3 
    virtual_router_id 51
    priority 100
    advert_int 1
    #nopreempt
    authentication {
        auth_type PASS
        auth_pass 1234
    }
    track_script {
    check_run
    }
    notify_master /home/mysql/mymaster.sh
    notify_backup /home/mysql/mybackup.sh
    notify_stop /home/mysql/mystop.sh

    virtual_ipaddress {
        192.168.158.30/24
    }

  }

3.2> Slave端配置

[root@localhost keepalived]# cat keepalived.conf
global_defs {
   router_id MySQL-HA
} 

vrrp_script check_run {
script "/home/mysql/mycheck.sh"
interval 10
}

vrrp_sync_group VG1 {
group {
VI_1
}
}

vrrp_instance VI_1 {
    state MASTER
    #state BACKUP
    interface enp0s3 
    virtual_router_id 51
    priority 90
    advert_int 1
    #nopreempt
    authentication {
        auth_type PASS
        auth_pass 1234
    }
    track_script {
    check_run
    }
    notify_master /home/mysql/mymaster.sh
    notify_backup /home/mysql/mybackup.sh
    notify_stop /home/mysql/mystop.sh

    virtual_ipaddress {
        192.168.158.30/24
    }
  }
[root@localhost keepalived]#

3.3> 從新啓動相關服務

# systemctl restart keepalived

3、切換驗證

1. 保證兩臺主機上面keepalived、MySQL服務都是正常啓動着的.
2. 中止主端
2.1> 將MySQL進程殺死

[root@localhost ~]# systemctl stop  mysqld

2.2> 檢查狀態
主端查看腳本切換日誌

[root@localhost ~]# tail -100f /home/mysql/logs/mysql_switch.log
......
2019-08-27 23:34:34 The scripts mycheck.sh is running ...
2019-08-27 23:34:44 The scripts mycheck.sh is running ...
2019-08-27 23:34:54 The scripts mycheck.sh is running ...
2019-08-27 23:35:04 The scripts mycheck.sh is running ...
2019-08-27 23:35:14 The scripts mycheck.sh is running ...
2019-08-27 23:35:25 The mycheck.sh, mysql is down, after switch...

2.3> 主端查看浮動IP地址的切換過程。

#浮動IP地址原先在Master端，以下：
# 切換後，在從Master端驗查看，浮動IP已被切走到備機
# 在Slave端查看驗證，確認
# 外部ping浮動IP地址效果，有一個丟包

2.4> 主端Keepalived日誌/var/log/messages以下：

Aug 27 23:35:16 localhost systemd: Stopping MySQL Server...
Aug 27 23:35:19 localhost systemd: Stopped MySQL Server.
Aug 27 23:35:24 localhost systemd: Stopping SYSV: Start and stop Keepalived...
Aug 27 23:35:24 localhost Keepalived[10554]: Stopping Keepalived v1.2.13 (08/17,2019)
Aug 27 23:35:24 localhost Keepalived_vrrp[10557]: VRRP_Instance(VI_1) sending 0 priority
Aug 27 23:35:24 localhost Keepalived_vrrp[10557]: VRRP_Instance(VI_1) removing protocol VIPs.
Aug 27 23:35:24 localhost Keepalived_healthcheckers[10556]: Netlink reflector reports IP 192.168.158.30 removed
Aug 27 23:35:24 localhost keepalived: Stopping keepalived: [  OK  ]
Aug 27 23:35:24 localhost systemd: Stopped SYSV: Start and stop Keepalived.
Aug 27 23:35:29 localhost systemd: Started Session 23 of user root.
Aug 27 23:35:29 localhost systemd-logind: New session 23 of user root.
Aug 27 23:35:29 localhost systemd-logind: Removed session 23.

2.5> 備端查看切換日誌/home/mysql/logs/mysql_switch.log

2019-08-27 23:35:29 The scripts mycheck.sh is running ...
2019-08-27 23:35:30 The mymaster.sh, slave sync ok... 
2019-08-27 23:35:32 The mymaster.sh, Sync pos file sucess.
2019-08-27 23:35:39 The scripts mycheck.sh is running ...

2.6> 備端查看/var/log/messages.log日誌

Aug 27 23:35:28 localhost Keepalived_vrrp[23052]: VRRP_Instance(VI_1) Transition to MASTER STATE
Aug 27 23:35:28 localhost Keepalived_vrrp[23052]: VRRP_Group(VG1) Syncing instances to MASTER state
Aug 27 23:35:29 localhost Keepalived_vrrp[23052]: VRRP_Instance(VI_1) Entering MASTER STATE
Aug 27 23:35:29 localhost Keepalived_vrrp[23052]: VRRP_Instance(VI_1) setting protocol VIPs.
Aug 27 23:35:29 localhost Keepalived_healthcheckers[23051]: Netlink reflector reports IP 192.168.158.30 added
Aug 27 23:35:29 localhost Keepalived_vrrp[23052]: VRRP_Instance(VI_1) Sending gratuitous ARPs on enp0s3 for 192.168.158.30
Aug 27 23:35:34 localhost Keepalived_vrrp[23052]: VRRP_Instance(VI_1) Sending gratuitous ARPs on enp0s3 for 192.168.158.30