MySQL高可用MHA

MySQL高可用MHA

環境:node

角色 IP 主機名 Server ID 類型
master1 192.168.1.12 master1 1 主從節點寫
master2 192.168.1.13 master2 2 從節點讀
slave 192.168.1.14 slave 3 從節點讀
manager 192.168.1.7 manager 管理節點

1、環境部署

①更改主機名,添加hosts文件
[root@192 ~]# hostnamectl set-hostname master1
[root@192 ~]# hostnamectl set-hostname master2
[root@192 ~]# hostnamectl set-hostname slave
[root@192 ~]# hostnamectl set-hostname manager
[root@master1 ~]# cat /etc/hostsmysql

127.0.0.1   localhost localhost.localdomain localhost4 localhost4.localdomain4
::1         localhost localhost.localdomain localhost6 localhost6.localdomain6
192.168.1.12 master1
192.168.1.13 master2
192.168.1.14 slave
192.168.1.7 manager

[root@master1 ~]# for i in master2 slave manager ; do scp /etc/hosts $i:/etc/hosts;done
分發給其餘主機redis

②配置密鑰ssh(全部主機各配置相同操做)
例如master1:
[root@master1 ~]# ssh-keygen -t rsa
在全部主機都必須拷貝密鑰於其餘主機
[root@master1 ~]# for i in master1 master2 slave manager ; do ssh-copy-id $i ; done
拷貝密鑰
[root@master1 ~]# for i in master1 master2 slave manager ; do ssh $i hostname ; done
測試登陸sql

③關閉防火牆以及selinxvim

④集羣主機時間同步服務器

⑤安裝環境插件
[root@192 ~]# yum -y install epel-release
[root@192 ~]# yum -y install perl-DBD-MySQL perl-Config-Tiny perl-Log-Dispatch perl-Parallel-ForkManager perl-Config-IniFiles ncftp perl-Params-Validate perl-CPAN perl-Test-Mock-LWP.noarch perl-LWP-Authen-Negotiate.noarch perl-devel perl-ExtUtils-CBuilder perl-ExtUtils-MakeMakerapp

2、配置DB主從

步驟:less

①檢查mysql狀態
mysql半同步是由谷歌提供,master是semisync_master.so,slave是semisync_slave.so,如不清楚路徑能夠先進行查詢dom

mysql> show variables like '%plugin_dir%';ssh

Variable_name Value
plugin_dir /usr/local/mysql/lib/plugin/

再進行檢查是否支持動態載入
mysql> show variables like '%have_dynamic_loading%';

Variable_name Value
have_dynamic_loading YES

②安裝mysql半同步插件(在全部的mysql主機)
mysql> install plugin rpl_semi_sync_master soname 'semisync_master.so';
mysql> install plugin rpl_semi_sync_slave soname 'semisync_slave.so';
檢查是否正確安裝
mysql> show plugins;

rpl_semi_sync_master ACTIVE REPLICATION semisync_master.so GPL
rpl_semi_sync_slave ACTIVE REPLICATION semisync_slave.so GPL

查看半同步相關信息
mysql> show variables like 'rpl_semi_sync%';

Variable_name Value
rpl_semi_sync_master_enabled OFF
rpl_semi_sync_master_timeout 10000
rpl_semi_sync_master_trace_level 32
rpl_semi_sync_master_wait_for_slave_count 1
rpl_semi_sync_master_wait_no_slave ON
rpl_semi_sync_master_wait_point AFTER_SYNC
rpl_semi_sync_slave_enabled OFF
rpl_semi_sync_slave_trace_level 32

③配置mysql半同步複製
master1:
[root@master1 ~]# cat /etc/my.cnf

[mysqld]
basedir = /usr/local/mysql
datadir = /usr/local/mysql/data
port = 3306
server_id = 1
socket = /usr/local/mysql/mysql.sock
log-error=/usr/local/mysql/data/mysqld.err
log-bin = mysql-bin
log-bin-index = mysql-bin.index
rpl_semi_sync_master_enabled = 1
rpl_semi_sync_master_timeout = 1000
rpl_semi_sync_slave_enabled = 1
relay_log_purge = 0
relay-log = relay-bin
relay-log-index = slave-relay-bin.index
[client]
host = 127.0.0.1
user = root
password = 123.com

master2:
[root@master2 ~]# cat /etc/my.cnf

[mysqld]
basedir = /usr/local/mysql
datadir = /usr/local/mysql/data
port = 3306
server_id = 2
socket = /usr/local/mysql/mysql.sock
log-error=/usr/local/mysql/data/mysqld.err
log-bin = mysql-bin
log-bin-index = mysql-bin.index
rpl_semi_sync_master_enabled = 1
rpl_semi_sync_master_timeout = 1000
rpl_semi_sync_slave_enabled = 1
relay_log_purge = 0
relay-log = relay-bin
relay-log-index = slave-relay-bin.index
[client]
host = 127.0.0.1
user = root
password = 123.com

slave:
[root@slave ~]# cat /etc/my.cnf

[mysqld]
basedir = /usr/local/mysql
datadir = /usr/local/mysql/data
port = 3306
server_id = 3
socket = /usr/local/mysql/mysql.sock
log-error=/usr/local/mysql/data/mysqld.err
log-bin = mysql-bin
log-bin-index = mysql-bin.index
rpl_semi_sync_slave_enabled = 1
relay-log = relay-bin
relay-log-index = slave-relay-bin.index
read_only = 1
[client]
host = 127.0.0.1
user = root
password = 123.com

所有重啓服務
[root@master1 ~]# systemctl restart mysqld

mysql> show variables like '%rpl_semi_sync%';

Variable_name Value
rpl_semi_sync_master_enabled ON
rpl_semi_sync_master_timeout 1000
rpl_semi_sync_master_trace_level 32
rpl_semi_sync_master_wait_for_slave_count 1
rpl_semi_sync_master_wait_no_slave ON
rpl_semi_sync_master_wait_point AFTER_SYNC
rpl_semi_sync_slave_enabled ON
rpl_semi_sync_slave_trace_level 32

這是主從master的半同步狀態
mysql> show variables like '%rpl_semi_sync%';

Variable_name Value
rpl_semi_sync_master_enabled OFF
rpl_semi_sync_master_timeout 10000
rpl_semi_sync_master_trace_level 32
rpl_semi_sync_master_wait_for_slave_count 1
rpl_semi_sync_master_wait_no_slave ON
rpl_semi_sync_master_wait_point AFTER_SYNC
rpl_semi_sync_slave_enabled ON
rpl_semi_sync_slave_trace_level 32

這是slave的半同步狀態,發現主插件是關閉的

④權限與change
master1:

mysql> grant replication slave on *.* to rep@'%' identified by '123.com';
mysql> show master status;

+------------------+----------+--------------+------------------+-------------------+
| File             | Position | Binlog_Do_DB | Binlog_Ignore_DB | Executed_Gtid_Set |
+------------------+----------+--------------+------------------+-------------------+
| mysql-bin.000001 |      436 |              |                  |                   |
+------------------+----------+--------------+------------------+-------------------+

master2:

mysql> grant replication slave on *.* to rep@'%' identified by '123.com';
mysql> change master to master_host='192.168.1.12',master_user='rep',master_password='123.com',master_log_file='mysql-bin.000001',master_log_pos=436;
mysql> start slave;

slave:

mysql> change master to master_host='192.168.1.12',master_user='rep',master_password='123.com',master_log_file='mysql-bin.000001',master_log_pos=436;
mysql> start slave;

回到master1:

mysql> grant all privileges on *.* to manager@'%' identified by '123.com';
mysql> show status like '%rpl_semi_sync%';
+--------------------------------------------+-------+
| Variable_name                              | Value |
+--------------------------------------------+-------+
| Rpl_semi_sync_master_clients               | 2     |
| Rpl_semi_sync_master_net_avg_wait_time     | 0     |
+--------------------------------------------+-------+

由於已經配置主從複製,在master1上建立manager用戶同步到其餘主機受權manager監控節點工做

3、配置MHA

步驟:

①安裝MHA(manager節點兩個包都須要安裝,三臺DB只須要安裝node包)
例如master1:
[root@master1 ~]# tar zxf mha4mysql-node-0.56.tar.gz
[root@master1 ~]# cd mha4mysql-node-0.56/
[root@master1 mha4mysql-node-0.56]# perl Makefile.PL
[root@master1 mha4mysql-node-0.56]# make && make install

manager:
[root@manager ~]# tar zxf mha4mysql-node-0.56.tar.gz
[root@manager ~]# cd mha4mysql-node-0.56/
[root@manager mha4mysql-node-0.56]# perl Makefile.PL
[root@manager mha4mysql-node-0.56]# make && make install

[root@manager mha4mysql-node-0.56]# cd ..
[root@manager ~]# tar zxf mha4mysql-manager-0.56.tar.gz
[root@manager ~]# cd mha4mysql-manager-0.56/
[root@manager mha4mysql-manager-0.56]# perl Makefile.PL
[root@manager mha4mysql-manager-0.56]# make && make install

建立一些可用目錄
[root@manager ~]# cd /
[root@manager /]# mkdir etc/masterha
[root@manager /]# mkdir -p masterha/app1
[root@manager /]# mkdir scripts
[root@manager /]# cd ~/mha4mysql-manager-0.56/
[root@manager mha4mysql-manager-0.56]# cp samples/conf/* /etc/masterha/
[root@manager mha4mysql-manager-0.56]# cp samples/scripts/* /scripts/
[root@manager mha4mysql-manager-0.56]#

②配置MHA
[root@manager ~]# vim /etc/masterha/app1.cnf

[server default]
manager_workdir=//masterha/app1
manager_log=/masterha/app1/manager.log
user=manager
password=123.com
ssh_user=root
repl_user=rep
repl_password=123.com
ping_interval=1

[server1]
hostname=192.168.1.12
port=3306
master_binlog_dir=/usr/local/mysql/data
candidate_master=1

[server2]
hostname=192.168.1.13
port=3306
master_binlog_dir=/usr/local/mysql/data
candidate_master=1
         
[server3] 
hostname=192.168.1.14
port=3306
master_binlog_dir=/usr/local/mysql/data
no_master=1

③ssh有效性驗證
[root@manager ~]# masterha_check_ssh --global_conf=/etc/masterha/masterha_default.cnf --conf=/etc/masterha/app1.cnf

.......
Tue Apr 24 16:46:58 2018 - [debug]   ok.
Tue Apr 24 16:46:58 2018 - [debug]  Connecting via SSH from root@192.168.1.14(192.168.1.14:22) to root@192.168.1.13(192.168.1.13:22)..
Tue Apr 24 16:46:58 2018 - [debug]   ok.
Tue Apr 24 16:46:58 2018 - [info] All SSH connection tests passed successfully.

若有報錯,務必檢查無密ssh鏈接狀況,確保其餘主機已經分享公鑰

④集羣複製有效性驗證
[root@manager ~]# masterha_check_ssh --global_conf=/etc/masterha/masterha_default.cnf --conf=/etc/masterha/app1.cnf

..........
Tue Apr 24 16:50:24 2018 - [info]   Connecting to root@192.168.1.13(192.168.1.13:22).. 
Can't exec "mysqlbinlog": No such file or directory at /usr/local/share/perl5/MHA/BinlogManager.pm line 106.
mysqlbinlog version command failed with rc 1:0, please verify PATH, LD_LIBRARY_PATH, and client options
 at /usr/local/bin/apply_diff_relay_logs line 492.
Tue Apr 24 16:50:25 2018 - [error][/usr/local/share/perl5/MHA/MasterMonitor.pm, ln201] Slaves settings check failed!
Tue Apr 24 16:50:25 2018 - [error][/usr/local/share/perl5/MHA/MasterMonitor.pm, ln390] Slave configuration failed.
Tue Apr 24 16:50:25 2018 - [error][/usr/local/share/perl5/MHA/MasterMonitor.pm, ln401] Error happend on checking configurations.  at /usr/local/bin/masterha_check_repl line 48.
Tue Apr 24 16:50:25 2018 - [error][/usr/local/share/perl5/MHA/MasterMonitor.pm, ln500] Error happened on monitoring servers.
Tue Apr 24 16:50:25 2018 - [info] Got exit code 1 (Not master dead).

MySQL Replication Health is NOT OK!

發現報錯Can't exec "mysqlbinlog": No such file or directory at /usr/local/share/perl5/MHA/BinlogManager.pm line 106.

解決方法在全部DB服務器進行軟鏈接執行文件操做
[root@master1 ~]# ln -s /usr/local/mysql/bin/* /usr/local/bin

⑤啓動manager
[root@manager ~]# nohup masterha_manager --conf=/etc/masterha/app1.cnf &> /tmp/mha_manager.log &
[1] 8126
[root@manager ~]# masterha_check_status --conf=/etc/masterha/app1.cnf

app1 (pid:8126) is running(0:PING_OK), master:192.168.1.12

4、MHA高可用驗證

步驟:

①將master1中mysql服務宕掉
[root@master1 ~]# systemctl stop mysqld

②在manager查看MHA日誌
[root@manager ~]# cat /masterha/app1/manager.log

.......
Started automated(non-interactive) failover.
The latest slave 192.168.1.13(192.168.1.13:3306) has all relay logs for recovery.
Selected 192.168.1.13 as a new master.
192.168.1.13: OK: Applying all logs succeeded.
192.168.1.14: This host has the latest relay log events.
Generating relay diff files from the latest slave succeeded.
192.168.1.14: OK: Applying all logs succeeded. Slave started, replicating from 192.168.1.13.
192.168.1.13: Resetting slave info succeeded.
Master failover to 192.168.1.13(192.168.1.13:3306) completed successfully

發現master failover已經成功

③查看slave複製狀態
mysql> show slave status\G

*************************** 1. row ***************************
               Slave_IO_State: Waiting for master to send event
                  Master_Host: 192.168.1.13
                  Master_User: rep
                  Master_Port: 3306
                Connect_Retry: 60
              Master_Log_File: mysql-bin.000001
          Read_Master_Log_Pos: 436
......

發現master IP已經轉爲192.168.1.13,io線程與sql線程正常運行,MHA搭建成功

④MHA manager端平常操做
1)發生主從切換後,manager服務會自動停掉,卻在manager_workdir(/masterha/app1)目錄生成文件app1.failover.complate,若要啓動MHA,必須確保此文件已經刪除
[root@manager ~]# cd /masterha/
[root@manager masterha]# ls
app1
[root@manager masterha]# cd app1/
[root@manager app1]# ls
app1.failover.complete manager.log
[root@manager app1]# mv app1.failover.complete /tmp

2)從新定義master1位置,須要把master1設置爲master2的slave服務器
mysql> change master to master_host='192.168.1.13',master_user='rep',master_password='123.com',master_log_file='mysql-bin.000001',master_log_pos=436;
mysql> start slave;

3)中止MHA
[root@manager app1]# masterha_stop --conf=/etc/masterha/app1.cnf

4)啓動MHA
[root@manager app1]# nohup masterha_manager --conf=/etc/masterha/app1.cnf &> /tmp/mha_manager.log &
[1] 8613
[root@manager app1]# masterha_check_status --conf=/etc/masterha/app1.cnf
app1 (pid:8613) is running(0:PING_OK), master:192.168.1.13

5)主從切換後續工做
重構計劃,切換後的master2成爲主庫,想要master1重奪主庫,從新執行以上五步,包括中止master2庫。原主庫數據文件完整狀況能夠經過如下方式獲得change master命令
[root@manager app1]# grep -i 'change master to master' /masterha/app1/manager.log | tail -1

Tue Apr 24 17:16:59 2018 - [info]  All other slaves should start replication from here. Statement should be: CHANGE MASTER TO MASTER_HOST='192.168.1.12', MASTER_PORT=3306, MASTER_LOG_FILE='mysql-bin.000002', MASTER_LOG_POS=154, MASTER_USER='rep', MASTER_PASSWORD='xxx';

mysql> CHANGE MASTER TO MASTER_HOST='192.168.1.12', MASTER_PORT=3306, MASTER_LOG_FILE='mysql-bin.000002', MASTER_LOG_POS=154, MASTER_USER='rep', MASTER_PASSWORD='123.com';
mysql> start slave;
mysql> show slave status \G

*************************** 1. row ***************************
               Slave_IO_State: Waiting for master to send event
                  Master_Host: 192.168.1.12
                  Master_User: rep
                  Master_Port: 3306
                Connect_Retry: 60
              Master_Log_File: mysql-bin.000002
          Read_Master_Log_Pos: 154

[root@manager ~]# cd /masterha/app1/
[root@manager app1]# ls
app1.failover.complete manager.log
[root@manager app1]# mv app1.failover.complete /tmp
mv: overwrite ‘/tmp/app1.failover.complete’? y

[root@manager app1]# nohup masterha_manager --conf=/etc/masterha/app1.cnf &> /tmp/mha_manager.log &
[1] 9015
[root@manager app1]# masterha_check_status --conf=/etc/masterha/app1.cnf
app1 (pid:9015) is running(0:PING_OK), master:192.168.1.12

6)按期刪除中繼日誌
在配置主從複製中,slave設置了參數relay_log_purge=0,因此slave節點須要按期刪除中繼日誌,建議每一個slave節點刪除中繼日誌時間錯開
[root@slave ~]# crontab -e

0 5 * * * /usr/local/bin/purge_relay_logs --user=root --password=123.com --port=3306 --disable_relay_log_purge >> /var/log/purge_
relay.log 2>&

5、配置VIP

vip配置能夠採用兩種方式,一種經過keepalived方式管理虛擬ip浮動;另外一種經過腳本啓動虛擬ip方式(就是不須要任何第三方應用程序)

1.keepalived方式管理虛擬ip

步驟:

①在兩臺master安裝keepalived
[root@master1 ~]# wget http://www.keepalived.org/sof...
[root@master1 ~]# yum -y install openssl openssl-devel
[root@master1 ~]# cd keepalived-1.4.3/
[root@master1 keepalived-1.4.3]# ./configure --prefix=/ && make && make install
[root@master1 keepalived-1.4.3]# whereis keepalived
keepalived: /usr/sbin/keepalived /etc/keepalived

檢查cent7啓動腳本中執行程序位置
[root@master1 ~]# vim /usr/lib/systemd/system/keepalived.service

[Unit]
Description=LVS and VRRP High Availability Monitor
After= network-online.target syslog.target
Wants=network-online.target

[Service]
Type=forking
PIDFile=/var/run/keepalived.pid
KillMode=process
EnvironmentFile=-//etc/sysconfig/keepalived
ExecStart=/usr/sbin/keepalived $KEEPALIVED_OPTIONS
ExecReload=/bin/kill -HUP $MAINPID

[Install]
WantedBy=multi-user.target

②配置keepalived
master1:
[root@master1 ~]# cat /etc/keepalived/keepalived.conf

! Configuration File for keepalived

global_defs {
   router_id mysql-ha1
}

vrrp_instance VI_1 {
    state BACKUP
    interface eno16777736
    virtual_router_id 51
    priority 100
    advert_int 1
    authentication {
        auth_type PASS
        auth_pass 1111
    }
    virtual_ipaddress {
        192.168.1.100
    }
}

master2:
[root@master2 ~]# cat /etc/keepalived/keepalived.conf

! Configuration File for keepalived

global_defs {
   router_id mysql-ha2
}

vrrp_instance VI_1 {
    state BACKUP
    interface eno16777736
    virtual_router_id 51
    priority 90
    advert_int 1
    authentication {
        auth_type PASS
        auth_pass 1111
    }
    virtual_ipaddress {
        192.168.1.100
    }
}

③啓動keepalived
[root@master1 ~]# systemctl start keepalived
[root@master2 ~]# systemctl start keepalived
查看master1虛擬ip狀態
[root@master1 ~]# ip a

......
2: eno16777736: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen 1000
    link/ether 00:0c:29:db:f7:b8 brd ff:ff:ff:ff:ff:ff
    inet 192.168.1.12/24 brd 192.168.1.255 scope global dynamic eno16777736
       valid_lft 77805sec preferred_lft 77805sec
    inet 192.168.1.100/32 scope global eno16777736
       valid_lft forever preferred_lft forever
    inet6 fe80::20c:29ff:fedb:f7b8/64 scope link 
       valid_lft forever preferred_lft forever

④MHA引入keepalived
mysql服務進程掛掉後經過MHA中止keepalived,想要引入MHA,咱們只須要修改觸發腳本文件,添加master發生宕機後對keepalived的處理
[root@manager /]# vim /scripts/master_ip_failover

#!/usr/bin/env perl

#  Copyright (C) 2011 DeNA Co.,Ltd.
#
#  This program is free software; you can redistribute it and/or modify
#  it under the terms of the GNU General Public License as published by
#  the Free Software Foundation; either version 2 of the License, or
#  (at your option) any later version.
#
#  This program is distributed in the hope that it will be useful,
#  but WITHOUT ANY WARRANTY; without even the implied warranty of
#  MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
#  GNU General Public License for more details.
#
#  You should have received a copy of the GNU General Public License
#   along with this program; if not, write to the Free Software
#  Foundation, Inc.,
#  51 Franklin Street, Fifth Floor, Boston, MA  02110-1301  USA

## Note: This is a sample script and is not complete. Modify the script based on your environment.

use strict;
use warnings FATAL => 'all';

use Getopt::Long;
#use MHA::DBHelper;

my (
  $command,        $ssh_user,         $orig_master_host,
  $orig_master_ip, $orig_master_port, $new_master_host,
  $new_master_ip,  $new_master_port,  $new_master_user,
  $new_master_password
);

my $vip = '192.168.1.100';
my $ssh_start_vip = "/usr/bin/systemctl start keepalived";
my $ssh_stop_vip = "/usr/bin/systemctl stop keepalived";

GetOptions(
  'command=s'             => \$command,
  'ssh_user=s'            => \$ssh_user,
  'orig_master_host=s'    => \$orig_master_host,
  'orig_master_ip=s'      => \$orig_master_ip,
  'orig_master_port=i'    => \$orig_master_port,
  'new_master_host=s'     => \$new_master_host,
  'new_master_ip=s'       => \$new_master_ip,
  'new_master_port=i'     => \$new_master_port,
  'new_master_user=s'     => \$new_master_user,
  'new_master_password=s' => \$new_master_password,
);

exit &main();

sub main {

  print"\n\nIN SCRIPT TEST====$ssh_stop_vip==$ssh_start_vip===\n\n";

  if ( $command eq "stop" || $command eq "stopssh" ) {

    # $orig_master_host, $orig_master_ip, $orig_master_port are passed.
    # If you manage master ip address at global catalog database,
    # invalidate orig_master_ip here.
    my $exit_code = 1;
    eval {
      print "Disabling the VIP on old master:$orig_master_host \n";
      &stop_vip();
      # updating global catalog, etc
      $exit_code = 0;
    };
    if ($@) {
      warn "Got Error: $@\n";
      exit $exit_code;
    }
    exit $exit_code;
  }
  elsif ( $command eq "start" ) {

    # all arguments are passed.
    # If you manage master ip address at global catalog database,
    # activate new_master_ip here.
    # You can also grant write access (create user, set read_only=0, etc) here.
    my $exit_code = 10;
    eval {
     # my $new_master_handler = new MHA::DBHelper();

      # args: hostname, port, user, password, raise_error_or_not
      #$new_master_handler->connect( $new_master_ip, $new_master_port,
       # $new_master_user, $new_master_password, 1 );

      ## Set read_only=0 on the new master
      #$new_master_handler->disable_log_bin_local();
      #print "Set read_only=0 on the new master.\n";
      #$new_master_handler->disable_read_only();

      ## Creating an app user on the new master
      #print "Creating app user on the new master..\n";
      #FIXME_xxx_create_user( $new_master_handler->{dbh} );
      #$new_master_handler->enable_log_bin_local();
      #$new_master_handler->disconnect();

      ## Update master ip on the catalog database, etc
      #FIXME_xxx;
      print"Enabling the VIP - $vip on the new master - $new_master_host \n";
      &start_vip();
      $exit_code = 0;
    };
    if ($@) {
      warn $@;

      # If you want to continue failover, exit 10.
      exit $exit_code;
    }
    exit $exit_code;
  }
  elsif ( $command eq "status" ) {
    print "Checking the status of the script..OK \n";
    #`ssh $ssh_user\@cluster1\"$ssh_start_vip\"`;

    # do nothing
    exit 0;
  }
  else {
    &usage();
    exit 1;
  }
}

sub start_vip(){
    `ssh $ssh_user\@$new_master_host\"$ssh_start_vip\"`;
}

sub stop_vip(){
    return 0 unless ($ssh_user);
    `ssh $ssh_user\@$orig_master_host\"$ssh_stop_vip\"`;
}

sub usage {
  print
"Usage: master_ip_failover --command=start|stop|stopssh|status --orig_master_host=host --orig_master_ip=ip --orig_master_port=port --new_master_host=host --new_master_ip=ip --new_master_port=port\n";
}

⑤中止MHA
[root@manager ~]# masterha_stop --conf=/etc/masterha/app1.cnf

⑥添加參數
[root@manager ~]# vim /etc/masterha/app1.cnf

[server default]
master_ip_failover_script=/scripts/master_ip_failover
manager_workdir=//masterha/app1
manager_log=/masterha/app1/manager.log
........

⑦啓動MHA,而且查看狀態
[root@manager ~]# nohup masterha_manager --conf=/etc/masterha/app1.cnf &> /tmp/mha_manager.log &
[1] 10520
[root@manager ~]# masterha_check_status --conf=/etc/masterha/app1.cnf
app1 (pid:10520) is running(0:PING_OK), master:192.168.1.12

再次查看集羣狀態是否會報錯
[root@manager ~]# masterha_check_repl --conf=/etc/masterha/app1.cnf

.......
Checking the status of the script..OK 
Tue Apr 24 18:39:12 2018 - [info]  OK.
Tue Apr 24 18:39:12 2018 - [warning] shutdown_script is not defined.
Tue Apr 24 18:39:12 2018 - [info] Got exit code 0 (Not master dead).

MySQL Replication Health is OK.

能夠看到沒有報錯
master_ip_failover內容意思當主庫放生故障,會觸發MHA切換,manager會停掉主庫上的keepalived服務,觸發vip飄逸到從庫

⑧測試
宕掉master1中mysql服務
[root@master1 ~]# systemctl stop mysqld

查看master1中網卡綁定狀況
[root@master1 init.d]# ip a

.......
2: eno16777736: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen 1000
    link/ether 00:0c:29:db:f7:b8 brd ff:ff:ff:ff:ff:ff
    inet 192.168.1.12/24 brd 192.168.1.255 scope global dynamic eno16777736
       valid_lft 74322sec preferred_lft 74322sec
    inet 192.168.1.100/32 scope global eno16777736
       valid_lft forever preferred_lft forever
    inet6 fe80::20c:29ff:fedb:f7b8/64 scope link 
       valid_lft forever preferred_lft forever

發現vip地址並沒飄逸

回到manager查看manager.log文件,發現並無報錯,排查到爲master_ip_failover腳本並有殺死keepalived進程
解決方法將兩臺主機中拷貝keepalived腳本執行文件,在keepalived的tar包解壓後keepalived/etc/init.d/keepalived拷貝到/etc/init.d/下,修改master_ip_failover腳本文件參數項

......
my $ssh_start_vip = "/etc/init.d/keepalived start";
my $ssh_stop_vip = "/etc/init.d/keepalived stop";
......

刪除app1.failover.complete,從新定義master,啓動mha,再一次宕掉master1服務進行驗證
[root@master1 ~]# ps -ef | grep keep
root 26079 3646 0 19:47 pts/1 00:00:00 grep --color=auto keep
發現keepalived已經被關閉

再去觀察master2的vip地址
[root@master2 ~]# ip a

......
2: eno16777736: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen 1000
    link/ether 00:0c:29:4b:6a:1e brd ff:ff:ff:ff:ff:ff
    inet 192.168.1.13/24 brd 192.168.1.255 scope global dynamic eno16777736
       valid_lft 70169sec preferred_lft 70169sec
    inet 192.168.1.100/32 scope global eno16777736
       valid_lft forever preferred_lft forever
    inet6 fe80::20c:29ff:fe4b:6a1e/64 scope link 
       valid_lft forever preferred_lft forever

vip已經轉移驗證成功,注意後續重構mysql服務器

2.經過腳本實現虛擬IP切換

步驟:

①手動綁定vip
[root@master1 ~]# ifconfig eno16777736:0 192.168.1.100/24
[root@master1 ~]# ifconfig

eno16777736: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500
        inet 192.168.1.12  netmask 255.255.255.0  broadcast 192.168.1.255
        inet6 fe80::20c:29ff:fedb:f7b8  prefixlen 64  scopeid 0x20<link>
        ether 00:0c:29:db:f7:b8  txqueuelen 1000  (Ethernet)
        RX packets 31589  bytes 8780116 (8.3 MiB)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 25911  bytes 5726902 (5.4 MiB)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

eno16777736:0: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500
        inet 192.168.1.100  netmask 255.255.255.0  broadcast 192.168.1.255
        ether 00:0c:29:db:f7:b8  txqueuelen 1000  (Ethernet)

②修改app1.cnf文件和master_ip_failover文件
在[server default]下添加master_ip_failover_script=/scripts/master_ip_failover

在之上keepalived的master_ip_failover腳本中修改並添加四項參數

......
my $vip = '192.168.1.100';
my $key = '0';
my $ssh_start_vip = "/usr/sbin/ifconfig eno16777736:$key $vip";
my $ssh_stop_vip = "/usr/sbin/ifconfig eno16777736:$key down";
......

③從新啓動MHA
[root@manager ~]# masterha_stop --conf=/etc/masterha/app1.cnf
[root@manager ~]# nohup masterha_manager --conf=/etc/masterha/app1.cnf &> /tmp/mha_manager.log &

④測試
宕掉master1
[root@master1 ~]# systemctl stop mysqld

查看slave複製狀態
mysql> show slave status\G

*************************** 1. row ***************************
               Slave_IO_State: Waiting for master to send event
                  Master_Host: 192.168.1.13
                  Master_User: rep
                  Master_Port: 3306
                Connect_Retry: 60
              Master_Log_File: mysql-bin.000005
          Read_Master_Log_Pos: 154

查看master2 vip狀態
[root@master2 ~]# ifconfig

eno16777736: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500
        inet 192.168.1.13  netmask 255.255.255.0  broadcast 192.168.1.255
        inet6 fe80::20c:29ff:fedb:f7b8  prefixlen 64  scopeid 0x20<link>
        ether 00:0c:29:db:f7:b8  txqueuelen 1000  (Ethernet)
        RX packets 32959  bytes 8939121 (8.5 MiB)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 26418  bytes 5872252 (5.6 MiB)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

eno16777736:0: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500
        inet 192.168.1.100  netmask 255.255.255.0  broadcast 192.168.1.255
        ether 00:0c:29:db:f7:b8  txqueuelen 1000  (Ethernet)

vip已經被master2接管,驗證成功

6、總結

MHA軟件由兩部分組成,manager工具包和node工具包

Manager工具包工具:

masterha_check_ssh            檢查MHA的SSH配置情況
masterha_check_repl            檢查MySQL複製情況
masterha_manager                啓動MHA
masterha_master_monitor        檢測master是否宕機
masterha_master_switch        控制故障轉移
masterha_conf_host            添加或刪除配置的server信息

Node工具包工具:

save_binary_logs                保存和複製master的二進制日誌
apply_diff_relay_logs            識別差別的中繼日誌事件並將其差別的事件應用於其餘的slave
filter_mysqlbinlog                去除沒必要要的ROLLBACK事件
purge_relay_logs                清除中繼日誌
相關文章
相關標籤/搜索