Mysql - 高可用方案之MHA
1、概述
本文將介紹mysql的MHA(Master High Availability)方案，官方文檔地址：https://github.com/yoshinorim/mha4mysql-manager/wiki/Installation
MHA架構由三臺mysql服務器(一主兩從)和一臺manager節點組成，當主庫發生故障時，manager能自動從衆多從庫中選擇一臺slave log最新的從庫轉變成主庫，而後將其它全部節點從新指向新的主庫。將丟失數據的機率降至最低。html
寫庫故障發生前：
node
寫庫故障發生後：
mysql
2、節點介紹
本次實驗採用4臺虛擬機，操做系統版本Centos6.10，mysql版本5.7.25
manager 10.40.16.60 管理 管理集羣
node1 10.40.16.61 主庫 提供寫服務
node2 10.40.16.62 從庫 提供讀服務
node3 10.40.16.63 從庫 提供讀服務git
還須預留1個vip，如今不用配置，這裏先提一下，後面的安裝步驟用獲得
10.40.16.71 寫vipgithub
3、安裝
1. 配置一主二從
其中node1是主庫，node2和node3是從庫。具體的複製搭建這裏就省略，要是這都不會，那麼該文章對你就沒意思了。順便安利一個本身寫的mysql一鍵安裝腳本https://www.cnblogs.com/ddzj01/p/10678296.html
註明：集羣中使用的複製帳號爲repl，密碼是'123456'redis
node1(10.40.16.61)
(root@localhost)[test1]> show master status\G
*************************** 1. row ***************************
File: mysql-bin.000003
Position: 1067
Binlog_Do_DB:
Binlog_Ignore_DB:
Executed_Gtid_Set: sql
node2(10.40.16.62)
(root@localhost)[test1]> show slave status\G
*************************** 1. row ***************************
Slave_IO_State: Waiting for master to send event
Master_Host: 10.40.16.61
Master_User: repl
Master_Port: 3306
Connect_Retry: 60
Master_Log_File: mysql-bin.000003
Read_Master_Log_Pos: 1067
Relay_Log_File: slave-relay-bin.000002
Relay_Log_Pos: 1233
Relay_Master_Log_File: mysql-bin.000003
Slave_IO_Running: Yes
Slave_SQL_Running: Yes數據庫
node3(10.40.16.63)
(root@localhost)[test1]> show slave status\G
*************************** 1. row ***************************
Slave_IO_State: Waiting for master to send event
Master_Host: 10.40.16.61
Master_User: repl
Master_Port: 3306
Connect_Retry: 60
Master_Log_File: mysql-bin.000003
Read_Master_Log_Pos: 1067
Relay_Log_File: slave-relay-bin.000002
Relay_Log_Pos: 1233
Relay_Master_Log_File: mysql-bin.000003
Slave_IO_Running: Yes
Slave_SQL_Running: Yes安全
2. 下載MHA包
MHA Manager和MHA Node的rpm下載地址
https://github.com/yoshinorim/mha4mysql-manager/wiki/Downloads
服務器
3. 下載擴展包(node1&node2&node3&manager)
wget https://dl.fedoraproject.org/pub/epel/epel-release-latest-6.noarch.rpm
rpm -ivh epel-release-latest-6.noarch.rpm
編輯文件/etc/yum.repos.d/epel.repo
將該文件的第三行取消註釋，第四行添加註釋，以下所示
4. 安裝MHA包
每一個數據庫節點(node1&node2&node3)
yum install -y perl-DBD-MySQL
rpm -ivh mha4mysql-node-0.56-0.el6.noarch.rpm
node包在/usr/bin下面生成4個文件
save_binary_logs：保存和複製master的二進制日誌
apply_diff_relay_logs：識別差別的relay log並將差別的event應用到其它slave中
filter_mysqlbinlog：去除沒必要要的ROLLBACK事件（MHA已再也不使用這個工具）
purge_relay_logs：消除中繼日誌（不會堵塞SQL線程）
管理節點(manager)
yum install -y perl-DBD-MySQL
yum install -y perl-Config-Tiny
yum install -y perl-Log-Dispatch
yum install -y perl-Parallel-ForkManager
rpm -ivh mha4mysql-node-0.56-0.el6.noarch.rpm
rpm -ivh mha4mysql-manager-0.56-0.el6.noarch.rpm
manager包在/usr/bin下面生成9個文件
masterha_check_repl：檢查MySQL的複製情況
masterha_check_ssh：檢查MHA的SSH配置情況
masterha_check_status：檢測當前MHA運行狀態
masterha_conf_host：添加或刪除配置的server信息
masterha_manager：啓動MHA
masterha_master_monitor：檢測master是否宕機
masterha_master_switch：控制故障轉移（自動或手動）
masterha_secondary_check：二次檢查主庫是否真有問題
masterha_stop：關閉MHA
5. 配置互信
在全部節點把下面的命令都執行一遍(node1&node2&node3&manager)
ssh-keygen
ssh-copy-id -i /root/.ssh/id_rsa.pub root@10.40.16.60
ssh-copy-id -i /root/.ssh/id_rsa.pub root@10.40.16.61
ssh-copy-id -i /root/.ssh/id_rsa.pub root@10.40.16.62
ssh-copy-id -i /root/.ssh/id_rsa.pub root@10.40.16.63
4、配置MHA
1. 主節點上(node1)添加集羣監控帳號
(root@localhost)[(none)]> grant all privileges on *.* to 'monitor'@'%' identified by '123456';
2. 每一個從節點上(node2&node3)
set global read_only = 1;
set global relay_log_purge = 0;
之因此關閉從庫的relay_log_purge，是由於 在默認狀況下，從服務器上的中繼日誌會在SQL線程執行完後被自動刪除。可是在MHA環境中，這些中繼日誌在恢復其它從服務器時可能會被用到，所以須要禁用中繼日誌的自動清除。改成按期手動清除SQL線程應用完的中繼日誌。
每一個從節點添加定時任務，最好時間節點錯開
crontab -e
0 4 * * * /usr/bin/purge_relay_logs --user=monitor --password=123456 -disable_relay_log_purge --workdir=/tmp/ >> /tmp/purge_relay_logs.log 2>&1
workdir：爲relay logs建立硬連接的目錄，purge_relay_logs腳本執行的時候會如今該目錄下面建立relay log的硬連接，而後短期開啓relay_log_purge這個參數，當relay log刪除完畢後，再關閉relay_log_purge這個參數，最後刪除硬連接。
3. 全部數據庫節點創建兩個軟鏈接(node1&node2&node3)
mha運行時須要用到/usr/bin下的兩個mysql命令
ln -s /usr/local/mysql/bin/mysqlbinlog /usr/bin/mysqlbinlog
ln -s /usr/local/mysql/bin/mysql /usr/bin/mysql
4. 在manager節點上準備自定義腳本
master_ip_failover：自動切換腳本(當主庫發生故障時)
#!/usr/bin/env perl # Copyright (C) 2011 DeNA Co.,Ltd. # # This program is free software; you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by # the Free Software Foundation; either version 2 of the License, or # (at your option) any later version. # # This program is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU General Public License for more details. # # You should have received a copy of the GNU General Public License # along with this program; if not, write to the Free Software # Foundation, Inc., # 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA ## Note: This is a sample script and is not complete. Modify the script based on your environment. use strict; use warnings FATAL => 'all'; use Getopt::Long; use MHA::DBHelper; my ( $command, $ssh_user, $orig_master_host, $orig_master_ip, $orig_master_port, $new_master_host, $new_master_ip, $new_master_port, $new_master_user, $new_master_password ); my $vip = '10.40.16.71'; my $key = "2"; my $ssh_start_vip = "/sbin/ifconfig eth0:$key $vip/24"; my $ssh_stop_vip = "/sbin/ifconfig eth0:$key down"; my $ssh_send_garp = "/sbin/arping -U $vip -I eth0 -c 1"; GetOptions( 'command=s' => \$command, 'ssh_user=s' => \$ssh_user, 'orig_master_host=s' => \$orig_master_host, 'orig_master_ip=s' => \$orig_master_ip, 'orig_master_port=i' => \$orig_master_port, 'new_master_host=s' => \$new_master_host, 'new_master_ip=s' => \$new_master_ip, 'new_master_port=i' => \$new_master_port, 'new_master_user=s' => \$new_master_user, 'new_master_password=s' => \$new_master_password, ); exit &main(); sub main { if ( $command eq "stop" || $command eq "stopssh" ) { # $orig_master_host, $orig_master_ip, $orig_master_port are passed. # If you manage master ip address at global catalog database, # invalidate orig_master_ip here. my $exit_code = 1; eval { print "Disabling the VIP an old master: $orig_master_host \n"; &stop_vip(); $exit_code = 0; }; if ($@) { warn "Got Error: $@\n"; exit $exit_code; } exit $exit_code; } elsif ( $command eq "start" ) { # all arguments are passed. # If you manage master ip address at global catalog database, # activate new_master_ip here. # You can also grant write access (create user, set read_only=0, etc) here. my $exit_code = 10; eval { my $new_master_handler = new MHA::DBHelper(); # args: hostname, port, user, password, raise_error_or_not $new_master_handler->connect( $new_master_ip, $new_master_port, $new_master_user, $new_master_password, 1 ); ## Set read_only=0 on the new master $new_master_handler->disable_log_bin_local(); print "Set read_only=0 on the new master.\n"; $new_master_handler->disable_read_only(); ## Creating an app user on the new master # print "Creating app user on the new master..\n"; # FIXME_xxx_create_user( $new_master_handler->{dbh} ); $new_master_handler->enable_log_bin_local(); $new_master_handler->disconnect(); print "Enabling the VIP $vip on the new master: $new_master_host \n"; &start_vip(); $exit_code = 0; }; if ($@) { warn $@; # If you want to continue failover, exit 10. exit $exit_code; } exit $exit_code; } elsif ( $command eq "status" ) { # do nothing exit 0; } else { &usage(); exit 1; } } sub start_vip(){ `ssh $ssh_user\@$new_master_host \" $ssh_start_vip \"`; `ssh $ssh_user\@$new_master_host \" $ssh_send_garp \"`; } sub stop_vip(){ return 0 unless ($ssh_user); `ssh $ssh_user\@$orig_master_host \" $ssh_stop_vip \"`; } sub usage { print "Usage: master_ip_failover --command=start|stop|stopssh|status --orig_master_host=host --orig_master_ip=ip --orig_master_port=port --new_master_host=host --new_master_ip=ip --new_master_port=port\n"; }
須要修改的地方，其它都是固定的
master_ip_online_change：手動切換腳本
#!/usr/bin/env perl # Copyright (C) 2011 DeNA Co.,Ltd. # # This program is free software; you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by # the Free Software Foundation; either version 2 of the License, or # (at your option) any later version. # # This program is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU General Public License for more details. # # You should have received a copy of the GNU General Public License # along with this program; if not, write to the Free Software # Foundation, Inc., # 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA ## Note: This is a sample script and is not complete. Modify the script based on your environment. use strict; use warnings FATAL => 'all'; use Getopt::Long; use MHA::DBHelper; use MHA::NodeUtil; use Time::HiRes qw( sleep gettimeofday tv_interval ); use Data::Dumper; my $_tstart; my $_running_interval = 0.1; my $vip = '10.40.16.71'; my $key = "2"; my $ssh_start_vip = "/sbin/ifconfig eth0:$key $vip/24"; my $ssh_stop_vip = "/sbin/ifconfig eth0:$key down"; my $ssh_send_garp = "/sbin/arping -U $vip -I eth0 -c 1"; my ( $command, $orig_master_is_new_slave, $orig_master_host, $orig_master_ip, $orig_master_port, $orig_master_user, $orig_master_password, $orig_master_ssh_user, $new_master_host, $new_master_ip, $new_master_port, $new_master_user, $new_master_password, $new_master_ssh_user, ); GetOptions( 'command=s' => \$command, 'orig_master_is_new_slave' => \$orig_master_is_new_slave, 'orig_master_host=s' => \$orig_master_host, 'orig_master_ip=s' => \$orig_master_ip, 'orig_master_port=i' => \$orig_master_port, 'orig_master_user=s' => \$orig_master_user, 'orig_master_password=s' => \$orig_master_password, 'orig_master_ssh_user=s' => \$orig_master_ssh_user, 'new_master_host=s' => \$new_master_host, 'new_master_ip=s' => \$new_master_ip, 'new_master_port=i' => \$new_master_port, 'new_master_user=s' => \$new_master_user, 'new_master_password=s' => \$new_master_password, 'new_master_ssh_user=s' => \$new_master_ssh_user, ); exit &main(); sub start_vip(){ `ssh $new_master_ssh_user\@$new_master_host \" $ssh_start_vip \"`; `ssh $new_master_ssh_user\@$new_master_host \" $ssh_send_garp \"`; } sub stop_vip(){ `ssh $orig_master_ssh_user\@$orig_master_host \" $ssh_stop_vip \"`; } sub current_time_us { my ( $sec, $microsec ) = gettimeofday(); my $curdate = localtime($sec); return $curdate . "" . sprintf( "%06d", $microsec ); } sub sleep_until { my $elapsed = tv_interval($_tstart); if ( $_running_interval > $elapsed ) { sleep( $_running_interval - $elapsed ); } } sub get_threads_util { my $dbh = shift; my $my_connection_id = shift; my $running_time_threshold = shift; my $type = shift; $running_time_threshold = 0 unless ($running_time_threshold); $type = 0 unless ($type); my @threads; my $sth = $dbh->prepare("SHOW PROCESSLIST"); $sth->execute(); while ( my $ref = $sth->fetchrow_hashref() ) { my $id = $ref->{Id}; my $user = $ref->{User}; my $host = $ref->{Host}; my $command = $ref->{Command}; my $state = $ref->{State}; my $query_time = $ref->{Time}; my $info = $ref->{Info}; $info =~ s/^\s*(.*?)\s*$/$1/ if defined($info); next if ( $my_connection_id == $id ); next if ( defined($query_time) && $query_time < $running_time_threshold ); next if ( defined($command) && $command eq "Binlog Dump" ); next if ( defined($user) && $user eq "system user" ); next if ( defined($command) && $command eq "Sleep" && defined($query_time) && $query_time >= 1 ); if ( $type >= 1 ) { next if ( defined($command) && $command eq "Sleep" ); next if ( defined($command) && $command eq "Connect" ); } if ( $type >= 2 ) { next if ( defined($info) && $info =~ m/^select/i ); next if ( defined($info) && $info =~ m/^show/i ); } push @threads, $ref; } return @threads; } sub main { if ( $command eq "stop" ) { ## Gracefully killing connections on the current master # 1. Set read_only= 1 on the new master # 2. DROP USER so that no app user can establish new connections # 3. Set read_only= 1 on the current master # 4. Kill current queries # * Any database access failure will result in script die. my $exit_code = 1; eval { ## Setting read_only=1 on the new master (to avoid accident) my $new_master_handler = new MHA::DBHelper(); # args: hostname, port, user, password, raise_error(die_on_error)_or_not $new_master_handler->connect( $new_master_ip, $new_master_port, $new_master_user, $new_master_password, 1 ); print current_time_us() . " Set read_only on the new master.. "; $new_master_handler->enable_read_only(); if ( $new_master_handler->is_read_only() ) { print "ok.\n"; } else { die "Failed!\n"; } $new_master_handler->disconnect(); # Connecting to the orig master, die if any database error happens my $orig_master_handler = new MHA::DBHelper(); $orig_master_handler->connect( $orig_master_ip, $orig_master_port, $orig_master_user, $orig_master_password, 1 ); ## Drop application user so that nobody can connect. Disabling per-session binlog beforehand $orig_master_handler->disable_log_bin_local(); # print current_time_us() . " Drpping app user on the orig master..\n"; #drop_app_user($orig_master_handler); ## Waiting for N * 100 milliseconds so that current connections can exit my $time_until_read_only = 15; $_tstart = [gettimeofday]; my @threads = get_threads_util( $orig_master_handler->{dbh}, $orig_master_handler->{connection_id} ); while ( $time_until_read_only > 0 && $#threads >= 0 ) { if ( $time_until_read_only % 5 == 0 ) { printf "%s Waiting all running %d threads are disconnected.. (max %d milliseconds)\n", current_time_us(), $#threads + 1, $time_until_read_only * 100; if ( $#threads < 5 ) { print Data::Dumper->new( [$_] )->Indent(0)->Terse(1)->Dump . "\n" foreach (@threads); } } sleep_until(); $_tstart = [gettimeofday]; $time_until_read_only--; @threads = get_threads_util( $orig_master_handler->{dbh}, $orig_master_handler->{connection_id} ); } ## Setting read_only=1 on the current master so that nobody(except SUPER) can write print current_time_us() . " Set read_only=1 on the orig master.. "; $orig_master_handler->enable_read_only(); if ( $orig_master_handler->is_read_only() ) { print "ok.\n"; } else { die "Failed!\n"; } ## Waiting for M * 100 milliseconds so that current update queries can complete my $time_until_kill_threads = 5; @threads = get_threads_util( $orig_master_handler->{dbh}, $orig_master_handler->{connection_id} ); while ( $time_until_kill_threads > 0 && $#threads >= 0 ) { if ( $time_until_kill_threads % 5 == 0 ) { printf "%s Waiting all running %d queries are disconnected.. (max %d milliseconds)\n", current_time_us(), $#threads + 1, $time_until_kill_threads * 100; if ( $#threads < 5 ) { print Data::Dumper->new( [$_] )->Indent(0)->Terse(1)->Dump . "\n" foreach (@threads); } } sleep_until(); $_tstart = [gettimeofday]; $time_until_kill_threads--; @threads = get_threads_util( $orig_master_handler->{dbh}, $orig_master_handler->{connection_id} ); } ## Terminating all threads print current_time_us() . " Killing all application threads..\n"; $orig_master_handler->kill_threads(@threads) if ( $#threads >= 0 ); print current_time_us() . " done.\n"; $orig_master_handler->enable_log_bin_local(); $orig_master_handler->disconnect(); ## Droping the VIP print "Disabling the VIP an old master: $orig_master_host \n"; &stop_vip(); ## After finishing the script, MHA executes FLUSH TABLES WITH READ LOCK $exit_code = 0; }; if ($@) { warn "Got Error: $@\n"; exit $exit_code; } exit $exit_code; } elsif ( $command eq "start" ) { ## Activating master ip on the new master # 1. Create app user with write privileges # 2. Moving backup script if needed # 3. Register new master's ip to the catalog database # We don't return error even though activating updatable accounts/ip failed so that we don't interrupt slaves' recovery. # If exit code is 0 or 10, MHA does not abort my $exit_code = 10; eval { my $new_master_handler = new MHA::DBHelper(); # args: hostname, port, user, password, raise_error_or_not $new_master_handler->connect( $new_master_ip, $new_master_port, $new_master_user, $new_master_password, 1 ); ## Set read_only=0 on the new master $new_master_handler->disable_log_bin_local(); print current_time_us() . " Set read_only=0 on the new master.\n"; $new_master_handler->disable_read_only(); ## Creating an app user on the new master #print current_time_us() . " Creating app user on the new master..\n"; # create_app_user($new_master_handler); print "Enabling the VIP $vip on the new master: $new_master_host \n"; &start_vip(); $new_master_handler->enable_log_bin_local(); $new_master_handler->disconnect(); ## Update master ip on the catalog database, etc $exit_code = 0; }; if ($@) { warn "Got Error: $@\n"; exit $exit_code; } exit $exit_code; } elsif ( $command eq "status" ) { # do nothing exit 0; } else { &usage(); exit 1; } } sub usage { print "Usage: master_ip_online_change --command=start|stop|status --orig_master_host=host --orig_master_ip=ip --orig_master_port=port --new_master_host=host --new_master_ip=ip --new_master_port=port\n"; die; }
須要修改的地方，其它都是固定的
send_master_failover_mail：當發生故障切換時，可經過該腳本發送告警郵件
#!/usr/bin/perl # Copyright (C) 2011 DeNA Co.,Ltd. # # This program is free software; you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by # the Free Software Foundation; either version 2 of the License, or # (at your option) any later version. # # This program is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU General Public License for more details. # # You should have received a copy of the GNU General Public License # along with this program; if not, write to the Free Software # Foundation, Inc., # 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA ## Note: This is a sample script and is not complete. Modify the script based on your environment. use strict; use warnings FATAL => 'all'; use Mail::Sender; use Getopt::Long; #new_master_host and new_slave_hosts are set only when recovering master succeeded my ( $dead_master_host, $new_master_host, $new_slave_hosts, $subject, $body ); my $smtp='smtp.163.com'; my $mail_from='xxxx@163.com'; my $mail_user='xxxx@163.com'; my $mail_pass='xxxx'; my $mail_to=['xxxx@163.com']; GetOptions( 'orig_master_host=s' => \$dead_master_host, 'new_master_host=s' => \$new_master_host, 'new_slave_hosts=s' => \$new_slave_hosts, 'subject=s' => \$subject, 'body=s' => \$body, ); mailToContacts($smtp,$mail_from,$mail_user,$mail_pass,$mail_to,$subject,$body); sub mailToContacts { my ( $smtp, $mail_from, $user, $passwd, $mail_to, $subject, $msg ) = @_; open my $DEBUG, "> /tmp/monitormail.log" or die "Can't open the debug file:$!\n"; my $sender = new Mail::Sender { ctype => 'text/plain; charset=utf-8', encoding => 'utf-8', smtp => $smtp, from => $mail_from, auth => 'LOGIN', TLS_allowed => '0', authid => $user, authpwd => $passwd, to => $mail_to, subject => $subject, debug => $DEBUG }; $sender->MailMsg( { msg => $msg, debug => $DEBUG } ) or print $Mail::Sender::Error; return 1; } # Do whatever you want here exit 0;
須要修改的地方，其它都是固定的
將上面的文件放入/usr/local/bin/
chmod +x master_ip_failover master_ip_online_change send_master_failover_mail
mkdir -p /etc/masterha/app1
vi /etc/masterha/app1/app1.cnf
[server default] manager_log=/etc/masterha/app1/manager.log //設置manager的日誌 manager_workdir=/etc/masterha/app1 //設置manager的工做目錄 master_binlog_dir=/opt/mydata/log/binlog //設置master默認保存binlog的位置，以便MHA能夠找到master的日誌 master_ip_failover_script=/usr/local/bin/master_ip_failover //設置自動failover時候的切換腳本 master_ip_online_change_script=/usr/local/bin/master_ip_online_change //設置手動切換時候的切換腳本 user=monitor //設置監控用戶 password=123456 //設置監控用戶的密碼 ping_interval=3 //設置監控主庫，發送ping包的時間間隔，默認是3秒，嘗試三次沒有迴應的時候進行自動failover remote_workdir=/tmp //設置master在發生切換時，提取binlog的內容所保存的位置，該目錄是在master上 repl_user=repl //設置複製環境中的複製用戶名 repl_password=123456 //設置複製用戶的密碼 report_script=/usr/local/bin/send_master_failover_mail //設置發生切換後發送的報警的腳本 secondary_check_script=/usr/bin/masterha_secondary_check -s 10.40.16.62 -s 10.40.16.63 --user=root --master_host=10.40.16.61 --master_ip=10.40.16.61 --master_port=3306 //一旦MHA到master的監控之間出現問題，MHA Manager將會判斷其它兩個slave是否能創建到master鏈接，經過多條網絡路由檢測master的可用性 shutdown_script="" //設置故障發生後關閉故障主機腳本（該腳本的主要做用是關閉主機防止發生腦裂） ssh_user=root //設置ssh的登陸用戶名 [server1] hostname=10.40.16.61 port=3306 [server2] hostname=10.40.16.62 port=3306 candidate_master=1 //設置爲候選master，若是設置該參數之後，發生主從切換之後將會將此從庫提高爲主庫，即便這個主庫不是集羣中最新的slave check_repl_delay=0 //默認狀況下若是一個slave落後master 100M的relay logs的話，MHA將不會選擇該slave做爲一個新的master，由於對於這個slave的恢復須要花費很長時間，經過設置check_repl_delay=0，MHA觸發切換在選擇一個新的master的時候將會忽略複製延時，這個參數對於設置了candidate_master=1的主機很是有用，由於它保證了這個候選主在切換過程當中必定是最新的master [server3] hostname=10.40.16.63 port=3306 no_master=1 //設置這個節點永遠不會選爲master
5、MHA檢查
在manager節點上執行下列檢查
檢查SSH的配置
masterha_check_ssh --conf=/etc/masterha/app1/app1.cnf
查看整個集羣的狀態(從庫需啓動slave進程)
masterha_check_repl --conf=/etc/masterha/app1/app1.cnf
檢查MHA Manager的狀態
masterha_check_status --conf=/etc/masterha/app1/app1.cnf
開啓MHA Manager監控
nohup masterha_manager --conf=/etc/masterha/app1/app1.cnf --remove_dead_master_conf --ignore_last_failover >> /etc/masterha/app1/manager.log 2>&1 &
remove_dead_master_conf：該參數表明當發生主從切換後，老的主庫的IP將會從配置文件中移除。
ignore_last_failover：在默認狀況下，MHA發生切換後將會在/etc/masterha/app1下產生app1.failover.complete文件，下次再次切換的時候若是發現該目錄下存在該文件且兩次切換的時間間隔不足8小時的話，將不容許觸發切換。除非在第一次切換後手動rm -rf /etc/masterha/app1/app1.failover.complete。該參數表明忽略上次MHA觸發切換產生的文件。
6、切換
1. 自動failover(主庫down了)
關閉主庫節點
[root@mysqla ~]# service mysql stop
查看manager日誌
[root@monitor ~]# less /etc/masterha/app1/manager.log
查看manager狀態
[root@monitor ~]# masterha_check_status --conf=/etc/masterha/app1/app1.cnf
app1 is stopped(2:NOT_RUNNING).
能夠看到切換完成後manager狀態就變成NOT_RUNNING了
在node3看從庫狀態
能夠看到集羣的主庫目前是node2
在node2看主庫狀態
(root@localhost)[(none)]> show variables like 'read_only';
能夠看到node2的read_only狀態已經由ON變成OFF了
能夠看到vip已經出如今node2上了
[root@mysqlb ~]# ifconfig
vip搭建完成以後並無vip，只有第一次切換以後纔會有，因此因此剛剛配置完mha的時候，若是想用vip，須要在主庫手工建立一個vip
ifconfig eth0:2 10.40.16.71 netmask 255.255.255.0 up
最後看看本身的郵箱，看有沒有收到一封郵件，呵呵:-)
2. 手工在線切換
注意這個實驗是接着上一個實驗來的
手工切換須要關閉MHA監控，若是MHA監控正在運行是不容許手工切換的
masterha_stop --conf=/etc/masterha/app1/app1.cnf
開啓node1的mysql服務
[root@mysqla ~]# service mysql start
查看node2的master log position
(root@localhost)[(none)]> show master status;
在node1開啓同步
(root@localhost)[(none)]> change master to master_host='10.40.16.62',master_port=3306,master_user='repl',master_password='123456',master_log_file='mysql-bin.000002',master_log_pos=1914;
(root@localhost)[(none)]> start slave;
從新編輯app1.cnf，由於在前面failover過程當中，mha已經將server1刪除了，故須要從新添加進來
vi /etc/masterha/app1/app1.cnf
[server default] manager_log=/etc/masterha/app1/manager.log manager_workdir=/etc/masterha/app1 master_binlog_dir=/opt/mydata/log/binlog master_ip_failover_script=/usr/local/bin/master_ip_failover master_ip_online_change_script=/usr/local/bin/master_ip_online_change password=123456 ping_interval=3 remote_workdir=/tmp repl_password=123456 repl_user=repl report_script=/usr/local/bin/send_master_failover_mail secondary_check_script=/usr/bin/masterha_secondary_check -s 10.40.16.61 -s 10.40.16.63 --user=root --master_host=10.40.16.62 --master_ip=10.40.16.62 --master_port=3306 shutdown_script="" ssh_user=root user=monitor [server1] hostname=10.40.16.61 candidate_master=1 check_repl_delay=0 port=3306 [server2] hostname=10.40.16.62 port=3306 [server3] hostname=10.40.16.63 no_master=1 port=3306
masterha_master_switch --conf=/etc/masterha/app1/app1.cnf --master_state=alive --new_master_host=10.40.16.61 --new_master_port=3306 --orig_master_is_new_slave --running_updates_limit=10000
orig_master_is_new_slave是將原master切換爲新主的slave，默認狀況下，是不添加的。
running_updates_limit默認爲1s，即若是主從延遲時間（Seconds_Behind_Master），或master show processlist中dml操做大於1s，則不會執行切換。
會出現兩次提示，須要填yes/no，填yes便可。
查看集羣狀況
(root@localhost)[(none)]> show master status\G
*************************** 1. row ***************************
File: mysql-bin.000004
Position: 154
Binlog_Do_DB:
Binlog_Ignore_DB:
Executed_Gtid_Set:
(root@localhost)[(none)]> show slave status\G
*************************** 1. row ***************************
Slave_IO_State: Waiting for master to send event
Master_Host: 10.40.16.61
Master_User: repl
Master_Port: 3306
Connect_Retry: 60
Master_Log_File: mysql-bin.000004
Read_Master_Log_Pos: 154
(root@localhost)[(none)]> show slave status\G
*************************** 1. row ***************************
Slave_IO_State: Waiting for master to send event
Master_Host: 10.40.16.61
Master_User: repl
Master_Port: 3306
Connect_Retry: 60
Master_Log_File: mysql-bin.000004
Read_Master_Log_Pos: 154
能夠看到手工切換也ok了。
3. 手工failover
還有種特殊狀況是mha監控沒有開，可是主庫掛掉了該怎麼手工failover
[root@monitor ~]# masterha_check_status --conf=/etc/masterha/app1/app1.cnf
app1 is stopped(2:NOT_RUNNING).
在manager節點進行failover切換
masterha_master_switch --conf=/etc/masterha/app1/app1.cnf --master_state=dead --dead_master_host=10.40.16.61 --dead_master_port=3306 --new_master_host=10.40.16.62 --new_master_port=3306 --ignore_last_failover
提示報錯，主庫並無掛掉，因此不能failover
在node1關閉主庫
[root@mysqla ~]# service mysql stop
在manager節點再進行failover切換
masterha_master_switch --conf=/etc/masterha/app1/app1.cnf --master_state=dead --dead_master_host=10.40.16.61 --dead_master_port=3306 --new_master_host=10.40.16.62 --new_master_port=3306 --ignore_last_failover
有兩次提示，都輸入yes
查看manager日誌
[root@monitor ~]# less /etc/masterha/app1/manager.log
這樣就完成了手工failover
7、MHA的優缺點
優勢：
1.能夠基於gtid的複製模式
2.mha在進行故障轉移時更不易產生數據丟失，由於主庫一旦發生故障了，若是主庫還能ping通，MHA會在主庫的/tmp(由參數remote_workdir設置)保存主庫最後的binlog，同時會將該binlog傳到manager節點的/etc/masterha/app1(由參數manager_workdir設置)中進行保存。mha配置mysql的半同步複製更能下降數據丟失的機率。
3.同一個監控節點能夠監控多個集羣
缺點：
1.mha只對主數據庫進行監控，也只有主服務器有vip
2.基於ssh免認證配置，存在必定的安全隱患
3.沒有提供從服務器的負載均衡
4.配置過程比較複雜
8、總結
MHA就介紹到這了，有空能夠再多作作實驗，感覺MHA的魅力。
文章參考：http://www.javashuo.com/article/p-hxxpeazl-m.html
原文出處：https://www.cnblogs.com/ddzj01/p/11550398.html
- 1. MySQL MHA高可用方案
- 2. MySQL高可用架構之MHA MySQL高可用架構之MHA
- 3. Mha-Atlas-MySQL高可用方案實踐。
- 4. MySQL高可用解決方案MHA
- 5. Mha-Atlas-MySQL高可用方案實踐
- 6. MySQL高可用解決方案---MHA
- 7. 基於MHA的MYSQL高可用方案
- 8. mysql MHA高可用方案介紹
- 9. Percona-mysql MHA高可用實戰方案
- 10. MHA + keepalived mysql高可用方案部署
- 更多相關文章...
- • PHP 可用的函數 - PHP參考手冊
- • Swift 可選鏈 - Swift 教程
- • Git可視化極簡易教程 — Git GUI使用方法
- • 常用的分佈式事務解決方案
-
每一个你不满意的现在，都有一个你没有努力的曾经。