ip分佈：
vip 192.168.238.222
| mha_manager |
192.168.238.131 |
監控服務器 |
| master |
192.168.238.128 |
主庫GTID複製模式 |
| slave—1 |
192.168.238.129 |
從庫，備用主庫 |
| slave-2 |
192.168.238.130 |
從庫 |
| mha版本 |
0.56 |
|
二進制mysql，mysql和mysqlbinlog命令軟鏈接到/usr/lcoal/bin
一主兩從模式，採用GTID模式複製（不詳述）
master主要配置以下：
master_info_repository = TABLE
relay_log_info_repository = TABLE
gtid_mode = on
enforce_gtid_consistency = 1
log_slave_updates
binlog_format = row
server-id = 3306
log-bin = /home/mysql/log/binlog/mybinlog
sync-master-info=1
slave-parallel-workers=2
binlog-checksum=CRC32
master-verify-checksum=1
slave-sql-verify-checksum=1
binlog-rows-query-log_events=1
slave 主要配置以下 ，已slave-1爲例，slave2注意修改server-id
master_info_repository = TABLE
relay_log_info_repository = TABLE
gtid_mode = on
enforce_gtid_consistency = 1
log_slave_updates
binlog_format = row
server-id = 3307
log-bin = /home/mysql/log/binlog/mybinlog
sync-master-info=1
slave-parallel-workers=2
binlog-checksum=CRC32
master-verify-checksum=1
slave-sql-verify-checksum=1
binlog-rows-query-log_events=1
#relay log
relay-log = /home/mysql/log/relaylog/relay-bin
relay-log-index = /home/mysql/log/relaylog/relay-bin.index
relay_log_recovery = 1
relay-log-purge = 1
而後再slave上進行change master ：
CHANGE MASTER TO MASTER_HOST='
192.168.238.128', MASTER_PORT=3306, MASTER_AUTO_POSITION=1, MASTER_USER='repl', MASTER_PASSWORD='repl123';
動態設置read_only=1，緣由：切換的時候提高爲主的時候能夠動態關閉不須要去重啓mysql。
主從搭建完成以後，在master受權root能夠容許manager連入。或者建立一個專門的mha帳號也能夠
====================
安裝前，四個節點所有安裝上依賴關係
rpm -ivh http://dl.fedoraproject.org/pub/epel/6/x86_64/epel-release-6-8.noarch.rpm
yum install perl-* -y
四個節點所有編譯安裝node
tar xf mha4mysql-node-0.56.tar.gz
cd mha4mysql-node-0.56
perl Makefile.PL
make && make install
單單在manager節點上安裝manager
tar xf mha4mysql-manager-0.56.tar.gz
cd mha4mysql-manager-0.56
perl Makefile.PL
make && make install
mha_node安裝完成以後會有以下工具包
# ll /usr/loca/bin
-r-xr-xr-x 1 root root 15498 Apr 20 10:05 apply_diff_relay_logs -- 識別差別的中繼日誌事件並將其差別的事件應用於其餘的slave
-r-xr-xr-x 1 root root 4807 Apr 20 10:05 filter_mysqlbinlog -- 過濾mysqlbinlog中的rollback事件（已經不使用）
-r-xr-xr-x 1 root root 7401 Apr 20 10:05 purge_relay_logs -- 清除中繼日誌（不會阻塞SQL線程）
-r-xr-xr-x 1 root root 7263 Apr 20 10:05 save_binary_logs -- 保存和讀取master的binlog
mha_manager安裝完成以後會有以下工具包
# ll /usr/loca/bin
-r-xr-xr-x 1 root root 15498 Apr 20 10:58 apply_diff_relay_logs
-r-xr-xr-x 1 root root 4807 Apr 20 10:58 filter_mysqlbinlog
-r-xr-xr-x 1 root root 1995 Apr 20 11:33 masterha_check_repl -- 檢查複製狀況
-r-xr-xr-x 1 root root 1779 Apr 20 11:33 masterha_check_ssh -- 檢查ssh聯通性
-r-xr-xr-x 1 root root 1865 Apr 20 11:33 masterha_check_status -- 檢查manager狀態
-r-xr-xr-x 1 root root 3201 Apr 20 11:33 masterha_conf_host -- 添加或者刪除配置的server信息
-r-xr-xr-x 1 root root 2517 Apr 20 11:33 masterha_manager -- 啓動mha
-r-xr-xr-x 1 root root 2165 Apr 20 11:33 masterha_master_monitor -- 檢測master是否宕機
-r-xr-xr-x 1 root root 2373 Apr 20 11:33 masterha_master_switch -- 控制故障轉移切換
-r-xr-xr-x 1 root root 3749 Apr 20 11:33 masterha_secondary_check
-r-xr-xr-x 1 root root 1739 Apr 20 11:33 masterha_stop
-r-xr-xr-x 1 root root 7401 Apr 20 10:58 purge_relay_logs
-r-xr-xr-x 1 root root 7263 Apr 20 10:58 save_binary_logs
在manager端配置mha的配置文件cat /mha/app1.cnf
#########################
[server default]
manager_workdir=/mha
manager_log=/mha/manager.log
master_binlog_dir=/home/mysql/log/binlog
password=1234567
user=root
ping_interval=1
remote_workdir=/mha
repl_password=repl123
repl_user=repl
ssh_user=root
client_bindir=/home/mysql/app/bin
master_ip_failover_script=/mha/master_ip_failover
[server1]
hostname=192.168.238.128
port=3306
[server2]
hostname=192.168.238.129
port=3306
master_binlog_dir=/home/mysql/log/binlog
candidate_master=1
[server3]
hostname=192.168.238.130
port=3306
master_binlog_dir=/home/mysql/log/binlog
在相關路徑添加腳本master_ip_failover見附件failover.pl
該腳本功能，進行故障轉移，而且綁定vip
在manager檢測mha_manager配置狀況以及集羣的存活狀況
masterha_check_ssh --conf=/mha/app1.cnf
masterha_check_repl --conf=/mha/app1.cnf
masterha_check_status --conf=/mha/app1.cnf
啓動mha以下
nohup masterha_manager --conf=/mha/app1.cnf --ignore_last_failover < /dev/null >/mha/manager.log 2>&1 &
故障切換流程
停掉主庫的mysql
查看manager.log
當出現master failover to ip_addre completed successfully，則說明故障轉移成功
說明，mha_manager完成故障轉移以後會退出，屬於正常狀況，若是須要持續不斷的運行，就是用daemontool
後續思考：
故障切換完成以後能夠發送郵件到郵件，根據manager是否退出來進行判斷
故障切換後，拉起原主庫做爲備庫&再啓動mha_manager腳本見附件do_mha.sh
腳本運行目錄/mha
[root@mon mha] # sh do_mha.sh app1
該腳本運行原理是，判斷manager進程是否存在。若是存在就不執行腳本，由於沒有發生過故障切換
若是manager不存在就運行腳本，說明發生過故障切換
（排除人工殺掉manager進程的狀況，只基於正常的故障轉移）
附件
failover.pl
#!/usr/bin/env perl
use strict;
use warnings FATAL => 'all';
use Getopt::Long;
use Net::Ping;
use Switch;
my ($command, $ssh_user, $orig_master_host, $orig_master_ip, $orig_master_port, $new_master_host, $new_master_ip, $new_master_port, $new_master_user, $new_master_password);
GetOptions(
'command=s' => \$command,
'ssh_user=s' => \$ssh_user,
'orig_master_host=s' => \$orig_master_host,
'orig_master_ip=s' => \$orig_master_ip,
'orig_master_port=i' => \$orig_master_port,
'new_master_host=s' => \$new_master_host,
'new_master_ip=s' => \$new_master_ip,
'new_master_port=i' => \$new_master_port,
'new_master_user=s' => \$new_master_user,
'new_master_password=s' => \$new_master_password,
);
my $vip = '192.168.238.222'; # Virtual IP
my $master_srv = '192.168.238.129';
my $timeout = 5;
my $key = "1";
my $gateway = '192.168.238.2';
my $interface = 'eth0';
my $ssh_start_vip = "/sbin/ifconfig $interface:$key $vip;/sbin/arping -I $interface -c 3 -s $vip $gateway >/dev/null 2>&1";
my $ssh_stop_vip = "/sbin/ifconfig $interface:$key down";
exit &main();
sub main {
#print "\n\nIN SCRIPT TEST====$ssh_stop_vip==$ssh_start_vip===\n\n";
print "\n\nIN SCRIPT TEST====$ssh_stop_vip==$ssh_start_vip===\n\n";
if ( $command eq "stop" || $command eq "stopssh" ) {
# $orig_master_host, $orig_master_ip, $orig_master_port are passed.
# If you manage master ip address at global catalog database,
# invalidate orig_master_ip here.
my $exit_code = 1;
eval {
print "Disabling the VIP on old master if the server is still UP: $orig_master_host \n";
my $p=Net::Ping->new('icmp');
&stop_vip() if $p->ping($master_srv, $timeout);
$p->close();
$exit_code = 0;
};
if ($@) {
warn "Got Error: $@\n";
exit $exit_code;
}
exit $exit_code;
}
elsif ( $command eq "start" ) {
# all arguments are passed.
# If you manage master ip address at global catalog database,
# activate new_master_ip here.
# You can also grant write access (create user, set read_only=0, etc) here.
my $exit_code = 10;
eval {
print "Enabling the VIP - $vip on the new master - $new_master_host \n";
&start_vip();
$exit_code = 0;
};
if ($@) {
warn $@;
exit $exit_code;
}
exit $exit_code;
}
elsif ( $command eq "status" ) {
print "Checking the Status of the script.. OK \n";
#`ssh $ssh_user\@$new_master_host \" $ssh_start_vip \"`;
`ssh $ssh_user\@$orig_master_host \" $ssh_start_vip \"`;
exit 0;
}
else {
&usage();
exit 1;
}
}
# A simple system call that enable the VIP on the new master
sub start_vip() {
`ssh $ssh_user\@$new_master_host \" $ssh_start_vip \"`;
}
# A simple system call that disable the VIP on the old_master
sub stop_vip() {
`ssh $ssh_user\@$orig_master_host \" $ssh_stop_vip \"`;
}
sub usage {
print
"Usage: master_ip_failover --command=start|stop|stopssh|status --orig_master_host=host --orig_master_ip=ip --orig_master_port=port --new_master_host=host --new_master_ip=ip --new_master_port=port\n";
}node
mha.shmysql
#!/bin/bash
# Script
# Name: do_mha.sh
# APP vendor: MHA
# Linux vendor: RHEL / CentOSlinux
# --------------------------
# variable and file pathsql
# Part:: this script
str_mha_application=$1bash
# Part:: Linux
# Account info
str_linux_username="root"服務器
# Part:: MySQL
# Account info
str_mysql_username="root"
str_mysql_password="123456"app
str_repl_username="repl"
str_repl_password="repl123"ssh
# Part:: MHA
# pid
str_pid_masterha_manager=`ps -ef | grep masterha_manager | grep "$str_mha_application" | grep perl | awk '{print $2}'`函數
# file
file_conf_mha_application="/mha/$str_mha_application.cnf"工具
# file: relation / by computed
file_log_mha_manager=`cat $file_conf_mha_application | grep --color manager_log | cut -d'=' -f2`
# variable: ip info
# 若是MHA中MySQL主庫的候選服務器數量超過了兩臺，也許下面這個list參數，就會排上用場
list_ip_candicate=`cat $file_conf_mha_application | grep -B 3 "^candidate" | grep "hostname" | cut -d'=' -f2`
str_ip_orig_master=`cat $file_log_mha_manager | grep --color "MySQL Master failover" | cut -d'(' -f2 | cut -d':' -f1 | tail -n 1`
str_ip_new_master=`cat $file_log_mha_manager | grep --color "MySQL Master failover" | cut -d'(' -f3 | cut -d':' -f1 | tail -n 1`
str_ip_mha_manager="192.168.238.131"
# 爲[change master]準備的參數
str_log_file_new_master=""
str_log_pos_new_master=""
# Part:: String SQL
str_sql_mysql_change_master=""
# --------------------------
# function
function do_sql() {
# variable
func_str_ip="$1"
func_str_sql="$2"
# action
# 本場景中不涉及到對MySQL某個庫的操做，因此沒有選擇[db]
mysql -u $str_mysql_username -h $func_str_ip -p"$str_mysql_password" -N -e "$func_str_sql" -P3306
}
# 獲取主庫狀態信息 非GTID複製
#function get_info_mysql_master_new_master() {
# version ONE
#str_log_file_new_master=`do_sql "$str_ip_new_master" "show master status" | awk '{print $1}'`
#str_log_pos_new_master=`do_sql "$str_ip_new_master" "show master status" | awk '{print $2}'`
#}
# 生成orig_master做爲slave加入new_master的[change master]SQL命令
function gen_sql_mysql_change_master() {
func_temp_master_host_sed=`cat $file_log_mha_manager | grep --color "All other slaves should start" | tail -n 1 | cut -d',' -f1 | cut -d'=' -f2 | cut -d\' -f2`
func_temp_repl_password_sed=`cat $file_log_mha_manager | grep --color "All other slaves should start" | tail -n 1 | rev | cut -d\' -f2`
echo "======================"
echo "@@ func variable: func_temp_repl_password_sed = $func_temp_repl_password_sed"
echo "======================"
str_sql_mysql_change_master=`cat $file_log_mha_manager | grep --color "All other slaves should start" | tail -n 1 | sed "s/'$func_temp_repl_password_sed'/'$str_repl_password'/g" | cut -d':' -f4`
str_sql_mysql_change_master=`echo $str_sql_mysql_change_master | sed "s/'$func_temp_master_host_sed'/'$str_ip_new_master'/g"`
}
# 對指定主機執行Linux命令
# 前提：
# 1. IP可達
# 2. SSH等價關係
function do_linux_by_ssh() {
# variable
func_str_ip="$1"
func_str_user="$2"
func_str_command="$3"
# action
ssh -T -t $func_str_user@$func_str_ip "$func_str_command"
}
function do_part_orig_master_is_new_slave() {
do_linux_by_ssh "$str_ip_orig_master" "root" "service mysql start"
do_sql "$str_ip_orig_master" "set global read_only=1;"
do_sql "$str_ip_orig_master" "$str_sql_mysql_change_master"
do_sql "$str_ip_orig_master" "start slave;"
}
function do_part_mha_master_manager_start() {
do_linux_by_ssh "$str_ip_mha_manager" "root" "nohup masterha_manager --conf=$file_conf_mha_application --ignore_last_failover &"
}
# 若是PID不存在，則執行該腳本，不然，退出
function runable_by_mha_manager_pid() {
echo "-----------------"
echo "Script for MySQL Master HA"
echo "-----------------"
echo "Begin:: "`date "+|%Y-%m-%d|%H:%M:%S|"`
if [[ "$str_pid_masterha_manager" == "" ]]
then
echo "## masterha_manager is [NOT ALIVED]."
else
echo "## masterha_manager is [ALIVED]."
echo "[masterha_manager] PID is:: $str_pid_masterha_manager"
# do something.
echo "## Exit Script"
exit 0
fi
}
# --------------------------
# action
# 若是PID不存在，則執行該腳本，不然，退出
echo "------------------"
echo "app: runable_by_mha_manager_pid"
runable_by_mha_manager_pid
echo ""
echo "------------------"
echo "app: gen_sql_mysql_change_master"
gen_sql_mysql_change_master
echo ""
echo "------------------"
echo "app: do_part_orig_master_is_new_slave"
do_part_orig_master_is_new_slave
echo ""
echo "------------------"
echo "app: do_part_mha_master_manager_start"
#do_part_mha_master_manager_start 若是腳本在費manager端，則使用該函數，不然就是用以下命令
nohup masterha_manager --conf=$file_conf_mha_application --ignore_last_failover &
echo ""
# --------------------------
# Show time
# ---------
# version one
# ---------
#echo "new master is:: $str_ip_new_master"
#echo "Master log file is:: $str_log_file_new_master"
#echo "Master log POS is:: $str_log_pos_new_master"
#echo "orig master --> new master ## SQL: CHANGE MASTER ## is:: $str_sql_mysql_change_master"
# ---------
# version two
# ---------
echo "================="
echo "MySQL info:"
echo "## Account and Password"
echo "username @ $str_mysql_username"
echo "password @ $str_mysql_password"
echo "--- for REPLICATION ---"
echo "repl @ username ## $str_repl_username"
echo "repl @ password ## $str_repl_password"
echo "## Master Server info"
echo "log file @ Master ## $str_log_file_new_master"
echo "log pos @ Master ## $str_log_pos_new_master"
echo ""
echo "================="
echo "SQL statement:"
echo "[CHANGE MASTER] -->"
echo "$str_sql_mysql_change_master"
echo ""
echo "================="
echo "MasterHA info:"
echo "## File and Path"
echo "MHA Global config file @ $file_conf_mha_global"
echo "MHA Application config file @ $file_conf_mha_application"
echo "MHA Log file:: masterha_manager @ $file_log_mha_manager"
echo "## Architecture"
echo "Candicate Server list::"
echo "$list_ip_candicate"
echo "## IP"
echo "MHA Manager Server:: $str_ip_mha_manager"
echo "Last:: new master:: $str_ip_new_master"
echo "Last:: orig master:: $str_ip_orig_master"
echo ""
# --------------------------echo "-----------------"echo "Finished:: "`date "+|%Y-%m-%d|%H:%M:%S|"`# Done