keepalived是vrrp協議的實現，原生設計目的是爲了高可用ipvs服務，keepalived可以配置文件中的定義生成ipvs規則，並可以對各RS的健康狀態進行檢測；經過共用的虛擬IP地址對外提供服務；每一個熱備組內同一時刻只有一臺主服務器提供服務，其餘服務器處於冗餘狀態，若當前在線的服務器宕機，其虛擬IP地址將會被其餘服務器接替（優先級決定接替順序），實現高可用爲後端主機提供服務。mysql
2、keepalived組件
Keepalived組件介紹linux
core:keepalived核心組件，主進程的啓動和維護，全局配置等。sql
vrrp stack:keepalived是基於vrrp協議實現高可用vps服務，vrrp則爲相關子進程爲其提供服務數據庫
check:檢測keepalived的健康狀態相關進程 vim
system call:系統調用後端
watch dog:監控check和vrrp進程的看管者，check負責檢測器子進程的健康狀態，當其檢測到master上的服務不可用時則通告vrrp將其轉移至backup服務器上。centos
三 環境準備
操做系統：centos7.1.1511（core）bash
數據庫： mysql5.7.21 社區版服務器
master1 ：10.0.0.11 安裝mysql 和keeplived負載均衡
master2 ：10.0.0.12 安裝mysql 和keeplived
VIP：10.0.0.20
要實現互爲主從，就必須 mster1-->master2設置主從同步 同時 master2--->master1 也設置主從同步
4、Mysql主主同步環境部署
---------------master1服務器操做記錄--------------- 在my.cnf文件的[mysqld]配置區域添加下面內容： [root@master1 ~]# vim /usr/local/mysql/my.cnf server-id = 1 log-bin = mysql-bin sync_binlog = 1 binlog_checksum = none binlog_format = mixed auto-increment-increment = 2 auto-increment-offset = 1 slave-skip-errors = all [root@master1 ~]# /etc/init.d/mysql restart Shutting down MySQL. SUCCESS! Starting MySQL.. SUCCESS!
建立一個複製用戶
出了小問題，因爲以前root用戶的密碼設置過於簡單在建立複製用戶時報以下錯誤
mysql> grant replication slave,replication client on *.* to repl@'10.0.0.%' identified by '1qaz@WSX'; ERROR 1820 (HY000): You must reset your password using ALTER USER statement before executing this statement.
按照提示將密碼設置的複雜一點 在受權建立就沒有問題了
mysql> alter user 'root'@'localhost' identified by '1qaz@WSX'; Query OK, 0 rows affected (0.00 sec) mysql> flush privileges; Query OK, 0 rows affected (0.00 sec)
mysql> grant replication slave,replication client on *.* to repl@'10.0.0.%' identified by '1qaz@WSX'; Query OK, 0 rows affected, 1 warning (0.00 sec) mysql> flush privileges; Query OK, 0 rows affected (0.00 sec)
鎖表，待同步配置完成在解鎖
mysql> flush tables with read lock; Query OK, 0 rows affected (0.00 sec)
查看當前的binlog以及數據所在位置
mysql> show master status; +------------------+----------+--------------+------------------+-------------------+ | File | Position | Binlog_Do_DB | Binlog_Ignore_DB | Executed_Gtid_Set | +------------------+----------+--------------+------------------+-------------------+ | mysql-bin.000006 | 996 | | | | +------------------+----------+--------------+------------------+-------------------+ 1 row in set (0.00 sec)
---------------master2服務器操做記錄--------------- 在my.cnf文件的[mysqld]配置區域添加下面內容： [root@master2 ~]# vim /usr/local/mysql/my.cnf server-id = 2 log-bin = mysql-bin sync_binlog = 1 binlog_checksum = none binlog_format = mixed auto-increment-increment = 2 auto-increment-offset = 2 slave-skip-errors = all [root@master2 ~]# /etc/init.d/mysql restart Shutting down MySQL.. SUCCESS! Starting MySQL.. SUCCESS!
mysql> grant replication slave,replication client on *.* to repl@'10.0.0.%' identified by '1qaz@WSX'; Query OK, 0 rows affected, 1 warning (0.01 sec) mysql> flush privileges; Query OK, 0 rows affected (0.00 sec)
mysql> flush tables with read lock;
Query OK, 0 rows affected (0.00 sec)
查看 master狀況
mysql> show master status; +------------------+----------+--------------+------------------+-------------------+ | File | Position | Binlog_Do_DB | Binlog_Ignore_DB | Executed_Gtid_Set | +------------------+----------+--------------+------------------+-------------------+ | mysql-bin.000001 | 150 | | | | +------------------+----------+--------------+------------------+-------------------+ 1 row in set (0.00 sec)
分別開啓同步對方
---------------master1服務器作同步操做--------------- mysql> unlock tables; //先解鎖，將對方數據同步到本身的數據庫中 mysql> slave stop； mysql> change master to master_host='10.0.0.12',master_user='repl',master_password='1qaz@WSX',master_log_file='mysql-bin.000001',master_log_pos=150; Query OK, 0 rows affected, 2 warnings (0.01 sec)
mysql> start slave; Query OK, 0 rows affected (0.01 sec)
mysql> show slave status \G;
Slave_IO_Running: Yes
Slave_SQL_Running: Yes
-------------master2服務器作同步操做--------------- mysql> unlock tables; //先解鎖，將對方數據同步到本身的數據庫中 mysql> slave stop； mysql> change master to master_host='10.0.0.11',master_user='repl',master_password='1qaz@WSX',master_log_file='mysql-bin.000006',master_log_pos=996; Query OK, 0 rows affected, 2 warnings (0.06 sec) mysql> start slave; Query OK, 0 rows affected (0.01 sec) mysql> show slave status \G;
Master_Log_File: mysql-bin.000006
Read_Master_Log_Pos: 996
Relay_Log_File: master2-relay-bin.000002
Relay_Log_Pos: 312
Relay_Master_Log_File: mysql-bin.000006
Slave_IO_Running: Yes
Slave_SQL_Running: Yes
以上代表雙方已經實現了mysql主主同步。
當運行一段時間後，要是發現同步有問題，好比只能單向同步，雙向同步失效。能夠從新執行下上面的change master同步操做，只不過這樣同步後，只能同步在此以後的更新數據。下面開始進行數據驗證：
-----------------主主同步效果驗證--------------------- 1）在master1數據庫上寫入新數據 mysql> unlock tables; Query OK, 0 rows affected (0.00 sec) mysql> create database huanqiu; Query OK, 1 row affected (0.01 sec) mysql> use huanqiu; Database changed
mysql> create table if not exists haha ( id int(10) PRIMARY KEY AUTO_INCREMENT, name varchar(50) NOT NULL);
Query OK, 0 rows affected (0.04 sec)
mysql> insert into haha values(2,'guojing');
Query OK, 1 row affected (0.00 sec)
mysql> insert into haha values(1,"huangrong");
Query OK, 1 row affected (0.00 sec)
mysql> select * from haha;
+----+-----------+
| id | name |
+----+-----------+
| 1 | huangrong |
| 2 | guojing |
+----+-----------+
2 rows in set (0.00 sec)
而後在master2數據庫上查看，發現數據已經同步過來了!
mysql> select * from huanqiu.haha;
+----+-----------+
| id | name |
+----+-----------+
| 1 | huangrong |
| 2 | guojing |
+----+-----------+
2 rows in set (0.00 sec)
2）在master2數據庫上寫入新數據 mysql> create database hehe; Query OK, 1 row affected (0.00 sec)
mysql> insert into huanqiu.haha values(3,"haha"),(4,"haha");
Query OK, 2 rows affected (0.00 sec)
Records: 2 Duplicates: 0 Warnings: 0
而後在master1數據庫上查看，發現數據也已經同步過來了! mysql> show databases; +--------------------+ | Database | +--------------------+ | information_schema | | hehe | | huanqiu | | mysql | | performance_schema | | test | +--------------------+ 6 rows in set (0.00 sec)
mysql> select * from huanqiu.haha;
+----+-----------+
| id | name |
+----+-----------+
| 1 | huangrong |
| 2 | guojing |
| 3 | haha |
| 4 | haha |
+----+-----------+
4 rows in set (0.00 sec)
至此，Mysql主主同步環境已經實現。
五 配置mysql+keepalived 高可用環境
1）安裝keepalived並將其配置成系統服務。master1和master2兩臺機器上一樣進行以下操做： [root@master1 ~]# yum install -y openssl-devel [root@master1 ~]# cd /usr/local/src/ [root@master1 src]# wget http://www.keepalived.org/software/keepalived-1.3.5.tar.gz [root@master1 src]# tar -zvxf keepalived-1.3.5.tar.gz [root@master1 src]# cd keepalived-1.3.5 [root@master1 keepalived-1.3.5]# ./configure --prefix=/usr/local/keepalived [root@master1 keepalived-1.3.5]# make && make install [root@master1 keepalived-1.3.5]# cp /usr/local/src/keepalived-1.3.5/keepalived/etc/init.d/keepalived /etc/rc.d/init.d/ [root@master1 keepalived-1.3.5]# cp /usr/local/keepalived/etc/sysconfig/keepalived /etc/sysconfig/ [root@master1 keepalived-1.3.5]# mkdir /etc/keepalived/ [root@master1 keepalived-1.3.5]# cp /usr/local/keepalived/etc/keepalived/keepalived.conf /etc/keepalived/ [root@master1 keepalived-1.3.5]# cp /usr/local/keepalived/sbin/keepalived /usr/sbin/ [root@master1 keepalived-1.3.5]# echo "/etc/init.d/keepalived start" >> /etc/rc.local
2）master1機器上的keepalived.conf配置。（下面配置中沒有使用lvs的負載均衡功能，因此不須要配置虛擬服務器virtual server）
[root@master1 ~]# cp /etc/keepalived/keepalived.conf /etc/keepalived/keepalived.conf.bak [root@master1 ~]# vim /etc/keepalived/keepalived.conf ＃清空默認內容，直接採用下面配置： ! Configuration File for keepalived global_defs { notification_email { ops@wangshibo.cn tech@wangshibo.cn } notification_email_from ops@wangshibo.cn smtp_server 127.0.0.1 smtp_connect_timeout 30 router_id MASTER-HA } vrrp_script chk_mysql_port { #檢測mysql服務是否在運行。有不少方式，好比進程，用腳本檢測等等 script "/opt/chk_mysql.sh" #這裏經過腳本監測 interval 2 #腳本執行間隔，每2s檢測一次 weight -5 #腳本結果致使的優先級變動，檢測失敗（腳本返回非0）則優先級 -5 fall 2 #檢測連續2次失敗纔算肯定是真失敗。會用weight減小優先級（1-255之間） rise 1 #檢測1次成功就算成功。但不修改優先級 } vrrp_instance VI_1 { state MASTER interface eth0 #指定虛擬ip的網卡接口 mcast_src_ip 10.0.0.11 virtual_router_id 51 #路由器標識，MASTER和BACKUP必須是一致的 priority 101 #定義優先級，數字越大，優先級越高，在同一個vrrp_instance下，MASTER的優先級必須大於BACKUP的優先級。這樣MASTER故障恢復後，就能夠將VIP資源再次搶回來 advert_int 1 authentication { auth_type PASS auth_pass 1111 } virtual_ipaddress { 10.0.0.20 } track_script { chk_mysql_port } }
編寫切換腳本。KeepAlived作心跳檢測，若是Master的MySQL服務掛了(3306端口掛了),那麼它就會選擇自殺。Slave的KeepAlived經過心跳檢測發現這個狀況，就會將VIP的請求接管
[root@master1 ~]# vim /opt/chk_mysql.sh #!/bin/bash counter=$(netstat -na|grep "LISTEN"|grep "3306"|wc -l) if [ "${counter}" -eq 0 ]; then /etc/init.d/keepalived stop fi
[root@master1 ~]# chmod 755 /opt/chk_mysql.sh 啓動keepalived服務 [root@master1 ~]# /etc/init.d/keepalived start 正在啓動 keepalived： [肯定]
4）master2機器上的keepalived配置。master2機器上的keepalived.conf文件只修改priority爲90、nopreempt不設置、real_server設置本地IP。
[root@master2 ~]# cp /etc/keepalived/keepalived.conf /etc/keepalived/keepalived.conf.bak [root@master2 ~]# >/etc/keepalived/keepalived.conf [root@master2 ~]# vim /etc/keepalived/keepalived.conf ! Configuration File for keepalived global_defs { notification_email { ops@qq.com tech@qq.com } notification_email_from ops@wangshibo.cn smtp_server 127.0.0.1 smtp_connect_timeout 30 router_id MASTER-HA } vrrp_script chk_mysql_port { script "/opt/chk_mysql.sh" interval 2 weight -5 fall 2 rise 1 } vrrp_instance VI_1 { state BACKUP interface eth0 mcast_src_ip 10.0.0.12 virtual_router_id 51 priority 99 advert_int 1 authentication { auth_type PASS auth_pass 1111 } virtual_ipaddress { 10.0.0.20 } track_script { chk_mysql_port } } [root@master2 ~]# cat /opt/chk_mysql.sh #!/bin/bash counter=$(netstat -na|grep "LISTEN"|grep "3306"|wc -l) if [ "${counter}" -eq 0 ]; then /etc/init.d/keepalived stop fi [root@master2 ~]# chmod 755 /opt/chk_mysql.sh [root@master2 ~]# /etc/init.d/keepalived start 正在啓動 keepalived： [肯定]
我這裏啓動時出現了問題，分析日誌
tail -f /var/log/message Mar 31 14:28:14 master1 systemd: Configuration file /usr/lib/systemd/system/ebtables.service is marked executable. Please remove executable permission bits. Proceeding anyway.
查看keepalived.service
# vi /lib/systemd/system/keepalived.service
[Unit]
Description=LVS and VRRP High Availability Monitor
After=syslog.target network-online.target
[Service]
Type=forking
# PIDFile=/usr/local/keepalived/var/run/keepalived.pid
# 上面這個註釋掉 改爲下面 由於這個默認路徑不存在因此就沒法寫入進程ID文件
PIDFile=/var/run/keepalived.pid
KillMode=process
EnvironmentFile=-/usr/local/keepalived/etc/sysconfig/keepalived
ExecStart=/usr/local/keepalived/sbin/keepalived $KEEPALIVED_OPTIONS
ExecReload=/bin/kill -HUP $MAINPID
[Install]
WantedBy=multi-user.target
5）master1和master2兩臺服務器都要受權容許root用戶遠程登陸，用於在客戶端登錄測試！
mysql> grant all on *.* to root@'10.0.0.%' identified by "1qaz@WSX"; Query OK, 0 rows affected (0.00 sec) mysql> flush privileges; Query OK, 0 rows affected (0.01 sec)
6）在master1和master2兩臺機器上設置iptables防火牆規則，以下：
[root@master1 ~]# cat /etc/sysconfig/iptables ........ -A INPUT -s 10.0.0.0/24 -d 224.0.0.18 -j ACCEPT #容許組播地址通訊 -A INPUT -s 10.0.0.0/24 -p vrrp -j ACCEPT #容許VRRP（虛擬路由器冗餘協）通訊 -A INPUT -m state --state NEW -m tcp -p tcp --dport 3306 -j ACCEPT #開放mysql的3306端口 [root@master1 ~]# /etc/init.d/iptables restart
六 Mysql＋keepalived故障轉移的高可用測試
1）經過Mysql客戶端經過VIP鏈接，看是否鏈接成功。
好比，在遠程一臺測試機上鍊接，經過vip地址能夠正常鏈接（下面的鏈接權限要是在服務端提早受權的）
[root@master1 ~]# mysql -uroot -p1qaz@WSX -h10.0.0.20 mysql: [Warning] Using a password on the command line interface can be insecure. Welcome to the MySQL monitor. Commands end with ; or \g. Your MySQL connection id is 11 Server version: 5.7.21-log MySQL Community Server (GPL) Copyright (c) 2000, 2018, Oracle and/or its affiliates. All rights reserved. Oracle is a registered trademark of Oracle Corporation and/or its affiliates. Other names may be trademarks of their respective owners. Type 'help;' or '\h' for help. Type '\c' to clear the current input statement.
mysql> select * from huanqiu.haha;
+----+-----------+
| id | name |
+----+-----------+
| 1 | huangrong |
| 2 | guojing |
| 3 | haha |
| 4 | haha |
+----+-----------+
4 rows in set (0.02 sec)
2）默認狀況下，vip是在master1上的。使用"ip addr"命令查看vip切換狀況
[root@master1 ~]# ip addr |grep 10.0 2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen 1000 inet 10.0.0.11/8 brd 10.255.255.255 scope global eth0 inet 10.0.0.20/32 scope global eth0
[root@master2 ~]# ip addr |grep 10.0
2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen 1000
inet 10.0.0.12/8 brd 10.255.255.255 scope global eth0
中止master1機器上的mysql服務，根據配置中的腳本，mysql服務停了，keepalived也會停，從而vip資源將會切換到master2機器上。（mysql服務沒有起來的時候，keepalived服務也沒法順利啓動！）
[root@master1 ~]# systemctl stop mysqld
[root@master1 ~]# ps -ef|grep mysql root 4431 2423 0 15:08 pts/0 00:00:00 grep --color=auto mysql [root@master1 ~]# ps -ef|grep keepalived root 4433 2423 0 15:08 pts/0 00:00:00 grep --color=auto keepalived
[root@master1 ~]# ip addr |grep 10.0 2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen 1000 inet 10.0.0.11/8 brd 10.255.255.255 scope global eth0
查看master2主機
[root@master2 ~]# ip addr |grep 10.0 2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen 1000 inet 10.0.0.12/8 brd 10.255.255.255 scope global eth0 inet 10.0.0.20/32 scope global eth0
3）再次啓動master1的mysql和keepalived服務。（注意：若是restart重啓mysql，那麼還要啓動下keepalived，由於mysql重啓，根據腳本會形成keepalived關閉）
注意：必定要先啓動mysql服務，而後再啓動keepalived服務。若是先啓動keepalived服務，按照上面的配置，mysql沒有起來，就會自動關閉keepalived。
[root@master1 ~]# systemctl start mysqld [root@master1 ~]# /etc/init.d/keepalived start Starting keepalived (via systemctl): [ OK ]
[root@master1 ~]# ip addr |grep 10.0 2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen 1000 inet 10.0.0.11/8 brd 10.255.255.255 scope global eth0 inet 10.0.0.20/32 scope global eth0
[root@master2 ~]# ip addr |grep 10.0
2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen 1000
inet 10.0.0.12/8 brd 10.255.255.255 scope global eth0
此時虛擬ip又回到了 master1 主機上了
以上在vip資源切換過程當中，對於客戶端鏈接mysql（使用vip鏈接）來講幾乎是沒有任何影響的。
---------------------------------舒適提示（Keepalived的搶佔和非搶佔模式）---------------------------------------
keepalive是基於vrrp協議在linux主機上以守護進程方式，根據配置文件實現健康檢查。
VRRP是一種選擇協議，它能夠把一個虛擬路由器的責任動態分配到局域網上的VRRP路由器中的一臺。
控制虛擬路由器IP地址的VRRP路由器稱爲主路由器，它負責轉發數據包到這些虛擬IP地址。
一旦主路由器不可用，這種選擇過程就提供了動態的故障轉移機制，這就容許虛擬路由器的IP地址能夠做爲終端主機的默認第一跳路由器。
keepalive經過組播，單播等方式（自定義），實現keepalive主備推選。工做模式分爲搶佔和非搶佔（經過參數nopreempt來控制）。
1）搶佔模式：
主服務正常工做時，虛擬IP會在主上，備不提供服務，當主服務優先級低於備的時候，備會自動搶佔虛擬IP，這時，主不提供服務，備提供服務。
也就是說，工做在搶佔模式下，不分主備，只管優先級。
如上配置，無論keepalived.conf裏的state配置成master仍是backup，只看誰的priority優先級高（通常而言，state爲MASTER的優先級要高於BACKUP）。
priority優先級高的那一個在故障恢復後，會自動將VIP資源再次搶佔回來！！
2）非搶佔模式：
這種方式經過參數nopreempt（通常設置在advert_int的那一行下面）來控制。無論priority優先級，只要MASTER機器發生故障，VIP資源就會被切換到BACKUP上。
而且當MASTER機器恢復後，也不會去將VIP資源搶佔回來，直至BACKUP機器發生故障時，才能自動切換回來。
千萬注意：
nopreempt這個參數只能用於state爲backup的狀況，因此在配置的時候要把master和backup的state都設置成backup，這樣纔會實現keepalived的非搶佔模式！
也就是說：
a）當state狀態一個爲master，一個爲backup的時候，加不加nopreempt這個參數都是同樣的效果。即都是根據priority優先級來決定誰搶佔vip資源的，是搶佔模式！
b）當state狀態都設置成backup，若是不配置nopreempt參數，那麼也是看priority優先級決定誰搶佔vip資源，即也是搶佔模式。
c）當state狀態都設置成backup，若是配置nopreempt參數，那麼就不會去考慮priority優先級了，是非搶佔模式！即只有vip當前所在機器發生故障，另外一臺機器才能接管vip。即便優先級高的那一臺機器恢復 後也不會主動搶回vip，只能等到對方發生故障，纔會將vip切回來。
---------------------------------mysql狀態檢測腳本優化---------------------------------
上面的mysql監測腳本有點過於簡單且粗暴，即腳本一旦監測到Master的mysql服務關閉，就馬上把keepalived服務關閉，從而實現vip轉移！
下面對該腳本進行優化，優化後，當監測到Master的mysql服務關閉後，就會將vip切換到Backup上（但此時Master的keepalived服務不會被暴力
kill
）
當Master的mysql服務恢復後，就會再次將VIP資源切回來！
[root@master ~]# cat /opt/chk_mysql.sh #!/bin/bash MYSQL=/usr/bin/mysql MYSQL_HOST=localhost MYSQL_USER=root MYSQL_PASSWORD=1qaz@WSX CHECK_TIME=3 #mysql is working MYSQL_OK is 1 , mysql down MYSQL_OK is 0 MYSQL_OK=1 function check_mysql_helth (){ $MYSQL -h $MYSQL_HOST -u $MYSQL_USER -p${MYSQL_PASSWORD} -e "show status;" >/dev/null 2>&1 if [ $? = 0 ] ;then MYSQL_OK=1 else MYSQL_OK=0 fi return $MYSQL_OK } while [ $CHECK_TIME -ne 0 ] do let "CHECK_TIME -= 1" check_mysql_helth if [ $MYSQL_OK = 1 ] ; then CHECK_TIME=0 exit 0 fi if [ $MYSQL_OK -eq 0 ] && [ $CHECK_TIME -eq 0 ] then pkill keepalived exit 1 fi sleep 1 done