雙機高可用、負載均衡、MySQL(讀寫分離、主從自動切換)架構設計
架構簡介
此架構主要是由keepalived實現雙機高可用，維護了一個外網VIP,一個內網VIP。正常狀況時，外網VIP和內網VIP都綁定在server1服務器，web請求發送到server1的Nginx，nginx對於靜態資源請求就直接在本機檢索並返回，對於PHP的動態請求，則負載均衡到server1和server2。對於SQL請求，會將此類請求發送到Atlas mysql中間件，Atlas接收到請求以後，把涉及寫操做的請求發送到內網VIP，讀請求操做發送到server2，這樣就實現了讀寫分離。php
當主服務器server1宕機時，keepalived檢測到後，當即把外網VIP和內網VIP綁定到server2，並把server2的mysql切 換成主庫。此時因爲外網VIP已經轉移到了server2，web請求將發送給server2的nginx。nginx檢測到server1宕機，再也不把 請求轉發到server1的php-fpm。以後的sql請求照常發送給本地的atlas，atlas把寫操做發送給內網VIP，讀操做發送給server2 mysql，因爲內網VIP已經綁定到server2了，server2的mysql同時接受寫操做和讀操做。python
當主服務器server1恢復後，keepalived不搶佔server2的VIP，繼續正常服務。咱們能夠把server1的mysql切換成主，也能夠切換成從。mysql
架構要求
要實現此架構，須要三個條件：linux
- 服務器能夠設置內網ip，而且設置的內網IP互通;
- 服務器能夠隨意綁定IDC分配給咱們使用的外網IP，即外網IP沒有綁定MAC地址;
- MySQL服務器支持GTID，即MySQL-5.6.5以上版本。
環境說明
server1nginx
eth0: 10.96.153.110(對外IP) eth1: 192.168.3.100(對內IP)
server2git
eth0: 10.96.153.114(對外IP) eth1: 192.168.3.101(對內IP)
系統都是CentOS-6。github
對外VIP: 10.96.153.239 對內VIP: 192.168.3.150
hosts設置web
/etc/hosts: 192.168.3.100 server1 192.168.3.101 server2
Nginx PHP MySQL安裝
這幾個軟件的安裝推薦使用EZHTTP來完成。sql
Nginx配置
Server1配置數據庫
http {
[...]
upstream php-server {
server 192.168.3.101:9000;
server 127.0.0.1:9000;
keepalive 100;
}
[...]
server {
[...]
location ~ \.php$ {
fastcgi_pass php-server;
fastcgi_index index.php;
fastcgi_param SCRIPT_FILENAME $document_root$fastcgi_script_name;
include fastcgi_params;
}
[...]
}
[...]
}
Server2配置
http {
[...]
upstream php-server {
server 192.168.3.100:9000;
server 127.0.0.1:9000;
keepalive 100;
}
[...]
server {
[...]
location ~ \.php$ {
fastcgi_pass php-server;
fastcgi_index index.php;
fastcgi_param SCRIPT_FILENAME $document_root$fastcgi_script_name;
include fastcgi_params;
}
[...]
}
[...]
}
這兩個配置主要的做用是設置php請求的負載均衡。
MySQL配置
mysql util安裝
咱們須要安裝mysql util裏的主從配置工具來實現主從切換。
cd /tmp wget http://dev.mysql.com/get/Downloads/MySQLGUITools/mysql-utilities-1.5.3.tar.gz tar xzf mysql-utilities-1.5.3.tar.gz cd mysql-utilities-1.5.3 python setup.py build python setup.py install
mysql my.cnf配置
server1:
[mysql] [...] protocol=tcp [...] [...] [mysqld] [...] # BINARY LOGGING # log-bin = /usr/local/mysql/data/mysql-bin expire-logs-days = 14 sync-binlog = 1 binlog-format=ROW log-slave-updates=true gtid-mode=on enforce-gtid-consistency =true master-info-repository=TABLE relay-log-info-repository=TABLE sync-master-info=1 server-id=1 report-host=server1 report-port=3306 [...]
server2:
[mysql] [...] protocol=tcp [...] [mysqld] [...] # BINARY LOGGING # log-bin = /usr/local/mysql/data/mysql-bin expire-logs-days = 14 sync-binlog = 1 binlog-format=ROW log-slave-updates=true gtid-mode=on enforce-gtid-consistency =true master-info-repository=TABLE relay-log-info-repository=TABLE sync-master-info=1 server-id=2 report-host=server2 report-port=3306 [...]
這兩個配置主要是設置了binlog和啓用gtid-mode，而且須要設置不一樣的server-id和report-host。
開放root賬號遠程權限：
咱們須要在兩臺mysql服務器設置root賬號遠程訪問權限。
mysql> grant all on *.* to 'root'@'192.168.3.%' identified by 'Xp29at5F37' with grant option; mysql> grant all on *.* to 'root'@'server1' identified by 'Xp29at5F37' with grant option; mysql> grant all on *.* to 'root'@'server2' identified by 'Xp29at5F37' with grant option; mysql> flush privileges;
設置mysql主從
在任意一臺執行以下命令：
mysqlreplicate --master=root:Xp29at5F37@server1:3306 --slave=root:Xp29at5F37@server2:3306 --rpl-user=rpl:o67DhtaW # master on server1: ... connected. # slave on server2: ... connected. # Checking for binary logging on master... # Setting up replication... # ...done.
顯示主從關係
mysqlrplshow --master=root:Xp29at5F37@server1 --discover-slaves-login=root:Xp29at5F37 # master on server1: ... connected. # Finding slaves for master: server1:3306 # Replication Topology Graph server1:3306 (MASTER) | +--- server2:3306 - (SLAVE)
檢查主從狀態
mysqlrplcheck --master=root:Xp29at5F37@server1 --slave=root:Xp29at5F37@server2 # master on server1: ... connected. # slave on server2: ... connected. test Description Status --------------------------------------------------------------------------- Checking for binary logging on master [pass] Are there binlog exceptions? [pass] Replication user exists? [pass] Checking server_id values [pass] Checking server_uuid values [pass] Is slave connected to master? [pass] Check master information file [pass] Checking InnoDB compatibility [pass] Checking storage engines compatibility [pass] Checking lower_case_table_names settings [pass] Checking slave delay (seconds behind master) [pass] # ...done.
在server2創建主從切換腳本
vi /data/sh/mysqlfailover.sh #!/bin/bash mysqlrpladmin --slave=root:Xp29at5F37@server2:3306 failover chmod +x /data/sh/mysqlfailover.sh
Keepalived配置
keepalived安裝(兩臺都裝)
yum -y install keepalived chkconfig keepalived on
keepalived配置(server1)
vi /etc/keepalived/keepalived.conf
vrrp_sync_group VG_1 {
group {
inside_network
outside_network
}
}
vrrp_instance inside_network {
state BACKUP
interface eth1
virtual_router_id 51
priority 101
advert_int 1
authentication {
auth_type PASS
auth_pass 3489
}
virtual_ipaddress {
192.168.3.150/24
}
nopreempt
}
vrrp_instance outside_network {
state BACKUP
interface eth0
virtual_router_id 50
priority 101
advert_int 1
authentication {
auth_type PASS
auth_pass 3489
}
virtual_ipaddress {
10.96.153.239/24
}
nopreempt
}
keepalived配置(server2)
vrrp_sync_group VG_1 {
group {
inside_network
outside_network
}
}
vrrp_instance inside_network {
state BACKUP
interface eth1
virtual_router_id 51
priority 100
advert_int 1
authentication {
auth_type PASS
auth_pass 3489
}
virtual_ipaddress {
192.168.3.150
}
notify_master /data/sh/mysqlfailover.sh
}
vrrp_instance outside_network {
state BACKUP
interface eth0
virtual_router_id 50
priority 100
advert_int 1
authentication {
auth_type PASS
auth_pass 3489
}
virtual_ipaddress {
10.96.153.239/24
}
}
此keepalived配置須要注意的是：
- 兩臺server的state都設置爲backup，server1增長nopreempt配置，而且server1 priority比server2高，這樣用來實現當server1從宕機恢復時，不搶佔VIP;
- server2設置notify_master /data/sh/mysqlfailover.sh,意味着server2接管server1後，執行這個腳本，以把server2的mysql提高爲主。
Atlas設置
atlas安裝
到這裏下載最新版本，https://github.com/Qihoo360/Atlas/releases
cd /tmp wget https://github.com/Qihoo360/Atlas/releases/download/2.2.1/Atlas-2.2.1.el6.x86_64.rpm rpm -i Atlas-2.2.1.el6.x86_64.rpm
atlas配置
cd /usr/local/mysql-proxy/conf cp test.cnf my.cnf vi my.cnf
調整以下參數，
proxy-backend-addresses = 192.168.3.150:3306 proxy-read-only-backend-addresses = 192.168.3.101:3306 pwds = root:qtyU1btXOo074Itvx0UR9Q== event-threads = 8
注意：
proxy-backend-addresse設置爲內網VIP
proxy-read-only-backend-addresses設置爲server2的IP
root:qtyU1btXOo074Itvx0UR9Q==設置數據庫的用戶和密碼，密碼是經過/usr/local/mysql-proxy/bin/encrypt Xp29at5F37生成。更詳細參數解釋請查看，Atlas配置詳解。
啓動atlas
/usr/local/mysql-proxy/bin/mysql-proxy --defaults-file=/usr/local/mysql-proxy/conf/my.cnf
以後程序裏配置mysql就配置127.0.0.1:1234就好。
server1主宕機測試
測試keepalived是否工做正常，咱們來模擬server1宕機。在server1上執行shutdown關機命令。此時咱們登陸server2，執行ip addr命令，輸出以下：
1: lo: mtu 16436 qdisc noqueue state UNKNOWN link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 inet 127.0.0.1/8 scope host lo inet6 ::1/128 scope host valid_lft forever preferred_lft forever 2: eth0: mtu 1500 qdisc pfifo_fast state UP qlen 1000 link/ether 00:0c:29:81:9d:42 brd ff:ff:ff:ff:ff:ff inet 10.96.153.114/24 brd 10.96.153.255 scope global eth0 inet 10.96.153.239/24 scope global secondary eth0 inet6 fe80::20c:29ff:fe81:9d42/64 scope link valid_lft forever preferred_lft forever 3: eth1: mtu 1500 qdisc pfifo_fast state UP qlen 1000 link/ether 00:0c:29:81:9d:4c brd ff:ff:ff:ff:ff:ff inet 192.168.3.101/24 brd 192.168.3.255 scope global eth1 inet 192.168.3.150/32 scope global eth1 inet6 fe80::20c:29ff:fe81:9d4c/64 scope link valid_lft forever preferred_lft forever 咱們看到對外VIP 10.96.153.239和對內IP 192.168.3.150已經轉移到server2了，證實keepalived運行正常。
測試是否自動切換了主從，登陸server2的mysql服務器，執行show status;命令，以下：
mysql> show slave statusG Empty set (0.00 sec)
咱們發現從狀態已經爲空，證實已經切換爲主了。
測試server1是否搶佔VIP，爲何要測試這個呢？若是server1恢復以後搶佔了VIP，而咱們的Atlas裏後端設置的是VIP，這樣 server1啓動以後，sql的寫操做就會向server1的mysql發送，而server1的mysql數據是舊於server2的，因此這樣會造 成數據不一致，這個是很是重要的測試。
咱們先來啓動server1，以後執行ip addr，輸出以下：
1: lo: mtu 16436 qdisc noqueue state UNKNOWN link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 inet 127.0.0.1/8 scope host lo inet6 ::1/128 scope host valid_lft forever preferred_lft forever 2: eth0: mtu 1500 qdisc pfifo_fast state UP qlen 1000 link/ether 00:0c:29:f1:4f:4e brd ff:ff:ff:ff:ff:ff inet 10.96.153.110/24 brd 10.96.153.255 scope global eth0 inet6 fe80::20c:29ff:fef1:4f4e/64 scope link valid_lft forever preferred_lft forever 3: eth1: mtu 1500 qdisc pfifo_fast state UP qlen 1000 link/ether 00:0c:29:f1:4f:58 brd ff:ff:ff:ff:ff:ff inet 192.168.3.100/24 brd 192.168.3.255 scope global eth1 inet6 fe80::20c:29ff:fef1:4f58/64 scope link valid_lft forever preferred_lft forever
咱們看到，server1並無搶佔VIP，測試正常。不過另人鬱悶的是，在虛擬機的環境並無測試成功，不知道爲何。
如何恢復server1
設置server1 mysql爲從，server1從宕機中恢復以後，mysql的數據已經舊於server2的數據了，這時咱們先設置server1 mysql爲從。
mysqlreplicate --master=root:Xp29at5F37@server2:3306 --slave=root:Xp29at5F37@server1:3306 --rpl-user=rpl:o67DhtaW # master on server2: ... connected. # slave on server1: ... connected. # Checking for binary logging on master... # Setting up replication... # ...done.
看到提示是設置成功了。
獲取server1 mysql數據數據同步狀況，server1 mysql剛從宕機恢復，有可能數據遠遠落後於server2 mysql，因此咱們先查看它們之間的數據同步狀況。登陸server1 mysql，執行以下sql:
mysql> show slave statusG *************************** 1. row *************************** Slave_IO_State: Waiting for master to send event Master_Host: server2 Master_User: rpl Master_Port: 3306 Connect_Retry: 60 Master_Log_File: mysql-bin.000004 Read_Master_Log_Pos: 2894 Relay_Log_File: mysql-relay-bin.000002 Relay_Log_Pos: 408 Relay_Master_Log_File: mysql-bin.000004 Slave_IO_Running: yes Slave_SQL_Running: Yes
咱們記下Read_Master_Log_Pos的值爲2894，登陸server2 mysql，執行以下sql:
mysql> show master statusG *************************** 1. row *************************** File: mysql-bin.000004 Position: 2894 Binlog_Do_DB: Binlog_Ignore_DB: Executed_Gtid_Set: 9347e042-9044-11e4-b4f0-000c29f14f4e:1-7, f5bbfc15-904a-11e4-b519-000c29819d42:1-6 1 row in set (0.00 sec)
記下Position的值，並與Read_Master_Log_Pos比較，若是這兩個值很是相近或相等，說明數據已經同步得差很少了，能夠進行切換操做;若是差得很遠，須要等待它們同步完成。
屏蔽mysql寫操做
咱們須要在切換時先禁止sql的寫操做，若是不這樣作，就會在切換時形成數據不一致的問題。屏蔽寫操做咱們在Atlas上操做。在server2執行登陸Atlas命令：
mysql -h127.0.0.1 -P2345 -uuser -ppwd mysql> SELECT * FROM backends; +-------------+--------------------+-------+------+ | backend_ndx | address | state | type | +-------------+--------------------+-------+------+ | 1 | 192.168.3.150:3306 | up | rw | | 2 | 192.168.3.101:3306 | up | ro | +-------------+--------------------+-------+------+ 2 rows in set (0.00 sec)
執行SELECT * FROM backends;後咱們看到backend id爲1,因此咱們執行SET OFFLINE 1;設置此後端下線。
mysql> SET OFFLINE 1; +-------------+--------------------+---------+------+ | backend_ndx | address | state | type | +-------------+--------------------+---------+------+ | 1 | 192.168.3.150:3306 | offline | rw | +-------------+--------------------+---------+------+ 1 row in set (0.00 sec)
mysql> SELECT * FROM backends; +-------------+--------------------+---------+------+ | backend_ndx | address | state | type | +-------------+--------------------+---------+------+ | 1 | 192.168.3.150:3306 | offline | rw | | 2 | 192.168.3.101:3306 | up | ro | +-------------+--------------------+---------+------+ 2 rows in set (0.00 sec)
這時客戶端就沒法寫入數據了。
恢復server1 mysql爲主
mysqlrpladmin --master=root:Xp29at5F37@server2:3306 --new-master=root:Xp29at5F37@server1:3306 --demote-master --discover-slaves-login=root:Xp29at5F37 switchover # Discovering slaves for master at server2:3306 # Discovering slave at server1:3306 # Found slave: server1:3306 # Checking privileges. # Performing switchover from master at server2:3306 to slave at server1:3306. # Checking candidate slave prerequisites. # Checking slaves configuration to master. # Waiting for slaves to catch up to old master. # Stopping slaves. # Performing STOP on all slaves. # Demoting old master to be a slave to the new master. # Switching slaves to new master. # Starting all slaves. # Performing START on all slaves. # Checking slaves for errors. # Switchover complete.
再次檢查是否恢復成功.
mysqlrplcheck --master=root:Xp29at5F37@server1 --slave=root:Xp29at5F37@server2 # master on server1: ... connected. # slave on server2: ... connected. Test Description Status --------------------------------------------------------------------------- Checking for binary logging on master [pass] Are there binlog exceptions? [pass] Replication user exists? [pass] Checking server_id values [pass] Checking server_uuid values [pass] Is slave connected to master? [pass] Check master information file [pass] Checking InnoDB compatibility [pass] Checking storage engines compatibility [pass] Checking lower_case_table_names settings [pass] Checking slave delay (seconds behind master) [pass] # ...done.
設置VIP回到server1，在server2機器上執行：
/etc/init.d/keepalived restart
而後在兩臺機器分別執行ip addr查看ip綁定狀態。
設置server2 atlas後端上線
server2上執行mysql -h127.0.0.1 -P2345 -uuser -ppwd登陸，而後執行SET ONLINE 1;設置上線（這裏1是後端的id，可使用SELECT * FROM backends;查看）
mysql> SET ONLINE 1;
+-------------+--------------------+---------+------+
| backend_ndx | address | state | type |
+-------------+--------------------+---------+------+
| 1 | 192.168.3.150:3306 | unknown | rw |
+-------------+--------------------+---------+------+
1 row in set (0.00 sec)
mysql> SELECT * FROM backends; +-------------+--------------------+-------+------+ | backend_ndx | address | state | type | +-------------+--------------------+-------+------+ | 1 | 192.168.3.150:3306 | up | rw | | 2 | 192.168.3.101:3306 | up | ro | +-------------+--------------------+-------+------+ 2 rows in set (0.00 sec)
到這裏server1就恢復爲主了。
相關文章
- 1. 雙機高可用、負載均衡、MySQL(讀寫分離、主從自動切換)架構設計
- 2. 雙機高可用、負載均衡、MySQL (讀寫分離、主從自動切換)架構設計
- 3. 高可用（負載均衡）MYSQL（讀寫分離，主從切換）
- 4. mysql高可用架構方案之二(keepalived+lvs+讀寫分離+負載均衡)
- 5. MySQL負載均衡讀寫分離
- 6. MySQL 讀寫分離與負載均衡
- 7. 史上最全的MySQL高可用架構之【主從複製】【故障轉移】【讀寫分離】【負載均衡】
- 8. MySQL的MHA+keepalived+負載均衡+讀寫分離架構
- 9. Gtid+MGR+atlas讀寫分離以及負載均衡高可用架構
- 10. 解決Nginx + Keepalived主從雙機熱備+自動切換，實現負載均衡及高可用
- 更多相關文章...
- • 移動設備 統計 - 瀏覽器信息
- • Swift 自動引用計數（ARC） - Swift 教程
- • 漫談MySQL的鎖機制
- • 使用Rxjava計算圓周率
相關標籤/搜索
每日一句
-
每一个你不满意的现在，都有一个你没有努力的曾经。
歡迎關注本站公眾號,獲取更多信息
相關文章
- 1. 雙機高可用、負載均衡、MySQL(讀寫分離、主從自動切換)架構設計
- 2. 雙機高可用、負載均衡、MySQL (讀寫分離、主從自動切換)架構設計
- 3. 高可用（負載均衡）MYSQL（讀寫分離，主從切換）
- 4. mysql高可用架構方案之二(keepalived+lvs+讀寫分離+負載均衡)
- 5. MySQL負載均衡讀寫分離
- 6. MySQL 讀寫分離與負載均衡
- 7. 史上最全的MySQL高可用架構之【主從複製】【故障轉移】【讀寫分離】【負載均衡】
- 8. MySQL的MHA+keepalived+負載均衡+讀寫分離架構
- 9. Gtid+MGR+atlas讀寫分離以及負載均衡高可用架構
- 10. 解決Nginx + Keepalived主從雙機熱備+自動切換，實現負載均衡及高可用