前面咱們已經對MySQL數據庫的安裝、命令、備份、索引、事務以及存儲引擎等各個方面有了初步的理解,而今天咱們將從宏觀的角度來理解什麼是MySQL數據庫的主從複製與讀寫分離。mysql
在實際的生產環境中,若是對MySQL數據庫的讀與寫都在一臺服務器上進行操做,不管是在安全性,高可用性、仍是高併發性等諸多方面都是沒法知足的;而這就須要對MySQL數據庫進行主從複製與讀寫分離。linux
咱們使用一個架構圖來引出MySQL數據庫主從複製的原理以及其做用,請看下圖:sql
服務器做用:shell
MySQL主服務器:負責數據的寫入;數據庫
MySQL從服務器:同步主服務器的數據並進行數據的輪循讀取;vim
那麼這三臺具有相同服務的MySQL服務器就稱爲「MySQL羣集」。咱們能夠從上圖中看出,這樣的安排實現數據同步的基礎上,實現數據的讀寫分離,從而在保證數據的可靠性的同時也大大減輕了主服務器的壓力。後端
下面咱們對MySQL的主從複製和讀寫分離進行逐一介紹並給出配置實例。緩存
MySQL主從複製與讀寫分離有着緊密的聯繫,能夠這麼說,MySQL的讀寫分離的實現須要基於主從複製的基礎之上。tomcat
咱們經過下圖來理解MySQL主、從服務器是如何進行復制同步的過程的。安全
1)首先,MySQL主服務器在更新數據庫或其餘進行數據庫相關操做時,會在二進制日誌文件中記錄這些改變(這咱們在前面的增量備份與恢復的文章中進行了講述,log-bin的設置以及如何使用mysqladmin命令刷新該日誌。)當寫入日誌完成後,主服務器會告知存儲引擎提交事務;
2)MySQL從服務器會將主服務器的二進制日誌文件(Binary log)複製到其中繼日誌(Relay log)中。中繼日誌一般存放在系統緩存中,所以中繼日誌的開銷很小;
3)從服務器經過自身線程從中繼日誌中讀取事件,更新自身的日誌文件使其與主服務器中的數據一致。
ps:複製過程當中有一個很重要的限制,即在從服務器上覆制是串行化的,這就代表主服務器上的並行更新操做不能在從服務器上並行操做。
所需設備(咱們在虛擬機上模擬進行配置)清單:
Centos7一臺:做爲主服務器——master,ip地址爲192.168.68.133
Centos7兩臺:做爲從服務器——slave一、slave2IP地址分別爲192.168.68.12九、192.168.68.132
所需安裝服務:
安裝而且配置ntp服務、都安裝了MySQL5.7版本的數據庫
首先,咱們聯想一下增量備份的恢復操做,咱們都是依賴於data目錄下的二進制日誌文件,經過兩種方法實現的,其中一種就是根據時間節點進行備份恢復操做的。那麼咱們須要進行MySQL主從服務器複製,就須要先同步全部MySQL服務器的系統時間。
1)服務器名稱設置以區別
hostnamectl set-hostname master su
2)安裝配置ntp服務
yum install ntp -y #修改ntp服務配置文件,添加下面的兩句 vim /etc/ntp.conf server 127.127.68.0 #服務器本地網段,127.127表示的就是192.168 fudge 127.127.68.0 stratum 8 #時區設置(東八區) #保存退出 systemctl start ntpd #可使用netstat -nutp | grep ntpd 命令查看服務開啓狀態
3)關閉防火牆和SELinux功能
systemctl stop firewalld setenforce 0
1)服務器名稱設置
hostnamectl set-hostname slave1(slave2) su
2)兩臺從服務器上安裝ntp和ntpdate服務並開啓服務
yum install ntp ntpdate -y systemctl start ntpd
3)兩臺從服務器上關閉防火牆和SELinux功能
systemctl stop firewalld setenforce 0
4)使用ntpdate命令進行時間同步
[root@slave1 ~]# /usr/sbin/ntpdate 192.168.68.133 9 Jan 15:35:13 ntpdate[67450]: the NTP socket is in use, exiting
1)修改mysql的主配置文件並重啓mysql服務
vim /etc/my.cnf #配置以下: log-bin = master-bin #二進制日誌文件 master-bin能夠本身設置 server-id = 1 #服務器的id號,用於區別 log-slave-updates=true #開啓從服務器更新日誌功能(結合複製流程鏈接) systemctl restart mysqld.service
2)進入數據庫進行權限設置(受權)與刷新
[root@lokott ~]# mysql -uroot -p Enter password: Welcome to the MySQL monitor. Commands end with ; or \g. Your MySQL connection id is 3 Server version: 5.7.17-log Source distribution Copyright (c) 2000, 2016, Oracle and/or its affiliates. All rights reserved. Oracle is a registered trademark of Oracle Corporation and/or its affiliates. Other names may be trademarks of their respective owners. Type 'help;' or '\h' for help. Type '\c' to clear the current input statement. mysql> grant replication slave on *.* to 'myslave'@'192.168.68.%' identified by '123456'; Query OK, 0 rows affected, 1 warning (0.00 sec) mysql> flush privileges;#刷新系統權限表 Query OK, 0 rows affected (0.00 sec) mysql> show master status; +-------------------+----------+--------------+------------------+-------------------+ | File | Position | Binlog_Do_DB | Binlog_Ignore_DB | Executed_Gtid_Set | +-------------------+----------+--------------+------------------+-------------------+ | master-bin.000001 | 603 | | | | +-------------------+----------+--------------+------------------+-------------------+ 1 row in set (0.00 sec)
注意核心命令:受權命令的解釋,還有以上位置Position 記錄數據603
命令格式:
grant replication slave on *.* to 'myslave'@'192.168.68.%' identified by '123456'; #grant ——受權命令 replication slave 從服務器複製操做 *.*表示的是全部數據庫全部表
其具體含義是:賦予192.168.68.0網段的主機(服務器)複製(同步)主服務器的全部數據庫數據;
1)修改主配置文件並重啓服務
vim /etc/my.cnf #配置以下 log-bin = mysql-bin server-id = 2 relay-log = relay-log-bin #中繼日誌 relay-log-index = slave-relay-log-bin.index #索引文件 #保存退出重啓 systemctl restart mysqld.service
2)進入數據庫配置同步
[root@slave1 ~]# mysql -uroot -p ...#省略部份內容 mysql> change master to master_host='192.168.68.133',master_user='myslave',master_password='123456',master_log_file='master-bin.000001',master_log_pos=603; Query OK, 0 rows affected, 2 warnings (0.01 sec) mysql> start slave; Query OK, 0 rows affected (0.00 sec) mysql> show slave status\G *************************** 1. row *************************** Slave_IO_State: Waiting for master to send event Master_Host: 192.168.68.133 Master_User: myslave Master_Port: 3306 Connect_Retry: 60 Master_Log_File: master-bin.000001 Read_Master_Log_Pos: 603 Relay_Log_File: relay-log-bin.000002 Relay_Log_Pos: 321 Relay_Master_Log_File: master-bin.000001 Slave_IO_Running: Yes Slave_SQL_Running: Yes ...#省略部份內容 上面的兩行表示從服務器給出的IO線程和數據庫SQL語句都在運行狀態 Auto_Position: 0 Replicate_Rewrite_DB: Channel_Name: Master_TLS_Version: 1 row in set (0.00 sec)
在主服務器上建立一個新的數據庫,新建表和數據;在服務器上進行查看;
主服務器:
create database test; Query OK, 1 row affected (0.00 sec)
從服務上slave1查詢:
mysql> show databases; +--------------------+ | Database | +--------------------+ | information_schema | | mysql | | performance_schema | | sys | | test | +--------------------+ 5 rows in set (0.00 sec) mysql> exit Bye [root@slave1 ~]#
從服務器slave2上查詢:
mysql> show databases; +--------------------+ | Database | +--------------------+ | information_schema | | mysql | | performance_schema | | sys | | test | +--------------------+ 5 rows in set (0.00 sec) mysql> exit Bye [root@slave2 ~]#
該配置實現的是從服務器經過主服務器給予的權限進行復制操做,咱們須要經過實驗配置理解上述的主從複製的原理以及其複製的過程。只有在理解掌握瞭如何進行MySQL數據庫的主從複製,才能理解和進行MySQL讀寫分離的配置操做。
讀寫分離的原理,簡單來講就是實現上圖中,在主服務器上寫數據,使用從服務器輪循讀取數據的功能。
咱們結合下圖來理解讀寫分離的過程
基於代理層實現:代理通常位於客戶端與服務器之間,代理服務器接到客戶端請求經過判斷後轉發到後端數據庫,有兩個表明性程序。
1)MySQL_Proxy:經過自帶的lua腳本進行SQL判斷;
2)Amoeba:Java語言開發,阿里巴巴將其用於生產環境,不支持事務與存儲過程;
環境:基於上面主從複製的流程進行後續陪配置
添加設備:Centos7兩臺:其中一臺做爲Amoeba代理服務器,另外一臺做爲客戶端測試服務器。
所需軟件包:在Amoeba代理服務器上須要安裝jdk和amoeba相關環境及應用(amoeba使用Java開發)
1)安裝jdk環境和ameoba環境
[root@amoeba ~]# cd tar/ [root@amoeba tar]# ls amoeba-mysql-binary-2.2.0.tar.gz jdk-6u14-linux-x64.bin
cd tar/ ls cp jdk-6u14-linux-x64.bin /usr/local/ cd /usr/local/ chmod +x jdk-6u14-linux-x64.bin ./jdk-6u14-linux-x64.bin mv jdk1.6.0_14/ /usr/local/jdk1.6 vim /etc/profile #添加以下幾行 export JAVA_HOME=/usr/local/jdk1.6 export CLASSPATH=$CLASSPATH:$JAVA_HOME/lib:$JAVA_HOME/jre/lib export PATH=$JAVA_HOME/lib:$JAVA_HOME/jre/bin/:$PATH:$HOME/bin export AMOEBA_HOME=/usr/local/amoeba export PATH=$PATH:$AMOEBA_HOME/bin source /etc/profile mkdir /usr/local/amoeba #回到壓縮包目錄下解壓amoeba tar zxf amoeba-mysql-binary-2.2.0.tar.gz -C /usr/local/amoeba cd /usr/local/
2)配置amoeba相關參數
配置路徑在/usr/local/amoeba/conf中,配置文件爲ameoba.xml,dbServers.xml。前者爲amoeba主配置文件(還記得tomcat嗎?),後者是數據庫服務器的相關配置文件。
一、更改ameoba.xml
30 <property name="user">amoeba</property> 31 32 <property name="password">123456</property> #user是咱們在client端登陸amoeba代理服務器的身份名稱,password是登陸密碼,待會咱們將會在client端上登陸 15 <property name="defaultPool">master</property> 116 117 <property name="writePool">master</property> 118 <property name="readPool">slaves</property> 119 #defaultPool表示默認的服務器 writePool表示指定寫服務器(組),readPool表示指定讀服務器(組),slaves將在dbServers.xml文件中配置
二、更改dbServers.xml
26 <property name="user">test</property> 27 28 <!-- mysql password --> 29 <property name="password">123123</property> 45 <dbServer name="master" parent="abstractServer"> 46 <factoryConfig> 47 <!-- mysql ip --> 48 <property name="ipAddress">192.168.68.133</property> 49 </factoryConfig> 50 </dbServer> 51 52 <dbServer name="slave1" parent="abstractServer"> 53 <factoryConfig> 54 <!-- mysql ip --> 55 <property name="ipAddress">192.168.68.129</property> 56 </factoryConfig> 57 </dbServer> 58 59 <dbServer name="slave2" parent="abstractServer"> 60 <factoryConfig> 61 <!-- mysql ip --> 62 <property name="ipAddress">192.168.68.132</property> 63 </factoryConfig> 64 </dbServer> 65 66 <dbServer name="slaves" virtual="true"> 67 <poolConfig class="com.meidusa.amoeba.server.MultipleServerPool"> 68 <!-- Load balancing strategy: 1=ROUNDROBIN , 2=WEIGHTBASED , 3=HA--> 69 <property name="loadbalance">1</property> 70 71 <!-- Separated by commas,such as: server1,server2,server1 --> 72 <property name="poolNames">slave1,slave2</property> 73 </poolConfig> 74 </dbServer>
三、開啓amoeba(能夠在另外一個terminal查看端口netstat -natp | grep 8066)
[root@amoeba bin]# amoeba start & [1] 121608 [root@amoeba bin]# log4j:WARN log4j config load completed from file:/usr/local/amoeba/conf/log4j.xml 2020-01-10 08:20:03,413 INFO context.MysqlRuntimeContext - Amoeba for Mysql current versoin=5.1.45-mysql-amoeba-proxy-2.2.0 log4j:WARN ip access config load completed from file:/usr/local/amoeba/conf/access_list.conf 2020-01-10 08:20:03,566 INFO net.ServerableConnectionManager - Amoeba for Mysql listening on 0.0.0.0/0.0.0.0:8066. 2020-01-10 08:20:03,567 INFO net.ServerableConnectionManager - Amoeba Monitor Server listening on /127.0.0.1:39466.
四、客戶機client端上安裝mysql數據庫,登陸查看
[root@localhost ~]# mysql -u amoeba -p123456 -h 192.168.68.144 -P8066 mysql: [Warning] Using a password on the command line interface can be insecure. Welcome to the MySQL monitor. Commands end with ; or \g. Your MySQL connection id is 2023306452 Server version: 5.1.45-mysql-amoeba-proxy-2.2.0 Source distribution Copyright (c) 2000, 2016, Oracle and/or its affiliates. All rights reserved. Oracle is a registered trademark of Oracle Corporation and/or its affiliates. Other names may be trademarks of their respective owners. Type 'help;' or '\h' for help. Type '\c' to clear the current input statement. mysql> show databases; +--------------------+ | Database | +--------------------+ | information_schema | | mysql | | performance_schema | | sys | | test | +--------------------+ 5 rows in set (0.00 sec) mysql> use test; Database changed mysql> show tables; Empty set (0.01 sec) #由於此時咱們尚未建立如何數據表和數據,因此在客戶端上是查看不到任何數據的。
五、咱們在主服務器上建立一個表而且在從服務器上查看是否存在這個表,存在則主從複製正常
主服務器:
mysql> use test; Database changed mysql> create table info (id int(5) not null primary key auto_increment,name varchar(10) not null); Query OK, 0 rows affected (0.02 sec) mysql> desc info; +-------+-------------+------+-----+---------+----------------+ | Field | Type | Null | Key | Default | Extra | +-------+-------------+------+-----+---------+----------------+ | id | int(5) | NO | PRI | NULL | auto_increment | | name | varchar(10) | NO | | NULL | | +-------+-------------+------+-----+---------+----------------+ 2 rows in set (0.01 sec) mysql> select * from info; Empty set (0.00 sec)
從服務器:
[root@slave1 ~]# mysql -uroot -p Enter password: Welcome to the MySQL monitor. Commands end with ; or \g. Your MySQL connection id is 14 Server version: 5.7.17-log Source distribution Copyright (c) 2000, 2016, Oracle and/or its affiliates. All rights reserved. Oracle is a registered trademark of Oracle Corporation and/or its affiliates. Other names may be trademarks of their respective owners. Type 'help;' or '\h' for help. Type '\c' to clear the current input statement. mysql> show databases; +--------------------+ | Database | +--------------------+ | information_schema | | mysql | | performance_schema | | sys | | test | +--------------------+ 5 rows in set (0.01 sec) mysql> use test Reading table information for completion of table and column names You can turn off this feature to get a quicker startup with -A Database changed mysql> show tables; +----------------+ | Tables_in_test | +----------------+ | info | +----------------+ 1 row in set (0.00 sec)
六、在client端上查看這個表
mysql> show tables; Empty set (0.01 sec) mysql> show tables; +----------------+ | Tables_in_test | +----------------+ | info | +----------------+ 1 row in set (0.00 sec)
七、那麼咱們如何驗證讀寫分離呢?還記得咱們在從服務器上的start slave命令嗎?咱們能夠中止主從複製,使用stop slave便可
mysql> stop slave; Query OK, 0 rows affected (0.01 sec)
八、咱們在client端插入新的數據,分別在主從服務器上進行查看
client端:
mysql> insert into info (name) values('zhangsan'); Query OK, 1 row affected (0.01 sec) mysql> select * from info; +----+----------+ | id | name | +----+----------+ | 2 | zhangsan | +----+----------+ 1 row in set (0.00 sec)
主服務器:
mysql> select * from info; +----+----------+ | id | name | +----+----------+ | 2 | zhangsan | +----+----------+ 1 row in set (0.00 sec)
從服務器:
mysql> select * from info; Empty set (0.00 sec)
從以上的實驗中,咱們能夠得出結論,通過主從複製與寫分離的配置,咱們能夠將寫數據的任務在主服務器上進行,主服務器寫入完成後,由從服務器通過主從複製的過程進行(start slave)複製,由於在生產環境中咱們不能stop slave(中止主從複製)的!!!。可是爲了驗證讀寫分離原理,咱們只能先stop slave了。
下面驗證讀分離。
咱們在從服務器上分別寫入一些數據(兩臺從服務器上寫的不同)
slave1:
mysql> insert into info (name) values('lisi'); Query OK, 1 row affected (0.00 sec) mysql> select * from info; +----+------+ | id | name | +----+------+ | 1 | lisi | +----+------+ 1 row in set (0.00 sec)
slave2:
mysql> insert into info (name) values('wangwu'); Query OK, 1 row affected (0.00 sec) mysql> select * from info; +----+--------+ | id | name | +----+--------+ | 1 | wangwu | +----+--------+ 1 row in set (0.00 sec)
client 查詢:(可能會有問題,由於是採用了auto_increment自增列)
所以咱們在從服務器上開啓複製功能,在主從服務器上修改字段,而後在從服務器上關閉複製功能,而且從新寫入不一樣的數據,在client端查看。
兩臺從服務器:
mysql> start slave; Query OK, 0 rows affected (0.01 sec)
主服務器:
mysql> alter table info modify id int(5) not null; Query OK, 1 row affected (0.02 sec) Records: 1 Duplicates: 0 Warnings: 0 mysql> desc info; +-------+-------------+------+-----+---------+-------+ | Field | Type | Null | Key | Default | Extra | +-------+-------------+------+-----+---------+-------+ | id | int(5) | NO | PRI | NULL | | | name | varchar(10) | NO | | NULL | | +-------+-------------+------+-----+---------+-------+ 2 rows in set (0.00 sec) mysql> delete from info; Query OK, 1 row affected (0.00 sec)
從服務器:
mysql> alter table info modify id int(5) not null; Query OK, 1 row affected (0.02 sec) Records: 1 Duplicates: 0 Warnings: 0 mysql> desc info; +-------+-------------+------+-----+---------+-------+ | Field | Type | Null | Key | Default | Extra | +-------+-------------+------+-----+---------+-------+ | id | int(5) | NO | PRI | NULL | | | name | varchar(10) | NO | | NULL | | +-------+-------------+------+-----+---------+-------+ 2 rows in set (0.00 sec) mysql> delete from info; Query OK, 1 row affected (0.00 sec)
client :從新寫入數據
mysql> insert into info (id,name) values(1,'zhangsan'); Query OK, 1 row affected (0.01 sec)
主服務器:
mysql> select * from info; +----+----------+ | id | name | +----+----------+ | 1 | zhangsan | +----+----------+ 1 row in set (0.00 sec) #此時須要從新查看狀態記錄position,此時以下: mysql> show master status; +-------------------+----------+--------------+------------------+-------------------+ | File | Position | Binlog_Do_DB | Binlog_Ignore_DB | Executed_Gtid_Set | +-------------------+----------+--------------+------------------+-------------------+ | master-bin.000001 | 2955 | | | | +-------------------+----------+--------------+------------------+-------------------+ 1 row in set (0.00 sec)
從服務器:
mysql> change master to master_host='192.168.68.133',master_user='myslave',master_password='123456',master_log_file='master-bin.000001',master_log_pos=2955; ERROR 3021 (HY000): This operation cannot be performed with a running slave io thread; run STOP SLAVE IO_THREAD FOR CHANNEL '' first. mysql> STOP SLAVE IO_THREAD FOR CHANNEL ''; Query OK, 0 rows affected (0.00 sec) mysql> stop slave; Query OK, 0 rows affected, 1 warning (0.00 sec) mysql> change master to master_host='192.168.68.133',master_user='myslave',master_password='123456',master_log_file='master-bin.000001',master_log_pos=2955; Query OK, 0 rows affected, 2 warnings (0.01 sec) mysql> start slave; Query OK, 0 rows affected (0.00 sec) mysql> show slave status\G *************************** 1. row *************************** Slave_IO_State: Waiting for master to send event Master_Host: 192.168.68.133 Master_User: myslave Master_Port: 3306 Connect_Retry: 60 Master_Log_File: master-bin.000001 Read_Master_Log_Pos: 2955 Relay_Log_File: relay-log-bin.000002 Relay_Log_Pos: 321 Relay_Master_Log_File: master-bin.000001 Slave_IO_Running: Yes Slave_SQL_Running: Yes Replicate_Do_DB: Replicate_Ignore_DB: Replicate_Do_Table: Replicate_Ignore_Table: Replicate_Wild_Do_Table: Replicate_Wild_Ignore_Table: Last_Errno: 0 Last_Error: Skip_Counter: 0 Exec_Master_Log_Pos: 2955 Relay_Log_Space: 526 Until_Condition: None Until_Log_File: Until_Log_Pos: 0 Master_SSL_Allowed: No Master_SSL_CA_File: Master_SSL_CA_Path: Master_SSL_Cert: Master_SSL_Cipher: Master_SSL_Key: Seconds_Behind_Master: 0 Master_SSL_Verify_Server_Cert: No Last_IO_Errno: 0 Last_IO_Error: Last_SQL_Errno: 0 Last_SQL_Error: Replicate_Ignore_Server_Ids: Master_Server_Id: 1 Master_UUID: 5bb93767-328a-11ea-820a-000c290bd936 Master_Info_File: /usr/local/mysql/data/master.info SQL_Delay: 0 SQL_Remaining_Delay: NULL Slave_SQL_Running_State: Slave has read all relay log; waiting for more updates Master_Retry_Count: 86400 Master_Bind: Last_IO_Error_Timestamp: Last_SQL_Error_Timestamp: Master_SSL_Crl: Master_SSL_Crlpath: Retrieved_Gtid_Set: Executed_Gtid_Set: Auto_Position: 0 Replicate_Rewrite_DB: Channel_Name: Master_TLS_Version: 1 row in set (0.00 sec)
算了,咱們將三個服務器上的數據表的數據都刪了從新進行主從複製吧;按照主從複製執行以後繼續進行讀分離的操做;
關閉從服務器主從複製:
mysql> stop slave; Query OK, 0 rows affected (0.00 sec)
從服務器上寫入不一樣數據:
slave1:
mysql> insert into info values(2,'lisi'); Query OK, 1 row affected (0.00 sec) mysql> select * from info; +----+----------+ | id | name | +----+----------+ | 1 | zhangsan | | 2 | lisi | +----+----------+ 2 rows in set (0.00 sec)
slave2:
mysql> insert into info values(3,'wangwu'); Query OK, 1 row affected (0.00 sec) mysql> select * from info; +----+----------+ | id | name | +----+----------+ | 1 | zhangsan | | 3 | wangwu | +----+----------+ 2 rows in set (0.00 sec)
client:
mysql> select * from info; +----+----------+ | id | name | +----+----------+ | 1 | zhangsan | | 2 | lisi | +----+----------+ 2 rows in set (0.00 sec) mysql> select * from info; +----+----------+ | id | name | +----+----------+ | 1 | zhangsan | | 3 | wangwu | +----+----------+ 2 rows in set (0.00 sec)
MySQL讀分離則是在MySQL的從服務器上進行數據讀取操做。其中兩臺作負載均衡,輪循讀取,減輕壓力,提升併發訪問。
本文主要介紹了MySQL主從複製與讀寫分離原理,以及如何配置和驗證主從複製與讀寫分離的實驗。
經過實驗咱們能夠得出如下結論:在生產環境中,咱們不會將從服務器中止複製同步功能,而且不會在從服務上寫數據,由於這樣會致使某些問題,若是在從服務器上寫了與主服務器衝突的數據,主服務器不會同步從的,主服務器上與從服務器所衝突的數據不會同步給從服務器。