MySQL/MariaDB數據庫的主從複製 html
做者:尹正傑 前端
版權聲明:原創做品,謝絕轉載!不然將追究法律責任。node
一.MySQL複製概述mysql
1>.傳統擴展方式sql
垂直擴展(也叫向上擴展,Sacle Up): 在單節點經過提高服務器硬件性能以達到提高軟件運行效率的問題,好比提高CPU,存儲,內存,網絡的性能。 在不少企業裏面,MySQL數據庫存放了一些重要的數據,當你的企業服務訪問量愈來愈大,單個服務器性能以及很難達到要求了,一個單服務器的性能終究有限,一般在生產環境中一個數據庫它的併發鏈接數能達到2000-3000已經很了不得了,在往上就撐不住了。 基於目前工藝條件所限,有可能單個服務器成本增大10倍,軟件的性能才提高2倍。
水平擴展(也叫橫向擴展,Scale Out):
增長多個相同配置的服務器對外提供服務,擴展起來相對簡單,好比如今的分佈式存儲系統,集羣等。
MySQL併發鏈接數有的時候並非和服務器硬件有關的,而是MySQL數據庫自己的性能就有極限。所以,當訪問量達到必定規模的時候,單個數據庫服務器性能壓根扛不住。
目前基於工藝,成本考慮,可能更多的會考慮使用水平擴展。
2>.MySQL複製介紹數據庫
(1)MySQL複製容許將主實例(master)上的數據同步到一個或多個從實例(slave)上,默認狀況下複製是異步進行的,從庫也不須要一直鏈接主庫來同步數據; (2)MySQL複製的數據粒度能夠是主實例上全部的數據庫,也能夠是指定的一個或多個數據庫,也能夠是一個數據庫裏的指定的表; (3)MySQL複製有兩種方法 傳統方式 基於主庫的bin-log將日誌事件和事件位置複製到從庫,從庫再加以應用來達到主從同步的目的。 Gtid方式 global transaction identitifiers 是基於事物來複制數據,所以也就不依賴日誌文件,同時又能更好的保證主從庫數據一致性。
3>.MySQL複製帶來的優點vim
擴展能力(讀寫分離,數據分佈均勻以提高性能):
經過複製能夠將MySQL的讀寫分到一個或多個slave上。這要求全部的寫操做和修改操做都必須在Master上完成,而讀操做能夠被分配到一個或多個salve上。將讀寫分離到不一樣服務器執行以後,MySQL的讀寫性能獲得提高。
數據庫備份:
因爲從實例(slave)實時同步主實例(master)的數據,所以主實例(master)節點壓力過大,此時能夠將備份做業部署到從庫。
須要注意的是,雖然主從複製可以起到必定程度上的備份,但並不能徹底代替備份工做(通常MySQL集羣都在同機房,建議作好異地備份,若該機房較長時間沒法恢復通訊,能夠啓用備份數據)。
數據分析和報表:
一樣,一些數據分析和報表的實現能夠在從實例執行,以減小對主庫的性能影響。
容災能力:
能夠在物理距離較遠的另外一個數據創建slave,保證在主實例所在地區遭遇災難時,在另外一個數據中心能快速恢復。
高可用和故障切換:
經過MHA技術實現當master節點不正常工做了,其它slave節點能夠自動選舉出新的節點成爲master,從而達到對master節點的備份效果。
MySQL升級測試:
在一個正常工做的MySQL集羣中,若要對一個集羣升級,可先下線一臺SLAVE MySQL服務器,該節點下線並不影響對外提供服務,等服務器升級完畢後從新上線,等待該節點運行正常後,陸續下線其它節點並升級,整個過程並不影響MySQL集羣的工做從而實現了平滑升級。
4>. MySQL複製有多種類型後端
異步複製 一個主庫,一個或多個從庫,數據異步同步到從庫。 同步複製 在MySQL cluster中特有的複製方式。 半同步複製 在異步複製的基礎上,確保任何一個主庫上的事物在提交以前至少有一個從庫已經收到該事物並日志記錄下來。 延遲複製 在異步複製的基礎上,人爲設定主庫和從庫的數據同步延遲時間,即保證數據延遲至少是這個參數。
二.MySQL複製的常見架構服務器
1>.一主一從網絡
以下圖所示:
用戶向App Server服務器發送請求,App Server判斷用戶發來的請求是讀仍是寫,若是是讀就將SQL發送給slave節點響應,若是是寫就將SQL發送給master節點響應,當master寫完數據後將修改後的數據同步到slave節點已達到兩個節點數據一致性的目的。
2>.一主多從
以下圖所示:
其實架構和一主一從相似,只不過當master節點完成寫操做後,會將數據同時同步到其它多個slave節點。
3>.MySQL垂直分區
主從複製基本上可以解決中小型企業的需求啦,但在不少場景下依然會遇到瓶頸問題,好比典型的雙十一,淘寶不少用戶下訂單這些數據都會被記錄到數據庫,別說單臺Mysql抗不住了,你直接使用單臺Oracle數據庫也未必能抗住啊。
由於主從複製架構中,只有master節點是負責寫的,有的時候master節點寫操做壓力很大,性能也倒不倒要求了該怎麼辦?
典型的解決辦法就是分庫,分表,將數據拆開到不一樣節點分佈式集羣。而這個拆分方法就是咱們所說的垂直分區和水平分片。
垂直拆分優勢:
所謂垂直拆分指的是將同一個數據庫的多張表分別存儲到多臺MySQL實例上存儲;
原本同一個數據庫的內容由以前1臺服務器提供服務到如今的3臺服務器提供服務,從而性能達到提高。
垂直拆分缺點:
若同一個數據庫多張表在同一個實例上是能夠進行一些內鏈接,外鏈接,關聯查詢等等,但因爲咱們將同一個數據庫多張表分開存儲就會致使咱們沒法執行各類join語句。
所以咱們在拆分表的時候必定要考慮清除這些表是否有關係,是否會用到join查詢,固然這個得和開發人員協商,由於表結構是它們設計的,哪些表有聯繫哪些表沒有聯繫它們會更清楚。
4>.MySQL水平分片(Sharding)
以下圖所示:
有"其它","用戶","消息"這3張表,咱們能夠根據用戶的id將表的內容進行分片處理,將奇數的用戶數據從這3張表拆出來放在一個MySQL服務器上存儲,把偶數的用戶從這3張表中拆出來放在另一臺MySQL服務器上。
這也意味着每張大表都被拆分紅2個小表各自放在不一樣的數據庫上啦。
上面知識舉個例子使用用戶id來拆分表數據,固然我們能夠根據實際業務來進行從拆分,好比根據省份,用戶積分等來拆分等。
以下圖所示:
當user 103用戶想要從網頁查詢他的數據,網頁會經過前端傳參給後端,後端經過ORM等技術轉換成相應的SQL語句來查詢數據庫信息。
這個時候咱們怎麼知道user 103的數據存放在那個數據庫實例上呢?
所以須要有一個分配管理器(Sharding Manager)來記錄用戶的數據在哪一臺服務器上存放,至關於要增長一個相似調度器的東西。
雖說數據庫的性能獲得提高,可是能夠明顯體會到系統的複雜度也提升了,須要加入更多的節點,此時任何一個節點出故障都會影響原來業務的使用,好比分片管理器不正常工做的話,儘管你數據庫再多也沒用,由於此時壓根查不到數據在哪臺節點上存放的,所以必需要想辦法解決分配管理器的高可用性。這就得引入新的技術來實現,好比keepalive等技術。
三.MySQL主從複製原理
以下圖所示:
1>.master節點收到寫操做,經過用戶權限驗證後請求更新數據信息;
2>.若是是Innodb存儲引擎會先寫事務日誌,寫完事務日誌之後再寫數據文件,生成數據文件的同時會記錄到二進制日誌中,而Myisam壓根就不支持事務,所以當接收到寫請求時會直接修改數據從而在二進制日誌中記錄;
3>.master節點經過dump thread將生成的二進制日誌讀取出來並經過網絡往從服務器發送;
4>.slave節點經過io thread將master的dump thread發送過來的數據接收並臨時寫入一箇中繼日誌(Relay log)文件;
5>.slave節點經過SQL thread讀取Relay log中的信息開始更新當前數據庫數據信息。
1>.主從複製線程
主節點: dump Thread:
爲每一個Slave的I/O Thread啓動一個dump線程,用於向其發送binary log events 從節點: I/O Thread:
向Master請求二進制日誌事件,並保存於中繼日誌中 SQL Thread:
從中繼日誌中讀取日誌事件,在本地完成重放
2>.跟複製功能相關的文件
master.info:
用於保存slave鏈接至master時的相關信息,例如帳號、密碼、服務器地址等 relay-log.info:
保存在當前slave節點上已經複製的當前二進制日誌和本地replay log日誌的對應關係
3>.主從複製特色
異步複製
主從數據不一致比較常見
4>.複製架構
Master/Slave
Master/Master
環狀複製
一主多從
從服務器還能夠再有從服務器
一從多主:適用於多個不一樣數據庫
5>.複製須要考慮二進制日誌事件記錄格式
基於語句的複製(statement based replication,5.0以前默認使用): 基於主庫將SQL語句寫入到bin log中完成複製。 基於行數據的複製(row based replication,5.1以後推薦使用該模式): 基於主庫將每一行數據變化的信息做爲時間寫入到bin log中完成日誌。默認就是基於行級別的複製,由於它相對語句複製邏輯更爲嚴謹。 混合複製(mixed based replication): 上述二者的結合。默認狀況下優先使用基於語句的複製,只有當部分語句若是基於語句複製不徹底的狀況下才會自動切換爲基於行數據的複製。
四.典型主從複製案例(未投入生產環境數據庫,即2個數據庫均爲空)
1>.主從配置過程
博主推薦閱讀: https://mariadb.com/kb/en/library/setting-up-replication/ https://dev.mysql.com/doc/refman/5.7/en/replication-configuration.html
2>.主節點配置
[root@node102.yinzhengjie.org.cn ~]# cat /etc/my.cnf [mysqld] binlog_format = row #推薦使用基於行數據的複製 log_bin = /data/logbin/mysql-bin #須要啓用二進制日誌,建議和數據文件分開放存放 server-id = 102 #爲當前節點設置一個全局唯一的ID號,用於標識當前MySQL實例 log-basename = master #可選項,設置datadir中日誌名稱,確保不依賴主機名 character-set-server = utf8mb4 default_storage_engine = InnoDB datadir = /var/lib/mysql socket = /var/lib/mysql/mysql.sock [mysqld_safe] log-error = /var/log/mariadb/mariadb.log pid-file = /var/run/mariadb/mariadb.pid !includedir /etc/my.cnf.d [root@node102.yinzhengjie.org.cn ~]#
[root@node102.yinzhengjie.org.cn ~]# mysql -uroot -pyinzhengjie Welcome to the MariaDB monitor. Commands end with ; or \g. Your MariaDB connection id is 6 Server version: 5.5.64-MariaDB MariaDB Server Copyright (c) 2000, 2018, Oracle, MariaDB Corporation Ab and others. Type 'help;' or '\h' for help. Type '\c' to clear the current input statement. MariaDB [(none)]> MariaDB [(none)]> SELECT user,host,password FROM mysql.user; +------+-----------+-------------------------------------------+ | user | host | password | +------+-----------+-------------------------------------------+ | root | localhost | *BD0B1F48FDC55BD27555FC2F22FF29A68A25A1D7 | | root | 127.0.0.1 | *BD0B1F48FDC55BD27555FC2F22FF29A68A25A1D7 | | root | ::1 | *BD0B1F48FDC55BD27555FC2F22FF29A68A25A1D7 | +------+-----------+-------------------------------------------+ 3 rows in set (0.00 sec) MariaDB [(none)]> MariaDB [(none)]> GRANT REPLICATION SLAVE ON *.* TO 'copy'@'172.30.1.10%' IDENTIFIED BY 'yinzhengjie'; Query OK, 0 rows affected (0.00 sec) MariaDB [(none)]> MariaDB [(none)]> SELECT user,host,password FROM mysql.user; +------+--------------+-------------------------------------------+ | user | host | password | +------+--------------+-------------------------------------------+ | root | localhost | *BD0B1F48FDC55BD27555FC2F22FF29A68A25A1D7 | | root | 127.0.0.1 | *BD0B1F48FDC55BD27555FC2F22FF29A68A25A1D7 | | root | ::1 | *BD0B1F48FDC55BD27555FC2F22FF29A68A25A1D7 | | copy | 172.30.1.10% | *BD0B1F48FDC55BD27555FC2F22FF29A68A25A1D7 | +------+--------------+-------------------------------------------+ 4 rows in set (0.00 sec) MariaDB [(none)]>
MariaDB [(none)]> SHOW MASTER LOGS; #查看主節點的二進制日誌當前所在位置 +-------------------+-----------+ | Log_name | File_size | +-------------------+-----------+ | master-bin.000001 | 245 | +-------------------+-----------+ 1 row in set (0.00 sec) MariaDB [(none)]>
3>.從節點配置
[root@node103.yinzhengjie.org.cn ~]# cat /etc/my.cnf [mysqld] server_id = 103 #爲當前節點設置一個全局唯的ID號 read_only = ON #設置數據庫只讀 relay_log = relay-log #relay log的文件路徑,默認值hostname-relay-bin relay_log_index = relay-log.index #默認值hostname-relay-bin.index datadir = /var/lib/mysql socket = /var/lib/mysql/mysql.sock [mysqld_safe] log-error=/var/log/mariadb/mariadb.log pid-file=/var/run/mariadb/mariadb.pid [root@node103.yinzhengjie.org.cn ~]#
[root@node103.yinzhengjie.org.cn ~]# mysql Welcome to the MariaDB monitor. Commands end with ; or \g. Your MariaDB connection id is 2 Server version: 5.5.64-MariaDB MariaDB Server Copyright (c) 2000, 2018, Oracle, MariaDB Corporation Ab and others. Type 'help;' or '\h' for help. Type '\c' to clear the current input statement. MariaDB [(none)]> MariaDB [(none)]> CHANGE MASTER TO -> MASTER_HOST='172.30.1.102', -> MASTER_USER='copy', -> MASTER_PASSWORD='yinzhengjie', -> MASTER_PORT=3306, -> MASTER_LOG_FILE='master-bin.000001', -> MASTER_LOG_POS=245, -> MASTER_CONNECT_RETRY=10; Query OK, 0 rows affected (0.00 sec) MariaDB [(none)]> MariaDB [(none)]>
MariaDB [(none)]> SHOW SLAVE STATUS\G *************************** 1. row *************************** Slave_IO_State: Master_Host: 172.30.1.102 Master_User: copy Master_Port: 3306 Connect_Retry: 10 Master_Log_File: master-bin.000001 Read_Master_Log_Pos: 245 Relay_Log_File: relay-log.000001 Relay_Log_Pos: 4 Relay_Master_Log_File: master-bin.000001 Slave_IO_Running: No Slave_SQL_Running: No Replicate_Do_DB: Replicate_Ignore_DB: Replicate_Do_Table: Replicate_Ignore_Table: Replicate_Wild_Do_Table: Replicate_Wild_Ignore_Table: Last_Errno: 0 Last_Error: Skip_Counter: 0 Exec_Master_Log_Pos: 245 Relay_Log_Space: 245 Until_Condition: None Until_Log_File: Until_Log_Pos: 0 Master_SSL_Allowed: No Master_SSL_CA_File: Master_SSL_CA_Path: Master_SSL_Cert: Master_SSL_Cipher: Master_SSL_Key: Seconds_Behind_Master: NULL Master_SSL_Verify_Server_Cert: No Last_IO_Errno: 0 Last_IO_Error: Last_SQL_Errno: 0 Last_SQL_Error: Replicate_Ignore_Server_Ids: Master_Server_Id: 0 1 row in set (0.00 sec) MariaDB [(none)]> MariaDB [(none)]> SHOW PROCESSLIST; #未啓動複製線程時,咱們查看線程時看不到複製的IO線程的,等開啓時迅速查看會有相應進程出現。 +----+------+-----------+------+---------+------+-------+------------------+----------+ | Id | User | Host | db | Command | Time | State | Info | Progress | +----+------+-----------+------+---------+------+-------+------------------+----------+ | 2 | root | localhost | NULL | Sleep | 69 | | NULL | 0.000 | | 3 | root | localhost | NULL | Query | 0 | NULL | SHOW PROCESSLIST | 0.000 | +----+------+-----------+------+---------+------+-------+------------------+----------+ 2 rows in set (0.00 sec) MariaDB [(none)]>
MariaDB [(none)]> SHOW SLAVE STATUS\G *************************** 1. row *************************** Slave_IO_State: Master_Host: 172.30.1.102 Master_User: copy Master_Port: 3306 Connect_Retry: 10 Master_Log_File: master-bin.000001 Read_Master_Log_Pos: 245 Relay_Log_File: relay-log.000001 Relay_Log_Pos: 4 Relay_Master_Log_File: master-bin.000001 Slave_IO_Running: No Slave_SQL_Running: No Replicate_Do_DB: Replicate_Ignore_DB: Replicate_Do_Table: Replicate_Ignore_Table: Replicate_Wild_Do_Table: Replicate_Wild_Ignore_Table: Last_Errno: 0 Last_Error: Skip_Counter: 0 Exec_Master_Log_Pos: 245 Relay_Log_Space: 245 Until_Condition: None Until_Log_File: Until_Log_Pos: 0 Master_SSL_Allowed: No Master_SSL_CA_File: Master_SSL_CA_Path: Master_SSL_Cert: Master_SSL_Cipher: Master_SSL_Key: Seconds_Behind_Master: NULL Master_SSL_Verify_Server_Cert: No Last_IO_Errno: 0 Last_IO_Error: Last_SQL_Errno: 0 Last_SQL_Error: Replicate_Ignore_Server_Ids: Master_Server_Id: 0 1 row in set (0.00 sec) MariaDB [(none)]> MariaDB [(none)]> START SLAVE; #啓動複製線程 Query OK, 0 rows affected (0.00 sec) MariaDB [(none)]> MariaDB [(none)]> MariaDB [(none)]> SHOW SLAVE STATUS\G *************************** 1. row *************************** Slave_IO_State: Waiting for master to send event Master_Host: 172.30.1.102 Master_User: copy Master_Port: 3306 Connect_Retry: 10 Master_Log_File: master-bin.000001 Read_Master_Log_Pos: 245 Relay_Log_File: relay-log.000002 Relay_Log_Pos: 530 Relay_Master_Log_File: master-bin.000001 Slave_IO_Running: Yes #此處說明IO線程啓動成功 Slave_SQL_Running: Yes #此處爲yes說明SQL線程啓動成功 Replicate_Do_DB: Replicate_Ignore_DB: Replicate_Do_Table: Replicate_Ignore_Table: Replicate_Wild_Do_Table: Replicate_Wild_Ignore_Table: Last_Errno: 0 Last_Error: Skip_Counter: 0 Exec_Master_Log_Pos: 245 Relay_Log_Space: 818 Until_Condition: None Until_Log_File: Until_Log_Pos: 0 Master_SSL_Allowed: No Master_SSL_CA_File: Master_SSL_CA_Path: Master_SSL_Cert: Master_SSL_Cipher: Master_SSL_Key: Seconds_Behind_Master: 0 #該參數表示延遲的秒數 Master_SSL_Verify_Server_Cert: No Last_IO_Errno: 0 Last_IO_Error: Last_SQL_Errno: 0 Last_SQL_Error: Replicate_Ignore_Server_Ids: Master_Server_Id: 102 1 row in set (0.00 sec) MariaDB [(none)]> MariaDB [(none)]> SHOW PROCESSLIST\G *************************** 1. row *************************** Id: 2 User: root Host: localhost db: NULL Command: Sleep Time: 369 State: Info: NULL Progress: 0.000 *************************** 2. row *************************** Id: 3 User: root Host: localhost db: NULL Command: Query Time: 0 State: NULL Info: SHOW PROCESSLIST Progress: 0.000 *************************** 3. row *************************** Id: 4 User: system user Host: db: NULL Command: Connect Time: 540 State: Waiting for master to send event Info: NULL Progress: 0.000 *************************** 4. row *************************** Id: 5 User: system user Host: db: NULL Command: Connect Time: 540 State: Slave has read all relay log; waiting for the slave I/O thread to update it Info: NULL Progress: 0.000 4 rows in set (0.00 sec) MariaDB [(none)]>
4>.驗證數據是否同步成功
[root@node103.yinzhengjie.org.cn ~]# mysql -e "SHOW DATABASES" #查看從庫現有數據庫 +--------------------+ | Database | +--------------------+ | information_schema | | mysql | | performance_schema | | test | +--------------------+ [root@node103.yinzhengjie.org.cn ~]#
[root@node102.yinzhengjie.org.cn ~]# mysql -uroot -pyinzhengjie Welcome to the MariaDB monitor. Commands end with ; or \g. Your MariaDB connection id is 6 Server version: 5.5.64-MariaDB MariaDB Server Copyright (c) 2000, 2018, Oracle, MariaDB Corporation Ab and others. Type 'help;' or '\h' for help. Type '\c' to clear the current input statement. MariaDB [(none)]> MariaDB [(none)]> CREATE DATABASE devops; Query OK, 1 row affected (0.00 sec) MariaDB [(none)]> MariaDB [(none)]> USE devops Database changed MariaDB [devops]> MariaDB [devops]> CREATE SCHEMA IF NOT EXISTS yinzhengjie2019 DEFAULT CHARACTER SET = utf8mb4; Query OK, 1 row affected (0.00 sec) MariaDB [devops]> MariaDB [devops]> CREATE TABLE students(id INT UNSIGNED AUTO_INCREMENT PRIMARY KEY,name VARCHAR(30) NOT NULL,sex ENUM('boy','girl') DEFAULT 'boy' ,age TINYINT UNSIGNED,mobile CHAR(11),address VARCHAR(50));Query OK, 0 rows affected (0.01 sec) MariaDB [devops]> INSERT INTO students (name,age,mobile,address) VALUES ('Jason Yin',18,10000,'beijing'),('Jay','40',10086,'Taiwan'); Query OK, 2 rows affected (0.00 sec) Records: 2 Duplicates: 0 Warnings: 0 MariaDB [devops]> INSERT INTO students SET name='yinzhengjie',age=27,address='shanxi'; Query OK, 1 row affected (0.01 sec) MariaDB [devops]> INSERT students (age,sex,name,mobile,address) VALUES (28,'girl','Gloria Tang Tsz-Kei',null,'Hong Kong'); Query OK, 1 row affected (0.00 sec) MariaDB [devops]> CREATE TABLE employee SELECT * FROM students; Query OK, 4 rows affected (0.00 sec) Records: 4 Duplicates: 0 Warnings: 0 MariaDB [devops]> MariaDB [devops]> SHOW TABLES; +------------------+ | Tables_in_devops | +------------------+ | employee | | students | +------------------+ 2 rows in set (0.00 sec) MariaDB [devops]> MariaDB [devops]> SELECT * FROM students; +----+---------------------+------+------+--------+-----------+ | id | name | sex | age | mobile | address | +----+---------------------+------+------+--------+-----------+ | 1 | Jason Yin | boy | 18 | 10000 | beijing | | 2 | Jay | boy | 40 | 10086 | Taiwan | | 3 | yinzhengjie | boy | 27 | NULL | shanxi | | 4 | Gloria Tang Tsz-Kei | girl | 28 | NULL | Hong Kong | +----+---------------------+------+------+--------+-----------+ 4 rows in set (0.00 sec) MariaDB [devops]> MariaDB [devops]> SELECT * FROM employee; +----+---------------------+------+------+--------+-----------+ | id | name | sex | age | mobile | address | +----+---------------------+------+------+--------+-----------+ | 1 | Jason Yin | boy | 18 | 10000 | beijing | | 2 | Jay | boy | 40 | 10086 | Taiwan | | 3 | yinzhengjie | boy | 27 | NULL | shanxi | | 4 | Gloria Tang Tsz-Kei | girl | 28 | NULL | Hong Kong | +----+---------------------+------+------+--------+-----------+ 4 rows in set (0.00 sec) MariaDB [devops]> MariaDB [devops]> SHOW DATABASES; +--------------------+ | Database | +--------------------+ | information_schema | | devops | | mysql | | performance_schema | | test | | yinzhengjie2019 | +--------------------+ 6 rows in set (0.00 sec) MariaDB [devops]> MariaDB [devops]> QUIT Bye [root@node102.yinzhengjie.org.cn ~]# [root@node102.yinzhengjie.org.cn ~]#
[root@node103.yinzhengjie.org.cn ~]# mysql -e "SHOW DATABASES" +--------------------+ | Database | +--------------------+ | information_schema | | devops | | mysql | | performance_schema | | test | | yinzhengjie2019 | +--------------------+ [root@node103.yinzhengjie.org.cn ~]# [root@node103.yinzhengjie.org.cn ~]# mysql -e "USE devops;SHOW TABLES" +------------------+ | Tables_in_devops | +------------------+ | employee | | students | +------------------+ [root@node103.yinzhengjie.org.cn ~]# [root@node103.yinzhengjie.org.cn ~]# mysql -e "USE devops;SELECT * FROM employee" +----+---------------------+------+------+--------+-----------+ | id | name | sex | age | mobile | address | +----+---------------------+------+------+--------+-----------+ | 1 | Jason Yin | boy | 18 | 10000 | beijing | | 2 | Jay | boy | 40 | 10086 | Taiwan | | 3 | yinzhengjie | boy | 27 | NULL | shanxi | | 4 | Gloria Tang Tsz-Kei | girl | 28 | NULL | Hong Kong | +----+---------------------+------+------+--------+-----------+ [root@node103.yinzhengjie.org.cn ~]# [root@node103.yinzhengjie.org.cn ~]# mysql -e "USE devops;SELECT * FROM students" +----+---------------------+------+------+--------+-----------+ | id | name | sex | age | mobile | address | +----+---------------------+------+------+--------+-----------+ | 1 | Jason Yin | boy | 18 | 10000 | beijing | | 2 | Jay | boy | 40 | 10086 | Taiwan | | 3 | yinzhengjie | boy | 27 | NULL | shanxi | | 4 | Gloria Tang Tsz-Kei | girl | 28 | NULL | Hong Kong | +----+---------------------+------+------+--------+-----------+ [root@node103.yinzhengjie.org.cn ~]#
[root@node102.yinzhengjie.org.cn ~]# mysql -uroot -pyinzhengjie Welcome to the MariaDB monitor. Commands end with ; or \g. Your MariaDB connection id is 7 Server version: 5.5.64-MariaDB MariaDB Server Copyright (c) 2000, 2018, Oracle, MariaDB Corporation Ab and others. Type 'help;' or '\h' for help. Type '\c' to clear the current input statement. MariaDB [(none)]> use yinzhengjie2019 Database changed MariaDB [yinzhengjie2019]> MariaDB [yinzhengjie2019]> CREATE TABLE testlog (id INT auto_increment PRIMARY KEY,name CHAR(50),age INT DEFAULT 20); Query OK, 0 rows affected (0.01 sec) MariaDB [yinzhengjie2019]> MariaDB [yinzhengjie2019]> DELIMITER $$ MariaDB [yinzhengjie2019]> MariaDB [yinzhengjie2019]> CREATE PROCEDURE pro_testlog() -> BEGIN -> DECLARE i INT; -> SET i = 1; -> WHILE i < 100000 -> DO -> INSERT INTO testlog(name,age) VALUES (CONCAT('yinzhengjie',i),i); -> SET i = i +1; -> END WHILE; -> END$$ Query OK, 0 rows affected (0.00 sec) MariaDB [yinzhengjie2019]> MariaDB [yinzhengjie2019]> DELIMITER ; MariaDB [yinzhengjie2019]> MariaDB [yinzhengjie2019]> SHOW PROCEDURE STATUS\G *************************** 1. row *************************** Db: yinzhengjie2019 Name: pro_testlog Type: PROCEDURE Definer: root@localhost Modified: 2019-11-08 12:04:06 Created: 2019-11-08 12:04:06 Security_type: DEFINER Comment: character_set_client: utf8 collation_connection: utf8_general_ci Database Collation: utf8mb4_general_ci 1 row in set (0.00 sec) MariaDB [yinzhengjie2019]> MariaDB [yinzhengjie2019]>
[root@node103.yinzhengjie.org.cn ~]# mysql -e "USE yinzhengjie2019;SHOW TABLES" +---------------------------+ | Tables_in_yinzhengjie2019 | +---------------------------+ | testlog | +---------------------------+ [root@node103.yinzhengjie.org.cn ~]# [root@node103.yinzhengjie.org.cn ~]# mysql -e "SHOW PROCEDURE STATUS\G" *************************** 1. row *************************** Db: yinzhengjie2019 Name: pro_testlog Type: PROCEDURE Definer: root@localhost Modified: 2019-11-08 12:04:06 Created: 2019-11-08 12:04:06 Security_type: DEFINER Comment: character_set_client: utf8 collation_connection: utf8_general_ci Database Collation: utf8mb4_general_ci [root@node103.yinzhengjie.org.cn ~]#
[root@node102.yinzhengjie.org.cn ~]# mysql -uroot -pyinzhengjie yinzhengjie2019 Reading table information for completion of table and column names You can turn off this feature to get a quicker startup with -A Welcome to the MariaDB monitor. Commands end with ; or \g. Your MariaDB connection id is 8 Server version: 5.5.64-MariaDB MariaDB Server Copyright (c) 2000, 2018, Oracle, MariaDB Corporation Ab and others. Type 'help;' or '\h' for help. Type '\c' to clear the current input statement. MariaDB [yinzhengjie2019]> MariaDB [yinzhengjie2019]> SELECT COUNT(*) FROM testlog; +----------+ | COUNT(*) | +----------+ | 0 | +----------+ 1 row in set (0.00 sec) MariaDB [yinzhengjie2019]> MariaDB [yinzhengjie2019]> CALL pro_testlog; Query OK, 1 row affected (59.41 sec) MariaDB [yinzhengjie2019]> CALL pro_testlog; Query OK, 1 row affected (58.75 sec) MariaDB [yinzhengjie2019]> MariaDB [yinzhengjie2019]> CALL pro_testlog; Query OK, 1 row affected (59.01 sec) MariaDB [yinzhengjie2019]> MariaDB [yinzhengjie2019]> CALL pro_testlog; Query OK, 1 row affected (58.75 sec) MariaDB [yinzhengjie2019]> MariaDB [yinzhengjie2019]> SELECT COUNT(*) FROM yinzhengjie2019.testlog; +----------+ | COUNT(*) | +----------+ | 399996 | +----------+ 1 row in set (0.09 sec) MariaDB [yinzhengjie2019]> MariaDB [yinzhengjie2019]> SHOW PROCESSLIST\G *************************** 1. row *************************** Id: 4 User: copy Host: node103.yinzhengjie.org.cn:49672 db: NULL Command: Binlog Dump Time: 2167 State: Master has sent all binlog to slave; waiting for binlog to be updated Info: NULL Progress: 0.000 *************************** 2. row *************************** Id: 8 User: root Host: localhost db: yinzhengjie2019 Command: Query Time: 0 State: NULL Info: SHOW PROCESSLIST Progress: 0.000 2 rows in set (0.00 sec) MariaDB [yinzhengjie2019]> MariaDB [yinzhengjie2019]>
MariaDB [(none)]> SHOW SLAVE STATUS\G *************************** 1. row *************************** Slave_IO_State: Waiting for master to send event Master_Host: 172.30.1.102 Master_User: copy Master_Port: 3306 Connect_Retry: 10 Master_Log_File: master-bin.000001 Read_Master_Log_Pos: 73098208 Relay_Log_File: relay-log.000002 Relay_Log_Pos: 73098271 Relay_Master_Log_File: master-bin.000001 Slave_IO_Running: Yes Slave_SQL_Running: Yes Replicate_Do_DB: Replicate_Ignore_DB: Replicate_Do_Table: Replicate_Ignore_Table: Replicate_Wild_Do_Table: Replicate_Wild_Ignore_Table: Last_Errno: 0 Last_Error: Skip_Counter: 0 Exec_Master_Log_Pos: 73097986 Relay_Log_Space: 73098781 Until_Condition: None Until_Log_File: Until_Log_Pos: 0 Master_SSL_Allowed: No Master_SSL_CA_File: Master_SSL_CA_Path: Master_SSL_Cert: Master_SSL_Cipher: Master_SSL_Key: Seconds_Behind_Master: 1 #發現此時從庫的數據和主庫不一致啦,數據同步的速度較慢,過一段時間咱們發現這個數字又爲0啦!說明數據同步成功啦! Master_SSL_Verify_Server_Cert: No Last_IO_Errno: 0 Last_IO_Error: Last_SQL_Errno: 0 Last_SQL_Error: Replicate_Ignore_Server_Ids: Master_Server_Id: 102 1 row in set (0.00 sec) MariaDB [(none)]> SELECT COUNT(*) FROM yinzhengjie2019.testlog; #不難發現,從庫的消息條數和主庫不一致,目前還處於正在同步的狀態 +----------+ | COUNT(*) | +----------+ | 379301 | +----------+ 1 row in set (0.11 sec) MariaDB [(none)]> MariaDB [(none)]> SHOW SLAVE STATUS\G *************************** 1. row *************************** Slave_IO_State: Waiting for master to send event Master_Host: 172.30.1.102 Master_User: copy Master_Port: 3306 Connect_Retry: 10 Master_Log_File: master-bin.000001 Read_Master_Log_Pos: 88757200 Relay_Log_File: relay-log.000002 Relay_Log_Pos: 88757485 Relay_Master_Log_File: master-bin.000001 Slave_IO_Running: Yes Slave_SQL_Running: Yes Replicate_Do_DB: Replicate_Ignore_DB: Replicate_Do_Table: Replicate_Ignore_Table: Replicate_Wild_Do_Table: Replicate_Wild_Ignore_Table: Last_Errno: 0 Last_Error: Skip_Counter: 0 Exec_Master_Log_Pos: 88757200 Relay_Log_Space: 88757773 Until_Condition: None Until_Log_File: Until_Log_Pos: 0 Master_SSL_Allowed: No Master_SSL_CA_File: Master_SSL_CA_Path: Master_SSL_Cert: Master_SSL_Cipher: Master_SSL_Key: Seconds_Behind_Master: 0 #此時從庫和主庫的數據是一致的啦! Master_SSL_Verify_Server_Cert: No Last_IO_Errno: 0 Last_IO_Error: Last_SQL_Errno: 0 Last_SQL_Error: Replicate_Ignore_Server_Ids: Master_Server_Id: 102 1 row in set (0.00 sec) MariaDB [(none)]> MariaDB [(none)]> SELECT COUNT(*) FROM yinzhengjie2019.testlog; #消息行數和主庫一致 +----------+ | COUNT(*) | +----------+ | 399996 | +----------+ 1 row in set (0.10 sec) MariaDB [(none)]>
五.模擬生產環境中主從複製(主節點已經運行了一段時間且有大量數據時,如何配置並啓動slave節點)
1>.主節點備份現有數據(咱們依舊上面的"node102.yinzhengjie.org.cn"節點做爲master)
[root@node102.yinzhengjie.org.cn ~]# cat /etc/hosts 127.0.0.1 localhost localhost.localdomain localhost4 localhost4.localdomain4 ::1 localhost localhost.localdomain localhost6 localhost6.localdomain6 172.30.1.101 node101.yinzhengjie.org.cn 172.30.1.102 node102.yinzhengjie.org.cn 172.30.1.103 node103.yinzhengjie.org.cn 172.30.1.104 node104.yinzhengjie.org.cn 172.30.1.105 node105.yinzhengjie.org.cn 172.30.1.106 node106.yinzhengjie.org.cn 172.30.1.107 node107.yinzhengjie.org.cn 172.30.1.108 node108.yinzhengjie.org.cn [root@node102.yinzhengjie.org.cn ~]#
[root@node102.yinzhengjie.org.cn ~]# cat /etc/my.cnf #主節點配置並不須要修改 [mysqld] binlog_format = row log_bin = /data/logbin/mysql-bin server-id = 102 log-basename = master character-set-server = utf8mb4 default_storage_engine = InnoDB datadir = /var/lib/mysql socket = /var/lib/mysql/mysql.sock [mysqld_safe] log-error = /var/log/mariadb/mariadb.log pid-file = /var/run/mariadb/mariadb.pid !includedir /etc/my.cnf.d [root@node102.yinzhengjie.org.cn ~]#
[root@node102.yinzhengjie.org.cn ~]# mysqldump -uroot -pyinzhengjie -A -F --single-transaction --master-data=1 > /root/all_bak.sql [root@node102.yinzhengjie.org.cn ~]# [root@node102.yinzhengjie.org.cn ~]# ll total 13596 -rw-r--r-- 1 root root 13918698 Nov 8 12:41 all_bak.sql [root@node102.yinzhengjie.org.cn ~]# [root@node102.yinzhengjie.org.cn ~]# scp all_bak.sql node104.yinzhengjie.org.cn:~ The authenticity of host 'node104.yinzhengjie.org.cn (172.30.1.104)' can't be established. ECDSA key fingerprint is SHA256:F3IVf82keybIystuO6PYRfwr0o5dTftrmAHJWzqO4IA. ECDSA key fingerprint is MD5:02:5d:d8:0a:4a:b4:70:0f:61:be:2c:97:56:db:24:e7. Are you sure you want to continue connecting (yes/no)? yes Warning: Permanently added 'node104.yinzhengjie.org.cn,172.30.1.104' (ECDSA) to the list of known hosts. root@node104.yinzhengjie.org.cn's password: all_bak.sql 100% 13MB 74.2MB/s 00:00 [root@node102.yinzhengjie.org.cn ~]#
2>.從節點配置(此時咱們引入新節點"node104.yinzhengjie.org.cn")
[root@node104.yinzhengjie.org.cn ~]# ll -h total 14M -rw-r--r-- 1 root root 14M Nov 8 12:43 all_bak.sql [root@node104.yinzhengjie.org.cn ~]# [root@node104.yinzhengjie.org.cn ~]# head -25 all_bak.sql | tail /*!40111 SET @OLD_SQL_NOTES=@@SQL_NOTES, SQL_NOTES=0 */; -- -- Position to start replication or point-in-time recovery from -- CHANGE MASTER TO MASTER_LOG_FILE='master-bin.000002', MASTER_LOG_POS=245; -- -- Current Database: `devops` [root@node104.yinzhengjie.org.cn ~]# [root@node104.yinzhengjie.org.cn ~]# vim all_bak.sql #須要傳過來的備份文件稍做修改,主要是配置使用有複製權限的用戶帳號鏈接至主服務器 [root@node104.yinzhengjie.org.cn ~]# [root@node104.yinzhengjie.org.cn ~]# head -30 all_bak.sql | tail #查看修改後的內容 CHANGE MASTER TO MASTER_HOST='172.30.1.102', MASTER_USER='copy', MASTER_PASSWORD='yinzhengjie', MASTER_PORT=3306, MASTER_CONNECT_RETRY=10, MASTER_LOG_FILE='master-bin.000002', MASTER_LOG_POS=245; [root@node104.yinzhengjie.org.cn ~]# [root@node104.yinzhengjie.org.cn ~]#
[root@node104.yinzhengjie.org.cn ~]# ll /var/lib/mysql/ total 0 [root@node104.yinzhengjie.org.cn ~]# [root@node104.yinzhengjie.org.cn ~]# systemctl start mariadb [root@node104.yinzhengjie.org.cn ~]# [root@node104.yinzhengjie.org.cn ~]# ll /var/lib/mysql/ total 37852 -rw-rw---- 1 mysql mysql 16384 Nov 8 12:55 aria_log.00000001 -rw-rw---- 1 mysql mysql 52 Nov 8 12:55 aria_log_control -rw-rw---- 1 mysql mysql 18874368 Nov 8 12:55 ibdata1 -rw-rw---- 1 mysql mysql 5242880 Nov 8 12:55 ib_logfile0 -rw-rw---- 1 mysql mysql 5242880 Nov 8 12:55 ib_logfile1 drwx------ 2 mysql mysql 4096 Nov 8 12:55 mysql srwxrwxrwx 1 mysql mysql 0 Nov 8 12:55 mysql.sock drwx------ 2 mysql mysql 4096 Nov 8 12:55 performance_schema drwx------ 2 mysql mysql 6 Nov 8 12:55 test [root@node104.yinzhengjie.org.cn ~]# [root@node104.yinzhengjie.org.cn ~]# ll total 13596 -rw-r--r-- 1 root root 13918853 Nov 8 12:50 all_bak.sql [root@node104.yinzhengjie.org.cn ~]# [root@node104.yinzhengjie.org.cn ~]# mysql < all_bak.sql #啓動MySQL服務後並導入我們修改的備份文件 [root@node104.yinzhengjie.org.cn ~]# [root@node104.yinzhengjie.org.cn ~]# ll /var/lib/mysql/ total 70636 -rw-rw---- 1 mysql mysql 16384 Nov 8 12:55 aria_log.00000001 -rw-rw---- 1 mysql mysql 52 Nov 8 12:55 aria_log_control drwx------ 2 mysql mysql 60 Nov 8 12:55 devops -rw-rw---- 1 mysql mysql 52428800 Nov 8 12:55 ibdata1 -rw-rw---- 1 mysql mysql 5242880 Nov 8 12:55 ib_logfile0 -rw-rw---- 1 mysql mysql 5242880 Nov 8 12:55 ib_logfile1 -rw-rw---- 1 mysql mysql 84 Nov 8 12:55 master.info drwx------ 2 mysql mysql 4096 Nov 8 12:55 mysql srwxrwxrwx 1 mysql mysql 0 Nov 8 12:55 mysql.sock drwx------ 2 mysql mysql 4096 Nov 8 12:55 performance_schema -rw-rw---- 1 mysql mysql 245 Nov 8 12:55 relay-log.000001 -rw-rw---- 1 mysql mysql 19 Nov 8 12:55 relay-log.index -rw-rw---- 1 mysql mysql 43 Nov 8 12:55 relay-log.info drwx------ 2 mysql mysql 6 Nov 8 12:55 test drwx------ 2 mysql mysql 39 Nov 8 12:55 yinzhengjie2019 [root@node104.yinzhengjie.org.cn ~]# [root@node104.yinzhengjie.org.cn ~]# cat /var/lib/mysql/master.info 18 master-bin.000002 245 172.30.1.102 copy yinzhengjie 3306 10 0 0 1800.000 0 [root@node104.yinzhengjie.org.cn ~]# [root@node104.yinzhengjie.org.cn ~]# [root@node104.yinzhengjie.org.cn ~]# cat /var/lib/mysql/relay-log.info ./relay-log.000001 4 master-bin.000002 245 [root@node104.yinzhengjie.org.cn ~]# [root@node104.yinzhengjie.org.cn ~]#
[root@node104.yinzhengjie.org.cn ~]# mysql Welcome to the MariaDB monitor. Commands end with ; or \g. Your MariaDB connection id is 3 Server version: 5.5.64-MariaDB MariaDB Server Copyright (c) 2000, 2018, Oracle, MariaDB Corporation Ab and others. Type 'help;' or '\h' for help. Type '\c' to clear the current input statement. MariaDB [(none)]> MariaDB [(none)]> MariaDB [(none)]> SHOW SLAVE STATUS\G *************************** 1. row *************************** Slave_IO_State: Master_Host: 172.30.1.102 Master_User: copy Master_Port: 3306 Connect_Retry: 10 Master_Log_File: master-bin.000002 Read_Master_Log_Pos: 245 Relay_Log_File: relay-log.000001 Relay_Log_Pos: 4 Relay_Master_Log_File: master-bin.000002 Slave_IO_Running: No Slave_SQL_Running: No Replicate_Do_DB: Replicate_Ignore_DB: Replicate_Do_Table: Replicate_Ignore_Table: Replicate_Wild_Do_Table: Replicate_Wild_Ignore_Table: Last_Errno: 0 Last_Error: Skip_Counter: 0 Exec_Master_Log_Pos: 245 Relay_Log_Space: 245 Until_Condition: None Until_Log_File: Until_Log_Pos: 0 Master_SSL_Allowed: No Master_SSL_CA_File: Master_SSL_CA_Path: Master_SSL_Cert: Master_SSL_Cipher: Master_SSL_Key: Seconds_Behind_Master: NULL Master_SSL_Verify_Server_Cert: No Last_IO_Errno: 0 Last_IO_Error: Last_SQL_Errno: 0 Last_SQL_Error: Replicate_Ignore_Server_Ids: Master_Server_Id: 0 1 row in set (0.00 sec) MariaDB [(none)]> MariaDB [(none)]> MariaDB [(none)]> START SLAVE; #啓動複製線程 Query OK, 0 rows affected (0.00 sec) MariaDB [(none)]> MariaDB [(none)]> SHOW SLAVE STATUS\G *************************** 1. row *************************** Slave_IO_State: Waiting for master to send event Master_Host: 172.30.1.102 Master_User: copy Master_Port: 3306 Connect_Retry: 10 Master_Log_File: master-bin.000002 Read_Master_Log_Pos: 245 Relay_Log_File: relay-log.000002 Relay_Log_Pos: 530 Relay_Master_Log_File: master-bin.000002 Slave_IO_Running: Yes Slave_SQL_Running: Yes Replicate_Do_DB: Replicate_Ignore_DB: Replicate_Do_Table: Replicate_Ignore_Table: Replicate_Wild_Do_Table: Replicate_Wild_Ignore_Table: Last_Errno: 0 Last_Error: Skip_Counter: 0 Exec_Master_Log_Pos: 245 Relay_Log_Space: 818 Until_Condition: None Until_Log_File: Until_Log_Pos: 0 Master_SSL_Allowed: No Master_SSL_CA_File: Master_SSL_CA_Path: Master_SSL_Cert: Master_SSL_Cipher: Master_SSL_Key: Seconds_Behind_Master: 0 Master_SSL_Verify_Server_Cert: No Last_IO_Errno: 0 Last_IO_Error: Last_SQL_Errno: 0 Last_SQL_Error: Replicate_Ignore_Server_Ids: Master_Server_Id: 102 1 row in set (0.00 sec) MariaDB [(none)]>
3>.驗證數據是否同步成功
[root@node102.yinzhengjie.org.cn ~]# mysql -uroot -pyinzhengjie yinzhengjie2019 Reading table information for completion of table and column names You can turn off this feature to get a quicker startup with -A Welcome to the MariaDB monitor. Commands end with ; or \g. Your MariaDB connection id is 12 Server version: 5.5.64-MariaDB MariaDB Server Copyright (c) 2000, 2018, Oracle, MariaDB Corporation Ab and others. Type 'help;' or '\h' for help. Type '\c' to clear the current input statement. MariaDB [yinzhengjie2019]> MariaDB [yinzhengjie2019]> MariaDB [yinzhengjie2019]> SELECT COUNT(*) FROM yinzhengjie2019.testlog; +----------+ | COUNT(*) | +----------+ | 399996 | +----------+ 1 row in set (0.17 sec) MariaDB [yinzhengjie2019]> MariaDB [yinzhengjie2019]> CALL pro_testlog; Query OK, 1 row affected (1 min 24.17 sec) MariaDB [yinzhengjie2019]> MariaDB [yinzhengjie2019]> CALL pro_testlog; Query OK, 1 row affected (1 min 21.85 sec) MariaDB [yinzhengjie2019]> MariaDB [yinzhengjie2019]> SELECT COUNT(*) FROM yinzhengjie2019.testlog; +----------+ | COUNT(*) | +----------+ | 599994 | +----------+ 1 row in set (0.14 sec) MariaDB [yinzhengjie2019]> MariaDB [yinzhengjie2019]>
[root@node104.yinzhengjie.org.cn ~]# mysql Welcome to the MariaDB monitor. Commands end with ; or \g. Your MariaDB connection id is 7 Server version: 5.5.64-MariaDB MariaDB Server Copyright (c) 2000, 2018, Oracle, MariaDB Corporation Ab and others. Type 'help;' or '\h' for help. Type '\c' to clear the current input statement. MariaDB [(none)]> MariaDB [(none)]> SELECT COUNT(*) FROM yinzhengjie2019.testlog; +----------+ | COUNT(*) | +----------+ | 599994 | +----------+ 1 row in set (0.14 sec) MariaDB [(none)]> MariaDB [(none)]>