一、受權test用戶經過任意主機鏈接當前mysqld,但每秒鐘最大查詢次數不得超過5次;此帳戶的同時鏈接次數不得超過3次;python
#] GRANT USAGE ON *.* TO test@'%' WITH MAX_QUERIES_PER_HOUR 5; #] GRANT USAGE ON *.* TO test@'%' WITH MAX_USER_CONNECTIONS 3;
二、導入hellodb.sql生成數據庫,完成如下練習:(hellodb.sql文件在贈送的隨堂筆記中,請自行下載導入)mysql
(1) 在students表中,查詢年齡大於25歲,且爲男性的同窗的名字和年齡;sql
SELECT Name,Age from students WHERE Age>25 AND Gender='M'; +--------------+-----+ | Name | Age | +--------------+-----+ | Xie Yanke | 53 | | Ding Dian | 32 | | Yu Yutong | 26 | | Shi Qing | 46 | | Tian Boguang | 33 | | Xu Xian | 27 | | Sun Dasheng | 100 | +--------------+-----+
(2) 以ClassID爲分組依據,顯示每組的平均年齡;數據庫
#] SELECT ClassID,avg(age) FROM students GROUP BY ClassID; +---------+----------+ | ClassID | avg(age) | +---------+----------+ | NULL | 63.5000 | | 1 | 20.5000 | | 2 | 36.0000 | | 3 | 20.2500 | | 4 | 24.7500 | | 5 | 46.0000 | | 6 | 20.7500 | | 7 | 19.6667| +---------+----------+
(3) 顯示第2題中平均年齡大於30的分組及平均年齡;vim
#] SELECT ClassID,avg(age) FROM students GROUP BY ClassID HAVING avg(age)>30; +---------+----------+ | ClassID | avg(age) | +---------+----------+ | NULL | 63.5000 | | 2 | 36.0000 | | 5 | 46.0000 | +---------+----------+
(4) 顯示以L開頭的名字的同窗的信息;緩存
#] SELECT * FROM students WHERE Name LIKE 'L%'; +-------+-------------+-----+--------+---------+-----------+ | StuID | Name | Age | Gender | ClassID | TeacherID | +-------+-------------+-----+--------+---------+-----------+ | 8 | Lin Daiyu | 17 | F | 7 | NULL | | 14 | Lu Wushuang | 17 | F | 3 | NULL | | 17 | Lin Chong | 25 | M | 4 | NULL | +-------+-------------+-----+--------+---------+-----------+
(5) 顯示TeacherID非空的同窗的相關信息;安全
#] SELECT * FROM students WHERE TeacherID IS NOT NULL; +-------+-------------+-----+--------+---------+-----------+ | StuID | Name | Age | Gender | ClassID | TeacherID | +-------+-------------+-----+--------+---------+-----------+ | 1 | Shi Zhongyu | 22 | M | 2 | 3 | | 2 | Shi Potian | 22 | M | 1 | 7 | | 3 | Xie Yanke | 53 | M | 2 | 16 | | 4 | Ding Dian | 32 | M | 4 | 4 | | 5 | Yu Yutong | 26 | M | 3 | 1 | +-------+-------------+-----+--------+---------+-----------+
(6) 以年齡排序後,顯示年齡最大的前10位同窗的信息;服務器
#] SELECT * FROM students ORDER BY Age DESC LIMIT 10; +-------+--------------+-----+--------+---------+-----------+ | StuID | Name | Age | Gender | ClassID | TeacherID | +-------+--------------+-----+--------+---------+-----------+ | 25 | Sun Dasheng | 100 | M | NULL | NULL | | 3 | Xie Yanke | 53 | M | 2 | 16 | | 6 | Shi Qing | 46 | M | 5 | NULL | | 13 | Tian Boguang | 33 | M | 2 | NULL | | 4 | Ding Dian | 32 | M | 4 | 4 | | 24 | Xu Xian | 27 | M | NULL | NULL | | 5 | Yu Yutong | 26 | M | 3 | 1 | | 17 | Lin Chong | 25 | M | 4 | NULL | | 23 | Ma Chao | 23 | M | 4 | NULL | | 18 | Hua Rong | 23 | M | 7 | NULL | +-------+--------------+-----+--------+---------+-----------+
(7) 查詢年齡大於等於20歲,小於等於25歲的同窗的信息;用三種方法;網絡
方法一:併發
#] SELECT * FROM students WHERE Age>=20 AND Age SELECT * FROM students WHERE Age BETWEEN 20 AND 25; +-------+---------------+-----+--------+---------+-----------+ | StuID | Name | Age | Gender | ClassID | TeacherID | +-------+---------------+-----+--------+---------+-----------+ | 1 | Shi Zhongyu | 22 | M | 2 | 3 | | 2 | Shi Potian | 22 | M | 1 | 7 | | 9 | Ren Yingying | 20 | F | 6 | NULL | | 11 | Yuan Chengzhi | 23 | M | 6 | NULL | | 16 | Xu Zhu | 21 | M | 1 | NULL | | 17 | Lin Chong | 25 | M | 4 | NULL | | 18 | Hua Rong | 23 | M | 7 | NULL | | 21 | Huang Yueying | 22 | F | 6 | NULL | | 22 | Xiao Qiao | 20 | F | 1 | NULL | | 23 | Ma Chao | 23 | M | 4 | NULL | +-------+---------------+-----+--------+---------+-----------+
mysql> SELECT * FROM students WHERE Age >=20 AND Age <=25; mysql> SELECT * FROM students WHERE Age BETWEEN 20 AND 25; mysql> SELECT * FROM students WHERE Age IN (20,21,22,23,24,25);
(8) 顯示前5位同窗的姓名、課程及成績;
#] SELECT st.Name,co.Course,sc.Score FROM students st,courses co,scores sc WHERE st.StuID=sc.StuID and sc.CourseID=co.CourseID LIMIT 5; +-------------+----------------+-------+ | Name | Course | Score | +-------------+----------------+-------+ | Shi Zhongyu | Kuihua Baodian | 77 | | Shi Zhongyu | Weituo Zhang | 93 | | Shi Potian | Kuihua Baodian | 47 | | Shi Potian | Daiyu Zanghua | 97 | | Xie Yanke | Kuihua Baodian | 88 | +-------------+----------------+-------+
(9) 顯示其成績高於80的同窗的名稱及課程;
#] SELECT st.Name,co.Course FROM students st,courses co,scores sc WHERE st.StuID=sc.StuID and sc.CourseID=co.CourseID AND sc.Score>80; +-------------+----------------+ | Name | Course | +-------------+----------------+ | Shi Zhongyu | Weituo Zhang | | Shi Potian | Daiyu Zanghua | | Xie Yanke | Kuihua Baodian | | Ding Dian | Kuihua Baodian | | Shi Qing | Hamo Gong | | Xi Ren | Hamo Gong | | Xi Ren | Dagou Bangfa | | Lin Daiyu | Jinshe Jianfa | +-------------+----------------+
(10) 求前8位同窗每位同窗本身兩門課的平均成績,並按降序排列;
#] SELECT st.Name,sc2.avg_score FROM (SELECT StuID,avg(Score) AS avg_score FROM scores GROUP BY StuID) AS sc2,students st where st.StuID=sc2.StuID ORDER BY avg_score DESC LIMIT 8; +-------------+-----------+ | Name | avg_score | +-------------+-----------+ | Shi Qing | 96.0000 | | Shi Zhongyu | 85.0000 | | Xi Ren | 84.5000 | | Xie Yanke | 81.5000 | | Ding Dian | 80.0000 | | Lin Daiyu | 75.0000 | | Shi Potian | 72.0000 | | Yu Yutong | 51.0000 | +-------------+-----------+
(11) 顯示每門課程課程名稱及學習了這門課的同窗的個數;
#] SELECT co2.Course,count(co2.StuID) FROM (SELECT co.Course,sc.StuID FROM courses co,scores sc WHERE co.CourseID=sc.CourseID) AS co2 GROUP BY co2.Course; +----------------+------------------+ | Course | count(co2.StuID) | +----------------+------------------+ | Dagou Bangfa | 2 | | Daiyu Zanghua | 2 | | Hamo Gong | 3 | | Jinshe Jianfa | 1 | | Kuihua Baodian | 4 | | Taiji Quan | 1 | | Weituo Zhang | 2 | +----------------+------------------+
三、回顧並詳細總結MySQL的存儲引擎、索引;
經常使用存儲引擎的對比:
特色 | MyISAM | InnoDB | MEMORY | MERGE | NDB |
存儲限制 | 有 | 64TB | 有 | 沒有 | 有 |
事務安全 | 支持 | ||||
鎖機制 | 表鎖 | 行鎖 | 表鎖 | 表鎖 | 行鎖 |
B樹索引 | 支持 | 支持 | 支持 | 支持 | 支持 |
哈希索引 | 支持 | ||||
全文索引 | 支持 | ||||
集羣索引 | 支持 | ||||
數據緩存 | 支持 | 支持 | 支持 | ||
索引緩存 | 支持 | 支持 | 支持 | 支持 | 支持 |
數據可壓縮 | 支持 | ||||
空間使用 | 低 | 高 | N/A | 低 | 低 |
內存使用 | 低 | 高 | 中等 | 低 | 高 |
批量插入的速度 | 高 | 低 | 高 | 高 | 高 |
支持外鍵 | 支持 |
InnoDB:5.5版本以後默認的存儲引擎,具備提交、回滾和崩潰恢復能力的事務安全機制,可是對比MyISAM,InnoDB寫處理效率差一些,而且會佔用更多的磁盤空間以保留數據和索引。
處理大量的短時間事務;
數據存儲於「表空間(table space)」中;
(1) 全部InnoDB表的數據和索引放置於同一個表空間中;
表空間文件:datadir定義的目錄下
數據文件:ibddata1, ibddata2, …
(2) 每一個表單獨使用一個表空間存儲表的數據和索引;
innodb_file_per_table=ON
數據文件(存儲數據和索引):tbl_name.ibd,
表格式定義:tbl_name.frm
基於MVCC來支持高併發,支持全部的四個隔離級別,默認級別爲REPEATABLE READ; 間隙鎖防止幻讀;
使用匯集索引
支持「自適應hash索引」
鎖粒度:行級鎖
MariaDB (XtraDB (percona))
數據存儲:表空間
併發:MVCC, 間隙鎖
索引:彙集索引、輔助索引
性能:預計操做、自適應hash、插入緩存區
備份:支持熱備(xtrabacup)
MyISAM:5.5以前版本的默認存儲引擎,MyISAM不支持事務、外鍵,其優點是訪問速度快,而且支持全文索引,對事務完整性沒有要求或者以SELECT、INSERT爲主的應用基本上均可以使用這個引擎來建立表。
支持全文索引(FULLTEXT index)、壓縮、空間函數(GIS); 但不支持事務,且爲表級鎖;
崩潰後沒法安全恢復
適用場景:只讀(或者寫較少)、表較小(能夠接受長時間進行修復操做)
Aria:crash-safe
文件:
tbl_name.frm: 表格式定義
tbl_name.MYD: 數據文件
tbl_name.MYI: 索引文件
特性:
加鎖和併發:表級鎖
修復:手工或自動修復、但可能丟失數據
索引:非彙集索引
延遲更新索引鍵:
壓縮表
行格式:dynamic, fixed, compressed, compact, redundent
其它的存儲引擎:
CSV:將普通的CSV(字段經過逗號分隔)做爲MySQL表使用;
MRG_MYISAM:將多個MyISAM表合併成爲一個虛擬表;
BLACKHOLE:相似於/dev/null,不真正存儲任何數據;
MEMORY:全部數據都保存於內存中,內存表;支持hash索引;表級鎖;
臨時表
PERFORMANCE_SCHEMA:僞存儲引擎;
ARCHIVE:只支持SELECT和INSERT操做;支持行級鎖和專用緩存區;
FEDERATED:用於訪問其它遠程MySQL服務器一個代理,它經過建立一個到遠程MySQL服務器的客戶端鏈接,並將查詢傳輸到遠程服務器執行,然後完成數據存取;
在MariaDB的上實現是FederatedX
MariaDB支持的其它存儲引擎:
OQGraph
SphinxSE
TokuDB
Cassandra
CONNECT
SQUENCE
MySQL中的索引:
基本法則:索引應該構建在被用做查詢條件的字段上;
索引類型:
B+ Tree索引:順序存儲,每個葉子節點到根結點的距離是相同的;左前綴索引,適合查詢範圍類的數據;
可使用B-Tree索引的查詢類型:
全值匹配:精確某個值, 「Jinjiao King」;
匹配最左前綴:只精確匹配起頭部分,」Jin%」
匹配範圍值:
精確匹配某一列並範圍匹配另外一列:
只訪問索引的查詢
不適合使用B-Tree索引的場景:
若是不從最左列開始,索引無效; (Age,Name)
不能跳過索引中的列;(StuID,Name,Age)
若是查詢中某個列是爲範圍查詢,那麼其右側的列都沒法再使用索引優化查詢;(StuID,Name)
Hash索引:基於哈希表實現,特別適用於精確匹配索引中的全部列;
注意:只有Memory存儲引擎支持顯式hash索引;
適用場景:
只支持等值比較查詢,包括=, IN(), <=>;
不適合使用hash索引的場景:
存儲的非爲值的順序,所以,不適用於順序查詢;
不支持模糊匹配;
空間索引(R-Tree):
MyISAM支持空間索引,它可使用諸如GEOMETRY這樣的地址空間數據類型。和B-Tree索引不一樣,空間索引不會要求WHERE子句使用索引的最左前綴。它同時全方位的索引了數據。這樣就能夠高效的使用任何數據組合進行查找,然而,必須使用MySQL GIS函數,例如MBRCONTAINS(),才能獲得這個好處。
全文索引(FULLTEXT):
FULLTEXT是MyISAM表的一種特殊索引,它從文本中找到關鍵字,而不是直接和索引中的值進行比較。
索引優勢:
索引能夠下降服務須要掃描的數據量,減小了IO次數;
索引能夠幫助服務器避免排序和使用臨時表;
索引能夠幫助將隨機I/O轉爲順序I/O;
高性能索引策略:
獨立使用列,儘可能避免其參與運算;
左前綴索引:索引構建於字段的左側的多少個字符,要經過索引選擇性來評估
索引選擇性:不重複的索引值和數據表的記錄總數的比值;
多列索引:
AND操做時更適合使用多列索引;
選擇合適的索引列次序:將選擇性最高放左側;
MySQL常見的三種備份方式:
1)mysqldump+binlog,徹底備份,經過備份二進制日誌實現增量備份
2)lvm2快照+binlog:幾乎熱備,物理備份
3)xtrabackup:
對InnoDB:熱備,支持徹底備份和增量備份
對MyISAM引擎:溫備,只支持徹底備份
環境:
源庫:192.168.0.23
目標庫:192.168.0.24
方式1:mysqldump+binlog
#+++++++++++++++++++++源庫++++++++++++++++++++++++++# ###開啓數據庫的二機制日誌### ~]# vim /etc/my.cnf [mysqld] log-bin=mysql_bin ~]# systemctl start mariadb.service > show global variables like '%log_bin%'; +---------------------------------+-------+ | Variable_name | Value | +---------------------------------+-------+ | log_bin | ON | | log_bin_trust_function_creators | OFF | | sql_log_bin | ON | +---------------------------------+-------+ ###使用mysqldump工具對數據庫進行全備### ~]# mysqldump -A --single-transaction --master-data=2 >/tmp/all.sql ###對數據庫作一些操做### > create database mydb; > use mydb; > create table t1 (id int,name char(10)); > insert into t1 values (1,'Tom'); > select * from t1; +------+------+ | id | name | +------+------+ | 1 | Tom | +------+------+ ###假設此時數據庫發生故障,須要經過最近一次的全備和二進制日誌在備庫上將數據庫恢復到故障點前的狀態### ###在備份文件肯定起始pos和日誌文件名### ~]# vim /tmp/all.sql ... CHANGE MASTER TO MASTER_LOG_FILE='mysql_bin.000003', MASTER_LOG_POS=245; ... ###在源庫上導出全備後生成的log日誌### ~]# mysqlbinlog --start-position=245 /var/lib/mysql/mysql_bin.000003 >/tmp/incre.sql ###將全備和增備數據傳送到目標庫系統上### ~]# scp /tmp/all.sql /tmp/incre.sql root@192.168.0.24:/tmp/ #+++++++++++++++++++++目標庫++++++++++++++++++++++++++# ###在目標庫上分別恢復全備和增備### > set sql_log_bin=0; #臨時關閉二進制日誌,爲了防止導入全備時產生大量二進制日誌 > source /tmp/all.sql; #導入全備 ~]# mysql </tmp/incre.sql #導入增備 ###恢復後庫和表數據都存在,恢復成功!### > select * from mydb.t1; +------+------+ | id | name | +------+------+ | 1 | Tom | +------+------+ 1 row in set (0.00 sec)
方式2:基於lvm2的快照
#+++++++++++++++++++++源庫++++++++++++++++++++++++++# ###建立一個LV,專門用來存放MySQL的數據文件### ~]# pvcreate /dev/sdb1 ~]# vgcreate myvg /dev/sdb1 ~]# lvcreate -n mydata -L 2G myvg ~]# mke2fs -t ext4 /dev/myvg/mydata ~]# mount /dev/myvg/mydata /var/lib/mysql ###製做lvm2快照### mysql> FLUSH TABLES WITH READ LOCK; #鎖定全部表 mysql> FLUSH LOGS; mysql> SHOW MASTER STATUS; #記錄二進制日誌文件及事件位置 +------------------+----------+--------------+------------------+ | File | Position | Binlog_Do_DB | Binlog_Ignore_DB | +------------------+----------+--------------+------------------+ | mysql_bin.000004 | 245 | | | +------------------+----------+--------------+------------------+ ~]# lvcreate -L 1G -s -n mydata_snap /dev/myvg/mydata #建立快照 mysql> UNLOCK TABLES; ###對數據庫作一些操做### mysql> create database mydb; mysql> use mydb; mysql> create table t1 (id int,name char(20)); mysql> insert into t1 values (1,'Tom'),(2,'Jerry'); mysql> select * from t1; +------+-------+ | id | name | +------+-------+ | 1 | Tom | | 2 | Jerry | +------+-------+ ###假設此時數據庫發生故障,須要使用lvm快照和二進制日誌在備庫上將數據庫恢復到故障點前的狀態 ###掛載並備份快照備份### ~]# mount /dev/myvg/mydata_snap /mnt/mydata_snap/ ~]# cd /mnt/mydata_snap/ ~]# tar -zcvf /tmp/mydata.tar.gz ./*###在源庫上導出全備後生成的log日誌###
###恢復後庫和表數據都存在,恢復成功!###
mysql> select * from mydb.t1; +------+-------+ | id | name | +------+-------+ | 1 | Tom | | 2 | Jerry | +------+-------+
方式3:xtrabackup
#+++++++++++++++++++++源庫++++++++++++++++++++++++++#mysql> create database mydb; mysql> use mydb; mysql> create table t1 (id int,name char(20)); mysql> insert into t1 values (1,'Tom'),(2,'Jerry'); mysql> select * from t1; +------+-------+ | id | name | +------+-------+ | 1 | Tom | | 2 | Jerry | +------+-------+ ~]# innobackupex --incremental /backup/ --incremental-basedir=/backup/2017-10-06_12-12-25 #這裏要指明最近一次全備文件夾名稱 ~]# ls /backup/ 2017-10-06_12-08-00 2017-10-06_12-12-15 ~]# scp -r /backup/* root@192.168.0.24:/backup#+++++++++++++++++++++目標庫++++++++++++++++++++++++++#
~]# innobackupex --apply-log --redo-only /backup/2017-10-06_12-08-00/ ~]# innobackupex --apply-log --redo-only /backup/2017-10-06_12-08-00/ --incremental-dir=/backup/2017-10-06_12-12-25/###使用合併後的全量備份集進行恢復###
~]# innobackupex --copy-back /backup/2017-10-06_12-08-00/
~]# chown -R mysql.mysql /var/lib/mysql/* ~]# systemctl start mariadb.service###恢復後庫和表數據都存在,恢復成功!###
mysql> select * from mydb.t1; +------+-------+ | id | name | +------+-------+ | 1 | Tom | | 2 | Jerry | +------+-------+
五、MySQL的複製有哪些類型,詳細描述複製原理,並完成實際的操做
方式1:主從模式
主從複製原理:
1) Slave服務器上執行start slave,開啓主從複製開關。
2) 此時,Slave服務器的IO線程會經過在Master上受權的複製用戶權限請求鏈接Master服務器,並請求從指定binlog日誌文件的指定位置(日誌文件名和位置就是在配置主從複製服務時執行change master命令時指定的)以後發送binlog日誌內容。
3) Master服務器接收到來自Slave服務器的IO線程的請求後,Master服務器上負責複製IO線程根據Slave服務器的IO線程請求的信息讀取指定binlog日誌文件指定位置以後的binlog日誌信息,而後返回給Slave端的IO線程。返回信息中除了binlog日誌內容外,還有本次返回日誌內容後在Master服務器的新的Binlog文件名稱以及在binlog中的下一個指定更新位置。
4) 當Slave服務器的IO線程獲取到來自Master服務器上IO線程發送日誌內容及日誌文件位置點後,將binlog日誌內容依次寫入到Slave端自身的relay log(即中繼日誌)文件(MySQL-relay-bin.xxxxxx)的最末端,並將新的binlog文件名和位置記錄到master-info文件中,以便下一次讀取Master端新binlog日誌時可以告訴Master服務器須要新binlog日誌的哪一個文件哪一個位置開始請求的binlog日誌內容。
5) Slave服務器端的SQL線程會實時的檢測本地RelayLog中新增長的日誌內容,而後及時的把Log文件中的內容解析成在Master端曾經執行的SQL語句的內容,並在自身Slave服務器上按語句的順序執行應用這些SQL語句,應用完畢後清理應用過的日誌。
6) 通過了上面的過程,就能夠確保在Master端和Slave端執行了一樣的SQL語句。當複製狀態正常的狀況下,Master端和Slave端的數據是徹底同樣的。
配置過程:
環境:
主庫:192.168.0.23
備庫:192.168.0.24
#+++++++++++++++++++++主庫++++++++++++++++++++++++++#mysql> change master to master_host='192.168.0.23',master_user='repluser',master_password='replpass',master_log_file='mysql_bin.000003',master_log_pos=497; mysql> start slave; mysql> show slave status\G; **************************** 1. row *************************** Slave_IO_State: Waiting for master to send event Master_Host: 192.168.0.23 Master_User: repluser Master_Port: 3306 Connect_Retry: 60 Master_Log_File: mysql_bin.000004 Read_Master_Log_Pos: 421 Relay_Log_File: mariadb-relay-bin.000004 Relay_Log_Pos: 616 Relay_Master_Log_File: mysql_bin.000004 Slave_IO_Running: Yes #只要Slave_IO_Running和Slave_SQL_Running都爲Yes,主從複製就配置成功了 Slave_SQL_Running: Yes Replicate_Do_DB: Replicate_Ignore_DB: Replicate_Do_Table: Replicate_Ignore_Table: Replicate_Wild_Do_Table: Replicate_Wild_Ignore_Table: Last_Errno: 0 Last_Error: Skip_Counter: 0 Exec_Master_Log_Pos: 421 Relay_Log_Space: 1285 Until_Condition: None Until_Log_File: Until_Log_Pos: 0 Master_SSL_Allowed: No Master_SSL_CA_File: Master_SSL_CA_Path: Master_SSL_Cert: Master_SSL_Cipher: Master_SSL_Key: Seconds_Behind_Master: 0 Master_SSL_Verify_Server_Cert: No Last_IO_Errno: 0 Last_IO_Error: Last_SQL_Errno: 0 Last_SQL_Error: Replicate_Ignore_Server_Ids: Master_Server_Id: 1注意:若是主節點已經運行了一段時間,且有大量數據時,應當經過備份主服務器並恢復數據至從服務器;複製起始位置爲備份時,二進制日誌文件及其POS;
方式2:主主模式
主主同步實現原理與主從同步類似,只不過兩個節點互爲主從,相對於主從模式,主主模式具備下列特色:
(1) 都啓用binlog和relay log
(2) 定義自動增加的id字段的增加方式
(3) 都受權有複製權限的用戶帳號
(4) 各自把對方指定爲主服務器
配置過程:
環境:
主庫:192.168.0.23
備庫:192.168.0.24
#+++++++++++++++++++++主庫1++++++++++++++++++++++++++#Master_Server_Id: 2
方式3:半同步模式
半同步工做方式:介於異步複製和全同步複製之間,主庫在執行完客戶端提交的事務後不是馬上返回給客戶端,而是等待至少一個從庫接收到並寫到relay log中才返回給客戶端。相對於異步複製,半同步複製提升了數據的安全性,同時它也形成了必定程度的延遲,這個延遲最少是一個TCP/IP往返的時間。因此,半同步複製最好在低延時的網絡中使用。
原理圖:
存在的問題:
客戶端事務在存儲引擎層提交後,在獲得從庫確認的過程當中,主庫宕機了,此時,可能的狀況有兩種:
1)事務還沒發送到從庫上。此時,客戶端會收到事務提交失敗的信息,客戶端會從新提交該事務到新的主上,當宕機的主庫從新啓動後,以從庫的身份從新加入到該主從結構中,會發現,該事務在從庫中被提交了兩次,一次是以前做爲主的時候,一次是被新主同步過來的。
2)事務已經發送到從庫上。此時,從庫已經收到並應用了該事務,可是客戶端仍然會收到事務提交失敗的信息,從新提交該事務到新的主上。
配置方式:
環境:
主節點:192.168.0.23
備節點:192.168.0.24
1)在主備節點上均安裝半同步插件
#+++++++++++++++++++++主庫++++++++++++++++++++++++++# mysql> INSTALL PLUGIN rpl_semi_sync_master SONAME 'semisync_master.so'; mysql> SELECT PLUGIN_NAME, PLUGIN_STATUS FROM INFORMATION_SCHEMA.PLUGINS WHERE PLUGIN_NAME LIKE '%semi%'; +----------------------+---------------+ | PLUGIN_NAME | PLUGIN_STATUS | +----------------------+---------------+ | rpl_semi_sync_master | ACTIVE | +----------------------+---------------+ #+++++++++++++++++++++備庫++++++++++++++++++++++++++# mysql> INSTALL PLUGIN rpl_semi_sync_slave SONAME 'semisync_slave.so'; mysql> SELECT PLUGIN_NAME, PLUGIN_STATUS FROM INFORMATION_SCHEMA.PLUGINS WHERE PLUGIN_NAME LIKE '%semi%'; +----------------------+---------------+ | PLUGIN_NAME | PLUGIN_STATUS | +----------------------+---------------+ | rpl_semi_sync_slave | ACTIVE | +----------------------+---------------+
2)修改主備節點上的配置文件 #+++++++++++++++++++++主庫++++++++++++++++++++++++++# ~]# vim /etc/my.cnf [mysqld] log-bin=mysql_bin server-id=1 #+++++++++++++++++++++備庫++++++++++++++++++++++++++# ~]# vim /etc/my.cnf [mysqld] log-bin=mysql_bin server-id=2 3)在主庫上建立複製用戶 mysql> grant replication slave,replication client on *.* to 'repluser'@'192.168.0.%' identified by 'replpass'; mysql> flush privileges; 4)查看主庫當前的binlog日誌位置 mysql> show master status; +------------------+----------+--------------+------------------+ | File | Position | Binlog_Do_DB | Binlog_Ignore_DB | +------------------+----------+--------------+------------------+ | mysql_bin.000003 | 507 | | | +------------------+----------+--------------+------------------+ 5)在備庫上配置slave mysql> change master to master_host='192.168.0.23',master_user='repluser',master_password='replpass',master_log_file='mysql_bin.000003',master_log_pos=507; 6)在主備節點上分別啓動半同步服務 #+++++++++++++++++++++主庫++++++++++++++++++++++++++# mysql> set global rpl_semi_sync_slave_enabled=1; mysql> show global variables like '%semi%'; +------------------------------------+-------+ | Variable_name | Value | +------------------------------------+-------+ | rpl_semi_sync_master_enabled | ON | | rpl_semi_sync_master_timeout | 10000 | | rpl_semi_sync_master_trace_level | 32 | | rpl_semi_sync_master_wait_no_slave | ON | +------------------------------------+-------+ #+++++++++++++++++++++備庫++++++++++++++++++++++++++# mysql> set global rpl_semi_sync_slave_enabled=1; mysql> show global variables like '%semi%'; +------------------------------------+-------+ | Variable_name | Value | +------------------------------------+-------+ | rpl_semi_sync_slave_enabled | ON | | rpl_semi_sync_slave_trace_level | 32 | +------------------------------------+-------+ 7)從節點上啓用半同步 mysql> start slave; Query OK, 0 rows affected (0.00 sec) mysql> show slave status\G; *************************** 1. row *************************** Slave_IO_State: Waiting for master to send event Master_Host: 192.168.0.23 Master_User: repluser Master_Port: 3306 Connect_Retry: 60 Master_Log_File: mysql_bin.000003 Read_Master_Log_Pos: 507 Relay_Log_File: mariadb-relay-bin.000002 Relay_Log_Pos: 529 Relay_Master_Log_File: mysql_bin.000003 Slave_IO_Running: Yes Slave_SQL_Running: Yes Replicate_Do_DB: Replicate_Ignore_DB: Replicate_Do_Table: Replicate_Ignore_Table: Replicate_Wild_Do_Table: Replicate_Wild_Ignore_Table: Last_Errno: 0 Last_Error: Skip_Counter: 0 Exec_Master_Log_Pos: 507 Relay_Log_Space: 825 Until_Condition: None Until_Log_File: Until_Log_Pos: 0 Master_SSL_Allowed: No Master_SSL_CA_File: Master_SSL_CA_Path: Master_SSL_Cert: Master_SSL_Cipher: Master_SSL_Key: Seconds_Behind_Master: 0 Master_SSL_Verify_Server_Cert: No Last_IO_Errno: 0 Last_IO_Error: Last_SQL_Errno: 0 Last_SQL_Error: Replicate_Ignore_Server_Ids: Master_Server_Id: 1
方式4:基於SSL的主從複製
配置過程:
環境:
主庫:192.168.0.23
備庫:192.168.0.24
1)在主備節點分別搭建CA服務器和客戶端
#+++++++++++++++++++++主庫++++++++++++++++++++++++++#e is 65537 (0x10001)
###生成自簽證書### ~]# openssl req -new -x509 -key /etc/pki/CA/private/cakey.pem -out /etc/pki/CA/cacert.pem You are about to be asked to enter information that will be incorporated into your certificate request. What you are about to enter is what is called a Distinguished Name or a DN. There are quite a few fields but you can leave some blank For some fields there will be a default value, If you enter '.', the field will be left blank. ----- Country Name (2 letter code) [XX]:CN State or Province Name (full name) []:SHANXI Locality Name (eg, city) [Default City]:XIAN Organization Name (eg, company) [Default Company Ltd]:DCITS Organizational Unit Name (eg, section) []:Ops Common Name (eg, your name or your server's hostname) []:ca.dcits.com Email Address []:fangtao@dcits.com ###爲CA提供所需的目錄及文件### ~]# touch /etc/pki/CA/{serial,index.txt} ~]# echo 01>/etc/pki/CA/serial #+++++++++++++++++++++備庫++++++++++++++++++++++++++# ###生成簽署請求### ~]# mkdir /etc/mysql/ssl ~]# cd /etc/mysql/ssl ~]# (umask 077;openssl genrsa -out mysql.key 2048) Generating RSA private key, 2048 bit long modulus .....................................................................+++ ...................+++ e is 65537 (0x10001) ~]# openssl req -new -key mysql.key -out mysql.csr -days 365 You are about to be asked to enter information that will be incorporated into your certificate request. What you are about to enter is what is called a Distinguished Name or a DN. There are quite a few fields but you can leave some blank For some fields there will be a default value, If you enter '.', the field will be left blank. ----- Country Name (2 letter code) [XX]:CN State or Province Name (full name) []:SHANXI Locality Name (eg, city) [Default City]:XIAN Organization Name (eg, company) [Default Company Ltd]:DCITS Organizational Unit Name (eg, section) []:Ops Common Name (eg, your name or your server's hostname) []:ca.dcits.com Email Address []:fangtao@dcits.com Please enter the following 'extra' attributes to be sent with your certificate request A challenge password []: An optional company name []: ###將請求發送給主庫(CA主機)### ~]# scp mysql.csr mysql.key root@192.168.0.23:/tmp/ ### #+++++++++++++++++++++主庫++++++++++++++++++++++++++# ###在主庫(CA服務器)上籤署證書### ~]# openssl ca -in /tmp/mysql.csr -out /etc/pki/CA/certs/mysql.crt -days 365 Using configuration from /etc/pki/tls/openssl.cnf Check that the request matches the signature Signature ok Certificate Details: Serial Number: 1 (0x1) Validity Not Before: Jul 19 10:05:05 2017 GMT Not After : Jul 19 10:05:05 2018 GMT Subject: countryName = CN stateOrProvinceName = SHANXI organizationName = DCITS organizationalUnitName = Ops commonName = ca.dcits.com emailAddress = fangtao@dcits.com X509v3 extensions: X509v3 Basic Constraints: CA:FALSE Netscape Comment: OpenSSL Generated Certificate X509v3 Subject Key Identifier: A4:59:A0:16:FB:D4:C2:27:1A:09:61:FA:C4:39:B7:CE:01:60:2A:28 X509v3 Authority Key Identifier: keyid:0B:15:FD:99:9B:55:BB:52:B1:F3:37:50:42:BC:59:52:0A:13:CB:D9 Certificate is to be certified until Jul 19 10:05:05 2018 GMT (365 days) Sign the certificate? [y/n]:y 1 out of 1 certificate requests certified, commit? [y/n]y Write out database with 1 new entries Data Base Updated ###將證書發送給備庫### ~]# scp mysql.crt cacert.pem root@192.168.0.24:/etc/mysql/ssl ###修改主庫上證書相關文件的權限### ~]# chown -R mysql.mysql /etc/pki/CA 2)分別在主備庫上修改配置文件,啓用ssl #+++++++++++++++++++++主庫++++++++++++++++++++++++++# ~]# vim /etc/my.cnf [mysqld] log-bin=mysql_bin server-id=1 ssl ssl_ca=/etc/pki/CA/cacert.pem ssl_cert=/etc/pki/CA/certs/mysql.crt ssl_key=/etc/pki/CA/private/mysql.key ~]# systemctl start mariadb.service mysq> show variables like '%ssl%'; +---------------+-------------------------------+ | Variable_name | Value | +---------------+-------------------------------+ | have_openssl | YES | | have_ssl | YES | | ssl_ca | /etc/pki/CA/cacert.pem | | ssl_capath | | | ssl_cert | /etc/pki/CA/certs/mysql.crt | | ssl_cipher | | | ssl_key | /etc/pki/CA/private/mysql.key | +---------------+-------------------------------+ #+++++++++++++++++++++備庫++++++++++++++++++++++++++# ~]# vim /etc/my.cnf [mysqld] server-id=2 log-bin=mysql_bin ssl ssl_ca=/etc/mysql/cacert.pem ssl_cert=/etc/mysql/mysql.crt ssl_key=/etc/mysql/mysql.key ~]# systemctl start mariadb.service mysq> show variables like '%ssl%'; +---------------+-----------------------+ | Variable_name | Value | +---------------+-----------------------+ | have_openssl | DISABLED | | have_ssl | DISABLED | | ssl_ca | /etc/mysql/cacert.pem | | ssl_capath | | | ssl_cert | /etc/mysql/mysql.crt | | ssl_cipher | | | ssl_key | /etc/mysql/mysql.key | +---------------+-----------------------+ 3)在主庫上建立複製帳號 mysql> grant replication slave,replication client on *.* to 'repluser'@'192.168.0.%' require ssl; mysql> set password for repluser=PASSWORD('replpass'); mysql> flush privileges; mysql> show master status; +------------------+----------+--------------+------------------+ | File | Position | Binlog_Do_DB | Binlog_Ignore_DB | +------------------+----------+--------------+------------------+ | mysql_bin.000004 | 484 | | | +------------------+----------+--------------+------------------+ 4)在從庫上開啓同步 mysql> change master to master_host='192.168.0.23', master_user='repluser', master_password='replpass', master_log_file='mysql_bin.000004', master_ssl=1, master_ssl_ca='/etc/mysql/ssl/cacert.pem', master_ssl_cert='/etc/mysql/ssl/mysql.crt', master_ssl_key='/etc/mysql/ssl/mysql.key'; 5)啓動主從同步並查看狀態 mysql> start slave; mysql> show slave status\G; *************************** 1. row *************************** Slave_IO_State: Waiting for master to send event Master_Host: 192.168.0.23 Master_User: repluser Master_Port: 3306 Connect_Retry: 60 Master_Log_File: mysql_bin.000008 Read_Master_Log_Pos: 245 Relay_Log_File: mariadb-relay-bin.000009 Relay_Log_Pos: 529 Relay_Master_Log_File: mysql_bin.000008 Slave_IO_Running: Yes #同步狀態正常! Slave_SQL_Running: Yes Replicate_Do_DB: Replicate_Ignore_DB: Replicate_Do_Table: Replicate_Ignore_Table: Replicate_Wild_Do_Table: Replicate_Wild_Ignore_Table: Last_Errno: 0 Last_Error: Skip_Counter: 0 Exec_Master_Log_Pos: 245 Relay_Log_Space: 825 Until_Condition: None Until_Log_File: Until_Log_Pos: 0 Master_SSL_Allowed: Yes Master_SSL_CA_File: /etc/mysql/ssl/cacert.pem #證書已被應用 Master_SSL_CA_Path: Master_SSL_Cert: /etc/mysql/ssl/mysql.crt Master_SSL_Cipher: Master_SSL_Key: /etc/mysql/ssl/mysql.key Seconds_Behind_Master: 0 Master_SSL_Verify_Server_Cert: No Last_IO_Errno: 0 Last_IO_Error: Last_SQL_Errno: 0 Last_SQL_Error: Replicate_Ignore_Server_Ids: Master_Server_Id: 1
方式5:過濾器 同步方式說明:讓從節點僅複製指定的數據庫,或指定數據庫的指定表; 有兩種實現方式:1) 主服務器僅向二進制日誌中記錄與特定數據庫(特定表)相關的事件;問題:時間還原沒法實現;不建議使用;2) 從服務器SQL_THREAD在replay中繼日誌中的事件時,僅讀取與特定數據庫(特定表)相關的事件並應用於本地;問題:會形成網絡及磁盤IO浪費 配置過程: 環境: 主庫:192.168.0.23 備庫:192.168.0.24 1)基於數據庫過濾 #+++++++++++++++++++++主庫++++++++++++++++++++++++++# ###修改主庫配置文件### ~]# vim /etc/my.cnf [mysqld] ... server-id=1 log-bin=master-bin ~]# systemctl start mariadb.service ###建立複製帳號### mysql> grant replication slave,replication client on *.* to 'repluser'@'192.168.0.%' identified by 'replpass'; mysql> flush privileges; mysql> show master status; #查看主庫當前的日誌pos +------------------+----------+--------------+------------------+ | File | Position | Binlog_Do_DB | Binlog_Ignore_DB | +------------------+----------+--------------+------------------+ | mysql_bin.000009 | 497 | | | +------------------+----------+--------------+------------------+ #+++++++++++++++++++++備庫++++++++++++++++++++++++++# ###修改備庫配置文件### ~]# vim /etc/my.cnf [mysqld] ... server-id=2 replicate-do-db=mydb #表示和哪一個數據庫相關的寫入類、修改類指令會被寫入 replicate-ignore-db=testdb #表示忽略(黑名單) ~]# systemctl start mariadb.service ###配置slave參數### mysql> change master to master_host='192.168.0.23',master_user='repluser',master_password='replpass',master_log_file='mysql_bin.000009',master_log_pos=497; mysql> start slave; mysql> show slave status\G; mysql> show global variables like 'replicate%'; +----------------------------------+-----------+ | Variable_name | Value | +----------------------------------+-----------+ | replicate_annotate_row_events | OFF | | replicate_do_db | mydb | | replicate_do_table | | | replicate_events_marked_for_skip | replicate | | replicate_ignore_db | testdb | | replicate_ignore_table | | | replicate_wild_do_table | | | replicate_wild_ignore_table | | +----------------------------------+-----------+
2)基於表過濾
步驟與基於庫一致,只是在修改備庫參數文件時使用:
replication-do-table=TABLE_NAME
replication-ignore-table=TABLE_NAME
3)基於表的通配符
步驟與基於庫一致,只是在修改備庫參數文件時使用:
replicate-wild-do-table=PATTERNreplicate-wild-ignore-table=PATTERN