gtid_executed和gtid_purged變量是如何初始化的
1、官方釋義
1.一、gtid_executed、gtid_purged
https://dev.mysql.com/doc/refman/5.7/en/replication-options-gtids.html#sysvar_gtid_executed
• gtid_executed
When used with global scope, this variable contains a representation of the set of all transactions executed on the server and GTIDs that have been set by a SET gtid_purged statement. This is the same as the value of the Executed_Gtid_Set column in the output of SHOW MASTER STATUS and SHOW SLAVE STATUS.
• gtid_purged
The set of all transactions that have been purged from the binary log. This is a subset of the set of transactions in gtid_executed.
gtid_executed(global)：MySQL數據庫已經執行過的Gtid事務，處於內存中。show master status/show slave status中的Executed_Gtid_Set也取自這裏
gtid_purged(global)：因爲binlog文件的刪除(如purge binary logs或者超過expire_logs_days設置)已經丟失的Gtid事務，它是gtid_executed的子集html
1.二、binlog_gtid_simple_recovery
https://dev.mysql.com/doc/refman/5.7/en/replication-options-gtids.html#sysvar_binlog_gtid_simple_recovery
binlog_gtid_simple_recovery=FALSE
• To initialize gtid_executed, binary log files are iterated from the newest file, stopping at the first binary log that has any Previous_gtids_log_event. All GTIDs from Previous_gtids_log_event and Gtid_log_events are read from this binary log file. This GTID set is stored internally and called gtids_in_binlog. The value of gtid_executed is computed as the union of this set and the GTIDs stored in the mysql.gtid_executed table.
This process could take a long time if you had a large number of binary log files without GTID events, for example created when gtid_mode=OFF.
• To initialize gtid_purged, binary log files are iterated from the oldest to the newest, stopping at the first binary log that contains either a Previous_gtids_log_event that is non-empty (that has at least one GTID) or that has at least one Gtid_log_event. From this binary log it reads Previous_gtids_log_event. This GTID set is subtracted from gtids_in_binlog and the result stored in the internal variable gtids_in_binlog_not_purged. The value of gtid_purged is initialized to the value of gtid_executed, minus gtids_in_binlog_not_purged.
binlog_gtid_simple_recovery=TRUE
which is the default in MySQL 5.7.7 and later, the server iterates only the oldest and the newest binary log files and the values of gtid_purged and gtid_executed are computed based only on Previous_gtids_log_event or Gtid_log_event found in these files. This ensures only two binary log files are iterated during server restart or when binary logs are being purged.
數據庫服務啓動時，gtid_executed和gtid_purged按下面方式初始化
binlog_gtid_simple_recovery=FALSE
• gtid_executed：從mysql-bin.index的末行往首行所對應的binlog查找，直到首個被找到包含Previous_gtids_log_event的binlog。而後讀取這個binlog的Previous_gtids_log_event和Gtid_log_events中的全部Gtid集合保存到內部變量gtids_in_binlog。而後使用gtids_in_binlog和mysql.gtid_executed表的並集初始化gtid_executed變量
若是你有大量非GTID的binlog(好比gtid_mode=off的狀況下建立)，初始化gtid_executed的過程會消耗較長的時間
• gtid_purged：從mysql-bin.index的首行往末行所對應的binlog查找，直到首個被找到包含非空Previous_gtids_log_event或者Gtid_log_event的binlog。而後讀取這個binlog的Previous_gtids_log_event，將gtids_in_binlog - Previous_gtids_log_event獲得的集合保存到內部變量gtids_in_binlog_not_purged。最後使用gtid_executed - gtids_in_binlog_not_purged初始化gtid_purged變量
binlog_gtid_simple_recovery=TRUE(MySQL5.7.7及以上默認)
只迭代mysql-bin.index的首行和末行所對應的binlog，gtid_executed和gtid_purged的值就是取這兩個binlog中的Previous_gtids_log_event/Gtid_log_event計算，固然gtid_executed變量的值還要結合mysql.gtid_executedmysql
1.三、mysql.gtid_executed
https://dev.mysql.com/doc/refman/5.7/en/replication-gtids-concepts.html#replication-gtids-gtid-executed-table
GTIDs are stored in the mysql.gtid_executed table only when gtid_mode is ON or ON_PERMISSIVE. The point at which GTIDs are stored depends on whether binary logging is enabled or disabled:
• If binary logging is disabled (log_bin is OFF), or if log_slave_updates is disabled, the server stores the GTID belonging to each transaction together with the transaction in the table. In addition, the table is compressed periodically at a user-configurable rate; see mysql.gtid_executed Table Compression, for more information. This situation can only apply on a replication slave where binary logging or slave update logging is disabled. It does not apply on a replication master, because on a master, binary logging must be enabled for replication to take place.
• If binary logging is enabled (log_bin is ON), whenever the binary log is rotated or the server is shutdown, the server writes GTIDs for all transactions that were written into the previous binary log into the mysql.gtid_executed table. This situation applies on a replication master, or a replication slave where binary logging is enabled.
In the event of the server stopping unexpectedly, the set of GTIDs from the current binary log is not saved in the mysql.gtid_executed table. In this case, these GTIDs are added to the table and to the set of GTIDs in the gtid_executed system variable during recovery.
When binary logging is enabled, the mysql.gtid_executed table does not provide a complete record of the GTIDs for all executed transactions. That information is provided by the global value of the gtid_executed system variable.
若是沒有開啓log_bin或者沒有開啓log_slave_updates，從庫在應用relay-log中的每一個事務會執行一個insert mysql.gtid_executed操做。這隻針對從庫而言~
若是開啓log_bin，在binlog發生rotate(flush binary logs/達到max_binlog_size)或者關閉服務時，會把全部寫入到binlog中的Gtid信息寫入到mysql.gtid_executed表。這適用於主庫和從庫~
例子：從庫log_bin=on，log_slave_updates=off，那麼在應用relay-log時會實時寫入mysql.gtid_executed，而在從庫直接寫入數據，須要等到發生rotate或者關閉服務才寫入~
mysql.gtid_executed壓縮：log-bin=off，每gtid_executed_compression_period壓縮一次；log-bin=on，日誌切換時壓縮~
若是發生異常crash，當前binlog中的Gtids信息沒能寫入到mysql.gtid_executed表，在恢復過程經過讀取binlog中的Previous_gtids_log_event/Gtid_log_event信息把這些Gtids添加到mysql.gtid_executed表和gtid_executed系統變量sql
2、實驗
基本環境：官方社區版MySQL 5.7.19數據庫
[mysqld] gtid-mode = on binlog_gtid_simple_recovery = true log-bin = /data/mysql/mysql3306/logs/mysql-bin
2.一、the oldest and newest file
前面初始化gtid_executed和gtid_purged時需迭代the newest和the oldest file. 最新和最舊分別對應的是哪一個binary log？
2.1.一、binary log與index file一致app
# 置空全部Gtid信息 mydba@192.168.85.132,3308 [(none)]> reset master; Query OK, 0 rows affected (0.04 sec) # 關閉服務 mydba@192.168.85.132,3308 [(none)]> shutdown; # 將另外一實例下的binlog拷貝到當前實例的/data/mysql/mysql3308/logs/目錄，並修改文件屬主 # binlog列表 [root@ZST1 logs]# ll total 32 -rw-r-----. 1 mysql mysql 2012 Jan 19 15:33 mysql-bin.000202 -rw-r-----. 1 mysql mysql 1528 Jan 19 15:33 mysql-bin.000203 -rw-r-----. 1 mysql mysql 545 Jan 19 15:33 mysql-bin.000204 -rw-r-----. 1 mysql mysql 873 Jan 19 15:33 mysql-bin.000205 -rw-r-----. 1 mysql mysql 217 Jan 19 15:33 mysql-bin.000206 -rw-r-----. 1 mysql mysql 1056 Jan 19 15:33 mysql-bin.000207 -rw-r-----. 1 mysql mysql 545 Jan 19 15:33 mysql-bin.000208 -rw-r--r--. 1 mysql mysql 308 Jan 19 15:40 mysql-bin.index [root@ZST1 logs]# cat mysql-bin.index /data/mysql/mysql3308/logs/mysql-bin.000202 /data/mysql/mysql3308/logs/mysql-bin.000203 /data/mysql/mysql3308/logs/mysql-bin.000204 /data/mysql/mysql3308/logs/mysql-bin.000205 /data/mysql/mysql3308/logs/mysql-bin.000206 /data/mysql/mysql3308/logs/mysql-bin.000207 /data/mysql/mysql3308/logs/mysql-bin.000208 [root@ZST1 logs]# # 啓動後登陸 mydba@192.168.85.132,3308 [(none)]> show binary logs; +------------------+-----------+ | Log_name | File_size | +------------------+-----------+ | mysql-bin.000202 | 2012 | | mysql-bin.000203 | 1528 | | mysql-bin.000204 | 545 | | mysql-bin.000205 | 873 | | mysql-bin.000206 | 217 | | mysql-bin.000207 | 1056 | | mysql-bin.000208 | 545 | | mysql-bin.000209 | 194 | +------------------+-----------+ 8 rows in set (0.00 sec) # 查看mysql.gtid_executed mydba@192.168.85.132,3308 [(none)]> select * from mysql.gtid_executed; +--------------------------------------+----------------+--------------+ | source_uuid | interval_start | interval_end | +--------------------------------------+----------------+--------------+ | 8ab82362-9c37-11e7-a858-000c29c1025c | 1 | 507528 | +--------------------------------------+----------------+--------------+ 1 row in set (0.00 sec) # 查看當前位置 mydba@192.168.85.132,3308 [(none)]> show master status; +------------------+----------+--------------+------------------+-----------------------------------------------+ | File | Position | Binlog_Do_DB | Binlog_Ignore_DB | Executed_Gtid_Set | +------------------+----------+--------------+------------------+-----------------------------------------------+ | mysql-bin.000209 | 194 | | | 8ab82362-9c37-11e7-a858-000c29c1025c:1-507528 | +------------------+----------+--------------+------------------+-----------------------------------------------+ 1 row in set (0.00 sec) # 查看gtid_purged mydba@192.168.85.132,3308 [(none)]> show variables like 'gtid_purged'; +---------------+-----------------------------------------------+ | Variable_name | Value | +---------------+-----------------------------------------------+ | gtid_purged | 8ab82362-9c37-11e7-a858-000c29c1025c:1-507510 | +---------------+-----------------------------------------------+ 1 row in set (0.04 sec)
解析mysql-bin.00020八、mysql-bin.000202運維
# 解析mysql-bin.000208 [root@ZST1 logs]# mysqlbinlog -vv --base64-output=decode-rows mysql-bin.000208 |more ... #180119 9:27:10 server id 1323306 end_log_pos 194 CRC32 0xed94846f Previous-GTIDs # 8ab82362-9c37-11e7-a858-000c29c1025c:1-507527 # at 194 #180119 9:41:15 server id 1323306 end_log_pos 259 CRC32 0x8b964e9a GTID last_committed=0 sequence_number=1 rbr_only=yes /*!50718 SET TRANSACTION ISOLATION LEVEL READ COMMITTED*//*!*/; SET @@SESSION.GTID_NEXT= '8ab82362-9c37-11e7-a858-000c29c1025c:507528'/*!*/; # at 259 #180119 9:41:15 server id 1323306 end_log_pos 344 CRC32 0x15e5bb70 Query thread_id=7 exec_time=0 error_code=0 SET TIMESTAMP=1516326075/*!*/; BEGIN /*!*/; # at 344 #180119 9:41:15 server id 1323306 end_log_pos 407 CRC32 0x16207d9b Table_map: `replcrash`.`py_user` mapped to number 253 # at 407 #180119 9:41:15 server id 1323306 end_log_pos 491 CRC32 0x4fedbc9d Write_rows: table id 253 flags: STMT_END_F ### INSERT INTO `replcrash`.`py_user` ### SET ### @1=3 /* INT meta=0 nullable=0 is_null=0 */ ### @2='d6366f0e-fcb9-11e7-ad55-000c29' /* VARSTRING(96) meta=96 nullable=1 is_null=0 */ ### @3='2018-01-19 09:41:15' /* DATETIME(0) meta=0 nullable=1 is_null=0 */ ### @4='1323306' /* VARSTRING(30) meta=30 nullable=1 is_null=0 */ # at 491 #180119 9:41:15 server id 1323306 end_log_pos 522 CRC32 0x46352a20 Xid = 55 COMMIT/*!*/; # at 522 #180119 10:31:53 server id 1323306 end_log_pos 545 CRC32 0x6e53eee8 Stop SET @@SESSION.GTID_NEXT= 'AUTOMATIC' /* added by mysqlbinlog */ /*!*/; DELIMITER ; # End of log file /*!50003 SET COMPLETION_TYPE=@OLD_COMPLETION_TYPE*/; /*!50530 SET @@SESSION.PSEUDO_SLAVE_MODE=0*/; [root@ZST1 logs]# gtids_in_binlog：[1-507528] mysql.gtid_executed：空 gtid_executed變量：gtids_in_binlog ∪ mysql.gtid_executed = [1-507528] # 解析mysql-bin.000202 [root@ZST1 logs]# mysqlbinlog -vv --base64-output=decode-rows mysql-bin.000202 |more /*!50530 SET @@SESSION.PSEUDO_SLAVE_MODE=1*/; /*!50003 SET @OLD_COMPLETION_TYPE=@@COMPLETION_TYPE,COMPLETION_TYPE=0*/; DELIMITER /*!*/; # at 4 #180115 15:11:02 server id 1323306 end_log_pos 123 CRC32 0xbb10fc9f Start: binlog v 4, server v 5.7.19-log created 180115 15:11:02 at startup ROLLBACK/*!*/; # at 123 #180115 15:11:02 server id 1323306 end_log_pos 194 CRC32 0x1e8fe6f0 Previous-GTIDs # 8ab82362-9c37-11e7-a858-000c29c1025c:1-507510 # at 194 ... gtids_in_binlog：[1-507528] oldest Previous_gtids_log_event：[1-507510] gtids_in_binlog_not_purged：gtids_in_binlog - oldest Previous_gtids_log_event = [1-507528] - [1-507510] = [507511-507528] gtid_purged變量：gtid_executed - gtids_in_binlog_not_purged = [1-507528] - [507511-507528] = [1-507510]
binlog解析獲得的數值和庫中查詢結果一致~
2.1.二、binary log與index file不一致ide
# 置空全部Gtid信息 mydba@192.168.85.132,3308 [(none)]> reset master; Query OK, 0 rows affected (0.04 sec) # 關閉服務 mydba@192.168.85.132,3308 [(none)]> shutdown; # 將另外一實例下的binlog拷貝到當前實例的/data/mysql/mysql3308/logs/目錄，並修改文件屬主 # 並調整mysql-bin.index中的順序 # binlog列表 [root@ZST1 logs]# ll total 32 -rw-r-----. 1 mysql mysql 2012 Jan 19 17:11 mysql-bin.000202 -rw-r-----. 1 mysql mysql 1528 Jan 19 17:11 mysql-bin.000203 -rw-r-----. 1 mysql mysql 545 Jan 19 17:11 mysql-bin.000204 -rw-r-----. 1 mysql mysql 873 Jan 19 17:11 mysql-bin.000205 -rw-r-----. 1 mysql mysql 217 Jan 19 17:11 mysql-bin.000206 -rw-r-----. 1 mysql mysql 1056 Jan 19 17:11 mysql-bin.000207 -rw-r-----. 1 mysql mysql 545 Jan 19 17:11 mysql-bin.000208 -rw-r-----. 1 mysql mysql 308 Jan 19 17:12 mysql-bin.index [root@ZST1 logs]# cat mysql-bin.index /data/mysql/mysql3308/logs/mysql-bin.000203 /data/mysql/mysql3308/logs/mysql-bin.000202 /data/mysql/mysql3308/logs/mysql-bin.000204 /data/mysql/mysql3308/logs/mysql-bin.000205 /data/mysql/mysql3308/logs/mysql-bin.000206 /data/mysql/mysql3308/logs/mysql-bin.000208 /data/mysql/mysql3308/logs/mysql-bin.000207 [root@ZST1 logs]# # 啓動後登陸 mydba@192.168.85.132,3308 [(none)]> show binary logs; +------------------+-----------+ | Log_name | File_size | +------------------+-----------+ | mysql-bin.000203 | 1528 | | mysql-bin.000202 | 2012 | | mysql-bin.000204 | 545 | | mysql-bin.000205 | 873 | | mysql-bin.000206 | 217 | | mysql-bin.000208 | 545 | | mysql-bin.000207 | 1056 | | mysql-bin.000209 | 194 | +------------------+-----------+ 8 rows in set (0.00 sec) # 查看mysql.gtid_executed mydba@192.168.85.132,3308 [(none)]> select * from mysql.gtid_executed; +--------------------------------------+----------------+--------------+ | source_uuid | interval_start | interval_end | +--------------------------------------+----------------+--------------+ | 8ab82362-9c37-11e7-a858-000c29c1025c | 1 | 507527 | +--------------------------------------+----------------+--------------+ 1 row in set (0.00 sec) # 查看當前位置 mydba@192.168.85.132,3308 [(none)]> show master status; +------------------+----------+--------------+------------------+-----------------------------------------------+ | File | Position | Binlog_Do_DB | Binlog_Ignore_DB | Executed_Gtid_Set | +------------------+----------+--------------+------------------+-----------------------------------------------+ | mysql-bin.000209 | 194 | | | 8ab82362-9c37-11e7-a858-000c29c1025c:1-507527 | +------------------+----------+--------------+------------------+-----------------------------------------------+ 1 row in set (0.00 sec) # 查看gtid_purged mydba@192.168.85.132,3308 [(none)]> show variables like 'gtid_purged'; +---------------+-----------------------------------------------+ | Variable_name | Value | +---------------+-----------------------------------------------+ | gtid_purged | 8ab82362-9c37-11e7-a858-000c29c1025c:1-507517 | +---------------+-----------------------------------------------+ 1 row in set (0.01 sec)
解析mysql-bin.00020七、mysql-bin.000203學習
# 解析mysql-bin.000207 [root@ZST1 logs]# mysqlbinlog -vv --base64-output=decode-rows mysql-bin.000207 |more ... COMMIT/*!*/; # at 681 #180119 9:26:55 server id 1323306 end_log_pos 746 CRC32 0x91996947 GTID last_committed=2 sequence_number=3 rbr_only=yes /*!50718 SET TRANSACTION ISOLATION LEVEL READ COMMITTED*//*!*/; SET @@SESSION.GTID_NEXT= '8ab82362-9c37-11e7-a858-000c29c1025c:507527'/*!*/; # at 746 #180119 9:26:55 server id 1323306 end_log_pos 831 CRC32 0x41cdb52a Query thread_id=7 exec_time=0 error_code=0 SET TIMESTAMP=1516325215/*!*/; BEGIN /*!*/; # at 831 #180119 9:26:55 server id 1323306 end_log_pos 894 CRC32 0xfcf11bdb Table_map: `replcrash`.`py_user` mapped to number 253 # at 894 #180119 9:26:55 server id 1323306 end_log_pos 978 CRC32 0x0550d2ab Write_rows: table id 253 flags: STMT_END_F ### INSERT INTO `replcrash`.`py_user` ### SET ### @1=2 /* INT meta=0 nullable=0 is_null=0 */ ### @2='d5986a46-fcb7-11e7-ad55-000c29' /* VARSTRING(96) meta=96 nullable=1 is_null=0 */ ### @3='2018-01-19 09:26:55' /* DATETIME(0) meta=0 nullable=1 is_null=0 */ ### @4='1323306' /* VARSTRING(30) meta=30 nullable=1 is_null=0 */ # at 978 #180119 9:26:55 server id 1323306 end_log_pos 1009 CRC32 0x15796ccd Xid = 51 COMMIT/*!*/; # at 1009 #180119 9:27:10 server id 1323306 end_log_pos 1056 CRC32 0xaf26375b Rotate to mysql-bin.000208 pos: 4 SET @@SESSION.GTID_NEXT= 'AUTOMATIC' /* added by mysqlbinlog */ /*!*/; DELIMITER ; # End of log file /*!50003 SET COMPLETION_TYPE=@OLD_COMPLETION_TYPE*/; /*!50530 SET @@SESSION.PSEUDO_SLAVE_MODE=0*/; [root@ZST1 logs]# gtids_in_binlog：[1-507527] mysql.gtid_executed：空 gtid_executed變量：gtids_in_binlog ∪ mysql.gtid_executed = [1-507527] # 解析mysql-bin.000203 [root@ZST1 logs]# mysqlbinlog -vv --base64-output=decode-rows mysql-bin.000203 |more /*!50530 SET @@SESSION.PSEUDO_SLAVE_MODE=1*/; /*!50003 SET @OLD_COMPLETION_TYPE=@@COMPLETION_TYPE,COMPLETION_TYPE=0*/; DELIMITER /*!*/; # at 4 #180117 9:16:20 server id 1323306 end_log_pos 123 CRC32 0xd61e82fe Start: binlog v 4, server v 5.7.19-log created 180117 9:16:20 at startup ROLLBACK/*!*/; # at 123 #180117 9:16:20 server id 1323306 end_log_pos 194 CRC32 0x962d8c48 Previous-GTIDs # 8ab82362-9c37-11e7-a858-000c29c1025c:1-507517 # at 194 ... gtids_in_binlog：[1-507527] oldest Previous_gtids_log_event：[1-507517] gtids_in_binlog_not_purged：gtids_in_binlog - oldest Previous_gtids_log_event = [1-507527] - [1-507517] = [507518-507527] gtid_purged變量：gtid_executed - gtids_in_binlog_not_purged = [1-507527] - [507518-507527] = [1-507517]
binlog解析獲得的數值和庫中查詢結果一致~
上面的結果說明mysql.index中的首行和末行對應的就是oldest和newest測試
2.二、gtid_executed是來自gtids_in_binlog和mysql.gtid_executed的並集
前面的例子在服務啓動時，讀取最新的binlog中的Previous_gtids_log_event/Gtid_log_event，使用這些Gtid信息初始化gtid_executed，並將其寫入到mysql.gtid_executed表
此時再也不重置Gtid信息，而是繼續縮小mysql-bin.index首行和末行的範圍ui
# 關閉服務 mydba@192.168.85.132,3308 [(none)]> shutdown; # 將另外一實例下的binlog拷貝到當前實例的/data/mysql/mysql3308/logs/目錄，並修改文件屬主 # 調整mysql-bin.index文件中列表順序 # binlog列表 [root@ZST1 logs]# ll total 32 -rw-r-----. 1 mysql mysql 2012 Jan 19 17:11 mysql-bin.000202 -rw-r-----. 1 mysql mysql 1528 Jan 19 17:11 mysql-bin.000203 -rw-r-----. 1 mysql mysql 545 Jan 19 17:11 mysql-bin.000204 -rw-r-----. 1 mysql mysql 873 Jan 19 17:11 mysql-bin.000205 -rw-r-----. 1 mysql mysql 217 Jan 19 17:11 mysql-bin.000206 -rw-r-----. 1 mysql mysql 1056 Jan 19 17:11 mysql-bin.000207 -rw-r-----. 1 mysql mysql 545 Jan 19 17:11 mysql-bin.000208 -rw-r-----. 1 mysql mysql 308 Jan 19 17:31 mysql-bin.index [root@ZST1 logs]# cat mysql-bin.index /data/mysql/mysql3308/logs/mysql-bin.000204 /data/mysql/mysql3308/logs/mysql-bin.000203 /data/mysql/mysql3308/logs/mysql-bin.000202 /data/mysql/mysql3308/logs/mysql-bin.000205 /data/mysql/mysql3308/logs/mysql-bin.000207 /data/mysql/mysql3308/logs/mysql-bin.000208 /data/mysql/mysql3308/logs/mysql-bin.000206 [root@ZST1 logs]# # 啓動後登陸 mydba@192.168.85.132,3308 [(none)]> show binary logs; +------------------+-----------+ | Log_name | File_size | +------------------+-----------+ | mysql-bin.000204 | 545 | | mysql-bin.000203 | 1528 | | mysql-bin.000202 | 2012 | | mysql-bin.000205 | 873 | | mysql-bin.000207 | 1056 | | mysql-bin.000208 | 545 | | mysql-bin.000206 | 217 | | mysql-bin.000209 | 194 | +------------------+-----------+ 8 rows in set (0.03 sec) # 查看mysql.gtid_executed mydba@192.168.85.132,3308 [(none)]> select * from mysql.gtid_executed; +--------------------------------------+----------------+--------------+ | source_uuid | interval_start | interval_end | +--------------------------------------+----------------+--------------+ | 8ab82362-9c37-11e7-a858-000c29c1025c | 1 | 507527 | +--------------------------------------+----------------+--------------+ 1 row in set (0.00 sec) # 查看當前位置 mydba@192.168.85.132,3308 [(none)]> show master status; +------------------+----------+--------------+------------------+-----------------------------------------------+ | File | Position | Binlog_Do_DB | Binlog_Ignore_DB | Executed_Gtid_Set | +------------------+----------+--------------+------------------+-----------------------------------------------+ | mysql-bin.000209 | 194 | | | 8ab82362-9c37-11e7-a858-000c29c1025c:1-507527 | +------------------+----------+--------------+------------------+-----------------------------------------------+ 1 row in set (0.00 sec) # 查看gtid_purged mydba@192.168.85.132,3308 [(none)]> show variables like 'gtid_purged'; +---------------+-------------------------------------------------------------+ | Variable_name | Value | +---------------+-------------------------------------------------------------+ | gtid_purged | 8ab82362-9c37-11e7-a858-000c29c1025c:1-507521:507525-507527 | +---------------+-------------------------------------------------------------+ 1 row in set (0.00 sec)
解析mysql-bin.00020六、mysql-bin.000204
# 解析mysql-bin.000206 [root@ZST1 logs]# mysqlbinlog -vv --base64-output=decode-rows mysql-bin.000206 |more /*!50530 SET @@SESSION.PSEUDO_SLAVE_MODE=1*/; /*!50003 SET @OLD_COMPLETION_TYPE=@@COMPLETION_TYPE,COMPLETION_TYPE=0*/; DELIMITER /*!*/; # at 4 #180117 11:20:53 server id 1323306 end_log_pos 123 CRC32 0x17238df5 Start: binlog v 4, server v 5.7.19-log created 180117 11:20:53 at startup ROLLBACK/*!*/; # at 123 #180117 11:20:53 server id 1323306 end_log_pos 194 CRC32 0x3c03e609 Previous-GTIDs # 8ab82362-9c37-11e7-a858-000c29c1025c:1-507524 # at 194 #180117 18:09:00 server id 1323306 end_log_pos 217 CRC32 0x53b3b407 Stop SET @@SESSION.GTID_NEXT= 'AUTOMATIC' /* added by mysqlbinlog */ /*!*/; DELIMITER ; # End of log file /*!50003 SET COMPLETION_TYPE=@OLD_COMPLETION_TYPE*/; /*!50530 SET @@SESSION.PSEUDO_SLAVE_MODE=0*/; [root@ZST1 logs]# gtids_in_binlog：[1-507524] mysql.gtid_executed：[1-507527] gtid_executed變量：gtids_in_binlog ∪ mysql.gtid_executed = [1-507527] # 解析mysql-bin.000204 [root@ZST1 logs]# mysqlbinlog -vv --base64-output=decode-rows mysql-bin.000204 |more /*!50530 SET @@SESSION.PSEUDO_SLAVE_MODE=1*/; /*!50003 SET @OLD_COMPLETION_TYPE=@@COMPLETION_TYPE,COMPLETION_TYPE=0*/; DELIMITER /*!*/; # at 4 #180117 10:26:57 server id 1323306 end_log_pos 123 CRC32 0xee45d7bf Start: binlog v 4, server v 5.7.19-log created 180117 10:26:57 # at 123 #180117 10:26:57 server id 1323306 end_log_pos 194 CRC32 0xf8024ebe Previous-GTIDs # 8ab82362-9c37-11e7-a858-000c29c1025c:1-507521 # at 194 ... gtids_in_binlog：[1-507524] oldest Previous_gtids_log_event：[1-507521] gtids_in_binlog_not_purged：gtids_in_binlog - oldest Previous_gtids_log_event = [1-507524] - [1-507521] = [507522-507524] gtid_purged變量：gtid_executed - gtids_in_binlog_not_purged = [1-507527] - [507522-507524] = [1-507521:507525-507527]
上面的結果說明gtid_executed變量取的是gtids_in_binlog和mysql.gtid_executed的並集，並非單純來自某一個值~
2.三、binlog在Rotate或者關閉服務時，會把Gtid信息寫入到mysql.gtid_executed表
2.3.一、關閉服務
寫入一批數據，關閉服務，刪除binlog後再啓動服務，查看mysql.gtid_executed表
# 置空全部Gtid信息 mydba@192.168.85.132,3306 [replcrash]> reset master; Query OK, 0 rows affected (0.01 sec) # 清空測試數據表 mydba@192.168.85.132,3306 [replcrash]> truncate table py_user; Query OK, 0 rows affected (1.49 sec) # 寫入數據(執行兩次) mydba@192.168.85.132,3306 [replcrash]> insert into py_user(name,server_id) select left(uuid(),30),@@server_id; Query OK, 1 row affected (0.00 sec) Records: 1 Duplicates: 0 Warnings: 0 # 查看錶中數據 mydba@192.168.85.132,3306 [replcrash]> select * from py_user; +-----+--------------------------------+---------------------+-----------+ | uid | name | add_time | server_id | +-----+--------------------------------+---------------------+-----------+ | 1 | 5a57574a-0242-11e8-9655-000c29 | 2018-01-26 10:41:04 | 1323306 | | 2 | 65e773f6-0242-11e8-9655-000c29 | 2018-01-26 10:41:23 | 1323306 | +-----+--------------------------------+---------------------+-----------+ 2 rows in set (0.00 sec) # 查看mysql.gtid_executed mydba@192.168.85.132,3306 [replcrash]> select * from mysql.gtid_executed; Empty set (0.00 sec) # 查看當前位置 mydba@192.168.85.132,3306 [replcrash]> show master status; +------------------+----------+--------------+------------------+------------------------------------------+ | File | Position | Binlog_Do_DB | Binlog_Ignore_DB | Executed_Gtid_Set | +------------------+----------+--------------+------------------+------------------------------------------+ | mysql-bin.000001 | 969 | | | 60863f8d-01af-11e8-bfdf-000c29c1025c:1-3 | +------------------+----------+--------------+------------------+------------------------------------------+ 1 row in set (0.00 sec) # 查看gtid_purged mydba@192.168.85.132,3306 [replcrash]> show variables like 'gtid_purged'; +---------------+-------+ | Variable_name | Value | +---------------+-------+ | gtid_purged | | +---------------+-------+ 1 row in set (0.00 sec) # 關閉服務 mydba@192.168.85.132,3306 [replcrash]> shutdown; # 此時的binlog [root@ZST1 logs]# pwd /data/mysql/mysql3306/logs [root@ZST1 logs]# ll total 8 -rw-r----- 1 mysql mysql 992 Jan 26 10:43 mysql-bin.000001 -rw-r----- 1 mysql mysql 44 Jan 26 10:38 mysql-bin.index [root@ZST1 logs]# cat mysql-bin.index /data/mysql/mysql3306/logs/mysql-bin.000001 [root@ZST1 logs]# mysql-bin.000001 記錄 60863f8d-01af-11e8-bfdf-000c29c1025c:1-3 的信息 # 刪除binlog [root@ZST1 logs]# cp -r ../logs ../logs_bak [root@ZST1 logs]# rm -rf * # 啓動服務 [root@ZST1 logs]# sh ~/start3306.sh # 查看mysql.gtid_executed mydba@192.168.85.132,3306 [replcrash]> select * from mysql.gtid_executed; +--------------------------------------+----------------+--------------+ | source_uuid | interval_start | interval_end | +--------------------------------------+----------------+--------------+ | 60863f8d-01af-11e8-bfdf-000c29c1025c | 1 | 3 | +--------------------------------------+----------------+--------------+ 1 row in set (0.00 sec) # 查看當前位置 mydba@192.168.85.132,3306 [replcrash]> show master status; +------------------+----------+--------------+------------------+------------------------------------------+ | File | Position | Binlog_Do_DB | Binlog_Ignore_DB | Executed_Gtid_Set | +------------------+----------+--------------+------------------+------------------------------------------+ | mysql-bin.000001 | 154 | | | 60863f8d-01af-11e8-bfdf-000c29c1025c:1-3 | +------------------+----------+--------------+------------------+------------------------------------------+ 1 row in set (0.00 sec) # 查看gtid_purged mydba@192.168.85.132,3306 [replcrash]> show variables like 'gtid_purged'; +---------------+------------------------------------------+ | Variable_name | Value | +---------------+------------------------------------------+ | gtid_purged | 60863f8d-01af-11e8-bfdf-000c29c1025c:1-3 | +---------------+------------------------------------------+ 1 row in set (0.01 sec)
重啓前mysql.gtid_executed的Gtid爲空，關閉服務後刪除全部binlog，重啓後mysql.gtid_executed的Gtid:1-3。它的信息不可能來自binlog，只有可能在關閉服務時Gtid已寫入到mysql.gtid_executed
2.3.二、flush binary logs
寫入一批數據，flush binary logs，查看mysql.gtid_executed表，並解析binlog日誌
# 寫入數據(執行兩次) mydba@192.168.85.132,3306 [replcrash]> insert into py_user(name,server_id) select left(uuid(),30),@@server_id; Query OK, 1 row affected (0.04 sec) Records: 1 Duplicates: 0 Warnings: 0 # 查看mysql.gtid_executed mydba@192.168.85.132,3306 [replcrash]> select * from mysql.gtid_executed; +--------------------------------------+----------------+--------------+ | source_uuid | interval_start | interval_end | +--------------------------------------+----------------+--------------+ | 60863f8d-01af-11e8-bfdf-000c29c1025c | 1 | 3 | +--------------------------------------+----------------+--------------+ 1 row in set (0.00 sec) # 查看當前位置 mydba@192.168.85.132,3306 [replcrash]> show master status; +------------------+----------+--------------+------------------+------------------------------------------+ | File | Position | Binlog_Do_DB | Binlog_Ignore_DB | Executed_Gtid_Set | +------------------+----------+--------------+------------------+------------------------------------------+ | mysql-bin.000001 | 810 | | | 60863f8d-01af-11e8-bfdf-000c29c1025c:1-5 | +------------------+----------+--------------+------------------+------------------------------------------+ 1 row in set (0.00 sec) # 查看mysql.gtid_purged mydba@192.168.85.132,3306 [replcrash]> show variables like 'gtid_purged'; +---------------+------------------------------------------+ | Variable_name | Value | +---------------+------------------------------------------+ | gtid_purged | 60863f8d-01af-11e8-bfdf-000c29c1025c:1-3 | +---------------+------------------------------------------+ 1 row in set (0.01 sec) 寫入數據，mysql.gtid_executed表並不會時時更新 # 切換日誌 mydba@192.168.85.132,3306 [replcrash]> flush binary logs; Query OK, 0 rows affected (0.01 sec) # 查看mysql.gtid_executed mydba@192.168.85.132,3306 [replcrash]> select * from mysql.gtid_executed; +--------------------------------------+----------------+--------------+ | source_uuid | interval_start | interval_end | +--------------------------------------+----------------+--------------+ | 60863f8d-01af-11e8-bfdf-000c29c1025c | 1 | 5 | +--------------------------------------+----------------+--------------+ 1 row in set (0.00 sec) # 查看當前位置 mydba@192.168.85.132,3306 [replcrash]> show master status; +------------------+----------+--------------+------------------+------------------------------------------+ | File | Position | Binlog_Do_DB | Binlog_Ignore_DB | Executed_Gtid_Set | +------------------+----------+--------------+------------------+------------------------------------------+ | mysql-bin.000002 | 194 | | | 60863f8d-01af-11e8-bfdf-000c29c1025c:1-5 | +------------------+----------+--------------+------------------+------------------------------------------+ 1 row in set (0.00 sec) # 查看mysql.gtid_purged mydba@192.168.85.132,3306 [replcrash]> show variables like 'gtid_purged'; +---------------+------------------------------------------+ | Variable_name | Value | +---------------+------------------------------------------+ | gtid_purged | 60863f8d-01af-11e8-bfdf-000c29c1025c:1-3 | +---------------+------------------------------------------+ 1 row in set (0.01 sec) flush logs後gtid_purged變量的值立刻更新到mysql.gtid_executed表
mysql.gtid_executed表並非時時更新，flush logs後gtid_executed變量的值立刻更新到mysql.gtid_executed表
解析binlog日誌
# 解析切換後的日誌 [root@ZST1 logs]# ll total 12 -rw-r----- 1 mysql mysql 857 Jan 26 11:13 mysql-bin.000001 -rw-r----- 1 mysql mysql 194 Jan 26 11:13 mysql-bin.000002 -rw-r----- 1 mysql mysql 88 Jan 26 11:13 mysql-bin.index [root@ZST1 logs]# mysqlbinlog -vv --base64-output=decode-rows mysql-bin.000002 /*!50530 SET @@SESSION.PSEUDO_SLAVE_MODE=1*/; /*!50003 SET @OLD_COMPLETION_TYPE=@@COMPLETION_TYPE,COMPLETION_TYPE=0*/; DELIMITER /*!*/; # at 4 #180126 11:13:02 server id 1323306 end_log_pos 123 CRC32 0xea188782 Start: binlog v 4, server v 5.7.21-log created 180126 11:13:02 # Warning: this binlog is either in use or was not closed properly. # at 123 #180126 11:13:02 server id 1323306 end_log_pos 194 CRC32 0xdf719f1d Previous-GTIDs # 60863f8d-01af-11e8-bfdf-000c29c1025c:4-5 SET @@SESSION.GTID_NEXT= 'AUTOMATIC' /* added by mysqlbinlog */ /*!*/; DELIMITER ; # End of log file /*!50003 SET COMPLETION_TYPE=@OLD_COMPLETION_TYPE*/; /*!50530 SET @@SESSION.PSEUDO_SLAVE_MODE=0*/; [root@ZST1 logs]#
注意Rotate後的binlog中Previous-GTIDs是4-5 ，而不是1-5。由於Rotate只把binary log中的Gtid寫入到新binary log，同時更新mysql.gtid_executed表
2.四、如何經過binlog和mysql.gtid_executed獲得gtid_executed、gtid_purged
搞清楚服務啓動前mysql.gtid_executed的值。對於5.7.7及之後默認只需迭代最新和最舊binlog中的Previous_gtids_log_event/Gtid_log_event，計算gtids_in_binlog、gtids_in_binlog_not_purged，而後根據公式計算gtid_executed、gtid_purged變量。能夠查看前面解析binlog的例子
3、參考文檔
Global Transaction ID Options and Variables：https://dev.mysql.com/doc/refman/5.7/en/replication-options-gtids.html
MySQL 5.7 Gtid內部學習(五) mysql.gtid_executed表/gtid_executed變量/gtid_purged變量的更改時機：https://yq.aliyun.com/articles/294004
[MySQL 5.6] GTID實現、運維變化及存在的bug：http://www.cnblogs.com/MYSQLZOUQI/p/3850578.html
binlog rotate引起的MySQL阻塞事件：https://mp.weixin.qq.com/s/XSnFkuYzIlGWMaXIl-oPeQ
相關文章
- 1. 變量初始化
- 2. java變量的初始化
- 3. 初始化logback的變量
- 4. C++的變量初始化
- 5. 對象是如何初始化的（iOS）
- 6. scala 變量初始化
- 7. 指針變量初始化
- 8. 『TensorFlow』變量初始化
- 9. Java變量初始化
- 10. tensorflow2.x變量初始化
- 更多相關文章...
- • XSD 如何使用? - XML Schema 教程
- • 如何僞造ARP響應？ - TCP/IP教程
- • 適用於PHP初學者的學習線路和建議
- • IntelliJ IDEA 代碼格式化配置和快捷鍵
相關標籤/搜索
每日一句
-
每一个你不满意的现在，都有一个你没有努力的曾经。
最新文章
- 1. python的安裝和Hello，World編寫
- 2. 重磅解讀：K8s Cluster Autoscaler模塊及對應華爲雲插件Deep Dive
- 3. 鴻蒙學習筆記2（永不斷更）
- 4. static關鍵字 和構造代碼塊
- 5. JVM筆記
- 6. 無法啓動 C/C++ 語言服務器。IntelliSense 功能將被禁用。錯誤: Missing binary at c:\Users\MSI-NB\.vscode\extensions\ms-vsc
- 7. 【Hive】Hive返回碼狀態含義
- 8. Java樹形結構遞歸（以時間換空間）和非遞歸（以空間換時間）
- 9. 數據預處理---缺失值
- 10. 都要2021年了，現代C++有什麼值得我們學習的？
歡迎關注本站公眾號,獲取更多信息
相關文章
- 1. 變量初始化
- 2. java變量的初始化
- 3. 初始化logback的變量
- 4. C++的變量初始化
- 5. 對象是如何初始化的（iOS）
- 6. scala 變量初始化
- 7. 指針變量初始化
- 8. 『TensorFlow』變量初始化
- 9. Java變量初始化
- 10. tensorflow2.x變量初始化