做者：八怪（高鵬） 中亦科技數據庫專家mysql
1、問題拋出sql
最近遇到一個問題，獲得棧以下（5.6.25）：數據庫
出現這個問題的時候只存在一個讀寫事務，那就是本事務。對這裏的紅色部分比較感興趣，可是這裏不是全部的內容都和這個問題相關，主要仍是圍繞可見性判斷和隱式鎖斷定進行，算是個人思考過程。可是對Innodb認知水平有限，若有誤導請諒解。使用的源碼版本5.7.29。數組
2、read view 簡述
關於read view說明的文章已經不少了，我這裏簡單記錄一下我學習的地方。一致性讀取（consistent read），根據隔離級別的不一樣，會在不一樣的時機創建read view，以下：mvc
RR 事務的第一個select命令發起的時候創建read view，直到事務提交釋放函數
RC 事務的每個select都會單獨創建read view學習
有了read view 就可以對每行數據的可見性進行判斷了，下面是read view中的關鍵屬性測試
m_up_limit_id：若是行的trx id 小於了m_up_limit_id則不可見。優化
m_low_limit_id：若是行的trx id 大於了m_low_limit_id則可見。ui
m_ids：是用於記錄創建read view時刻的讀寫事務的vector數組，用於對於m_up_limit_id和m_low_limit_id之間的trx須要根據它來進行斷定，是否處於活躍狀態。
m_low_limit_no則用於記錄創建read view時刻的最小trx no，主要用於purge線程判斷清理undo使用。
如何拿到值得具體能夠參見附錄，而對於可見性的判斷咱們能夠參考以下函數:
/** Check whether the changes by id are visible.
@param[in] id transaction id to check against the view
@param[in] name table name
@return whether the view sees the modifications of id. */
bool changes_visible(
trx_id_t id,
const table_name_t& name) const
MY_ATTRIBUTE((warn_unused_result))
{
ut_ad(id > 0);
if (id < m_up_limit_id || id == m_creator_trx_id) { //小於 可見
return(true);
}
check_trx_id_sanity(id, name);
if (id >= m_low_limit_id) { //大於不可見
return(false);
} else if (m_ids.empty()) { //若是之間的 active 爲空 則可見
return(true);
}
const ids_t::value_type* p = m_ids.data();
return(!std::binary_search(p, p + m_ids.size(), id)); //不然比較本trx id 是否在這之中，若是在不能夠見，反之可見
}
3、關於可見性判斷的幾個問題
一、有大量的刪除行，且已經提交，可是沒有被purge線程清理
這種狀況因爲大量刪除行（或者update）而且已經提交，可是因爲有長時間的select語句致使read view記錄的狀態也比較陳舊，所以根據m_low_limit_no的判斷purge線程是不能清理一些比較老舊的undo的，所以這會致使一個問題，若是這些del flag的記錄會存在於邏輯記錄鏈表內部，所以其餘select掃描的時候回根據next offset掃描到，可是根據可見性判斷條件這些del flag的記錄trx id小於本select語句的read view 的 m_up_limit_id，所以是可見的debug以下：
387 return(view->changes_visible(trx_id, index->table->name)); (gdb) p view->changes_visible(trx_id, index->table->name) $14 = true
可是由於已經標記爲del flag所以會作跳過處理以下：
row_search_mvcc:
if (rec_get_deleted_flag(rec, comp)) {
/* The record is delete-marked: we can skip it */
...
goto next_rec;
也就是實際上在長時間read view的「保護」下，咱們的undo不能清理，而且del flag不能清理還保存在block的邏輯鏈表中，掃描的時候會實際掃描到，只是作了跳過處理。所以會出現以下現象
| T1 | T2 | T3 |
|---|---|---|
| select sleep(1000) from test(模擬長時間查詢) | ||
| begin;delete from test10;commit; | ||
| select * from test10;(時間仍是好久) |
這就是上面說的緣由，雖然沒有數據了，可是查詢依舊很慢。
二、大量刪除，還未提交
那麼select掃描的時候會根據next offset 掃描到，可是因爲read view 判斷這些數據的trx id 位於 m_up_limit_id和m_low_limit_id之間，須要根據事務是否活躍（read view的m_ids，顯然這裏是活躍的）經過undo構建其前印象，以下判斷：
lock_clust_rec_cons_read_sees trx_id_t trx_id = row_get_rec_trx_id(rec, index, offsets); return(view->changes_visible(trx_id, index->table->name));
三、using index也可能回表
咱們知道若是執行計劃使用到using index那麼不會回表去取主鍵的數據，使用整個二級索引便可。可是這裏有一種特殊狀況，這裏進行描述。
對於二級索引而言，由於row記錄不包含trx id和undo ptr兩個僞列，那麼其可見性判斷和前的印象構建均須要回表獲取主鍵的記錄，固然可見性判斷能夠先根據本二級索引page的max trx id是否小於read view的m_up_limit_id來進行第一次粗略過濾，那麼可見性判斷的可能性就低不少，若是經過了這個比對，那麼剩餘精確判斷仍是須要回表經過主鍵來比對才行，以下：
對於二級索引回表操做來說，精確的可見性判斷放到了回表後的lock_clust_rec_cons_read_sees函數上，關於二級索引的回表，參考附錄。
對於不回表訪問（using index），經過了粗略判斷後（lock_sec_rec_cons_read_sees），若是遇到須要精確的可見性判斷，那麼也是要回表的，緣由前面解釋了（row記錄不包含trx id和undo ptr），參考附錄。
對於這個問題咱們能夠簡單的作以下的測試，固然須要打斷點才行：
測試表以下： mysql> show create table testimp4 \G *************************** 1. row *************************** Table: testimp4 Create Table: CREATE TABLE `testimp4` ( `id` int(11) NOT NULL AUTO_INCREMENT, `a` int(11) DEFAULT NULL, `b` int(11) DEFAULT NULL, `d` varchar(200) DEFAULT NULL, PRIMARY KEY (`id`), KEY `b` (`b`), KEY `d` (`d`) ) ENGINE=InnoDB AUTO_INCREMENT=10000 DEFAULT CHARSET=utf8 1 row in set (0.00 sec) mysql> begin; Query OK, 0 rows affected (0.00 sec) mysql> select * from testimp4; +------+------+------+------------------------------------+ | id | a | b | d | +------+------+------+------------------------------------+ | 5 | 5 | 300 | NULL | | 6 | 7000 | 7700 | 1124 | | 11 | 7000 | 7700 | 1124 | | 12 | 7000 | 7700 | 1124 | | 13 | 2900 | 1800 | NULL | | 14 | 2900 | 1800 | NULL | | 1000 | 88 | 1499 | NULL | | 4000 | 6000 | 5904 | iiiafsafasfihhhccccchhhigggofgo111 | | 4001 | 7000 | 7700 | 1124454555 | | 9999 | 9999 | 9999 | a | +------+------+------+------------------------------------+ 10 rows in set (0.00 sec)
對於下列語句的執行話是：
mysql> desc select b from testimp4 where b=300; +----+-------------+----------+------------+------+---------------+------+---------+-------+------+----------+-------------+ | id | select_type | table | partitions | type | possible_keys | key | key_len | ref | rows | filtered | Extra | +----+-------------+----------+------------+------+---------------+------+---------+-------+------+----------+-------------+ | 1 | SIMPLE | testimp4 | NULL | ref | b | b | 5 | const | 1 | 100.00 | Using index | +----+-------------+----------+------------+------+---------------+------+---------+-------+------+----------+-------------+ 1 row in set, 1 warning (0.00 sec)
咱們作以下語句：
| T1 | T2 |
|---|---|
| begin;delete from testimp4 where id=5;（不提交） | |
| select b from testimp4 where b=300;（這裏是須要回表的） |
這裏顯然T2（5 ,5 ,300 ,NULL ）的這條記錄已經被T1刪除了，可是沒有提交，T2首先判斷二級索引b上這行數據所在的page其max trx id是否小於本select語句的read view的m_up_limit_id，顯然這不成立，由於T1還會處於活躍狀態，而後就進入了回表判斷流程。棧以下：
#0 lock_clust_rec_cons_read_sees (rec=0x7fff060980a8 "\200", index=0x7ffec0499330, offsets=0x7fffe8399a70, view=0x33b1368) at /home/mysql/soft/percona-server-5.7.29-32/storage/innobase/lock/lock0lock.cc:369 #1 0x0000000001afbca4 in Row_sel_get_clust_rec_for_mysql::operator() (this=0x7fffe839a2d0, prebuilt=0x7ffec80c97a0, sec_index=0x7ffec049a2c0, rec=0x7fff060a008c "\200", thr=0x7ffec80c9f88, out_rec=0x7fffe839a310, offsets=0x7fffe839a2e8, offset_heap=0x7fffe839a2f0, vrow=0x0, mtr=0x7fffe8399d90) at /home/mysql/soft/percona-server-5.7.29-32/storage/innobase/row/row0sel.cc:3763 #2 0x0000000001b00a94 in row_search_mvcc (buf=0x7ffec80c8a00 <incomplete sequence \375>, mode=PAGE_CUR_GE, prebuilt=0x7ffec80c97a0, match_mode=1, direction=0) at /home/mysql/soft/percona-server-5.7.29-32/storage/innobase/row/row0sel.cc:6051
四、關於page的max trx id
咱們上面屢次提到二級索引page的max trx id，這個max trx id實際就是PAGE_MAX_TRX_ID，它位於page的offset 56後的8個字節，實際上這個值只會存在於二級索引上，主鍵沒有這個值，咱們能夠看到以下：
#define PAGE_MAX_TRX_ID 18 /* highest id of a trx which may have modified a record on the page; trx_id_t; defined only in secondary indexes and in the insert buffer tree */ [root@mgr2 test]# ./bcview testimp2.ibd 16 56 8 ****************************************************************** This Tool Is Uesed For Find The Data In Binary format(Hexadecimal) Usage:./bcview file blocksize offset cnt-bytes! file: Is Your File Will To Find Data! blocksize: Is N kb Block.Eg: 8 Is 8 Kb Blocksize(Oracle)! Eg: 16 Is 16 Kb Blocksize(Innodb)! offset:Is Every Block Offset Your Want Start! cnt-bytes:Is After Offset,How Bytes Your Want Gets! Edtor QQ:22389860! Used gcc version 4.1.2 20080704 (Red Hat 4.1.2-46) ****************************************************************** ----Current file size is :0.125000 Mb ----Current use set blockszie is 16 Kb ----Current file name is testimp2.ibd current block:00000000--Offset:00056--cnt bytes:08--data is:0021000000060000 current block:00000001--Offset:00056--cnt bytes:08--data is:0000000000000000 current block:00000002--Offset:00056--cnt bytes:08--data is:0001000000000000 current block:00000003--Offset:00056--cnt bytes:08--data is:0000000000000000(主鍵沒有這個值) current block:00000004--Offset:00056--cnt bytes:08--data is:0000000000016903(二級索引) current block:00000005--Offset:00056--cnt bytes:08--data is:0000000000016924(二級索引)
每次每行更新後會更新這個值，若是大於則修改，小於則不變。函數page_update_max_trx_id中有以下片斷
if (page_get_max_trx_id(buf_block_get_frame(block)) < trx_id) { //是否本次事務的trx id大於page的max trx id
page_set_max_trx_id(block, page_zip, trx_id, mtr);
}
4、關於加鎖的階段
咱們通常鎖須要加鎖的都是DML語句和select for update這樣的語句，這裏將加鎖分爲數據查找和數據修改兩個階段。
對於select for update：
主鍵訪問數據：訪問主鍵判斷是否存在隱式鎖，而後加顯示鎖。二級索引訪問數據（須要回表的狀況）：訪問二級索引判斷是否存在隱式鎖，而後加顯示鎖，接着回表主鍵判斷是否存在隱式鎖，而後加顯示鎖。
對於update/delete：
主鍵訪問修改數據：數據查找階段主鍵判斷是否存在隱式鎖，而後加顯示鎖。數據修改階段涉及到了其餘二級索引，那麼維護相應的二級索引加隱含鎖。
二級索引訪問修改數據：數據查找階段二級索引判斷是否存在隱式鎖（可能須要回表判斷），二級索引加顯示鎖，數據修改階段回表修改主鍵數據加顯示鎖，而後維護各個二級索引（修改字段涉及的二級索引或者修改主鍵則包含所有二級索引）加隱式鎖。
對於insert而言若是沒有堵塞（插入印象鎖和gap lock堵塞），那麼始終爲隱式鎖。
注意這裏咱們看到了隱式鎖，隱式鎖不會佔用row的結構體，所以在show engine innodb status裏面是看不到的，除非有其餘事務顯示將其轉換爲顯示鎖。咱們來作幾個例子以下（REPEATABLE READ隔離級別）：
表結構和數據 mysql> show create table testimp4 \G *************************** 1. row *************************** Table: testimp4 Create Table: CREATE TABLE `testimp4` ( `id` int(11) NOT NULL AUTO_INCREMENT, `a` int(11) DEFAULT NULL, `b` int(11) DEFAULT NULL, `d` varchar(200) DEFAULT NULL, PRIMARY KEY (`id`), KEY `b` (`b`), KEY `d` (`d`) ) ENGINE=InnoDB AUTO_INCREMENT=10000 DEFAULT CHARSET=utf8 1 row in set (0.00 sec) mysql> select *from testimp4; +------+------+------+------------------------------------+ | id | a | b | d | +------+------+------+------------------------------------+ | 5 | 5 | 300 | NULL | | 6 | 7000 | 7700 | 1124 | | 11 | 7000 | 7700 | 1124 | | 12 | 7000 | 7700 | 1124 | | 13 | 2900 | 1800 | NULL | | 14 | 2900 | 1800 | NULL | | 1000 | 88 | 1499 | NULL | | 4000 | 6000 | 5904 | iiiafsafasfihhhccccchhhigggofgo111 | | 4001 | 7000 | 7700 | 1124454555 | | 9999 | 9999 | 9999 | a | +------+------+------+------------------------------------+ 10 rows in set (0.00 sec)
4.1 插入數據
begin；insert into testimp4 values(10000,10000,10000,'gp');（不提交）
| TIME | S1 | S2 | S3 | S4 |
|---|---|---|---|---|
| T1 | begin；insert into testimp4 values(10000,10000,10000,'gp');（不提交） | |||
| T2 | select * from testimp4 where id=10000 for update | |||
| T3 | select * from testimp4 where b=10000 for update | |||
| T4 | select * from testimp4 where d='a' for update |
# T1時刻S1鎖狀態： ---TRANSACTION 94487, ACTIVE 5 sec 1 lock struct(s), heap size 1160, 0 row lock(s), undo log entries 1 MySQL thread id 11, OS thread handle 140737089492736, query id 482 localhost root starting show engine innodb status TABLE LOCK table `test`.`testimp4` trx id 94487 lock mode IX # T2時刻S1鎖狀態： ---TRANSACTION 94487, ACTIVE 271 sec 2 lock struct(s), heap size 1160, 1 row lock(s), undo log entries 1 MySQL thread id 11, OS thread handle 140737089492736, query id 484 localhost root starting show engine innodb status TABLE LOCK table `test`.`testimp4` trx id 94487 lock mode IX RECORD LOCKS space id 501 page no 3 n bits 80 index PRIMARY of table `test`.`testimp4` trx id 94487 lock_mode X locks rec but not gap Record lock, heap no 2 PHYSICAL RECORD: n_fields 6; compact format; info bits 0 0: len 4; hex 80002710; asc ' ;; 1: len 6; hex 000000017117; asc q ;; 2: len 7; hex d0000002c40110; asc ;; 3: len 4; hex 80002710; asc ' ;; 4: len 4; hex 80002710; asc ' ;; 5: len 2; hex 6770; asc gp;; # T3時刻S1鎖狀態： ---TRANSACTION 94487, ACTIVE 337 sec 3 lock struct(s), heap size 1160, 2 row lock(s), undo log entries 1 MySQL thread id 11, OS thread handle 140737089492736, query id 521 localhost root starting show engine innodb status TABLE LOCK table `test`.`testimp4` trx id 94487 lock mode IX RECORD LOCKS space id 501 page no 3 n bits 80 index PRIMARY of table `test`.`testimp4` trx id 94487 lock_mode X locks rec but not gap Record lock, heap no 2 PHYSICAL RECORD: n_fields 6; compact format; info bits 0 0: len 4; hex 80002710; asc ' ;; 1: len 6; hex 000000017117; asc q ;; 2: len 7; hex d0000002c40110; asc ;; 3: len 4; hex 80002710; asc ' ;; 4: len 4; hex 80002710; asc ' ;; 5: len 2; hex 6770; asc gp;; RECORD LOCKS space id 501 page no 4 n bits 80 index b of table `test`.`testimp4` trx id 94487 lock_mode X locks rec but not gap Record lock, heap no 2 PHYSICAL RECORD: n_fields 2; compact format; info bits 0 0: len 4; hex 80002710; asc ' ;; 1: len 4; hex 80002710; asc ' ;; # T4時刻S1鎖狀態： ---TRANSACTION 94487, ACTIVE 408 sec 4 lock struct(s), heap size 1160, 3 row lock(s), undo log entries 1 MySQL thread id 11, OS thread handle 140737089492736, query id 559 localhost root starting show engine innodb status TABLE LOCK table `test`.`testimp4` trx id 94487 lock mode IX RECORD LOCKS space id 501 page no 3 n bits 80 index PRIMARY of table `test`.`testimp4` trx id 94487 lock_mode X locks rec but not gap Record lock, heap no 2 PHYSICAL RECORD: n_fields 6; compact format; info bits 0 0: len 4; hex 80002710; asc ' ;; 1: len 6; hex 000000017117; asc q ;; 2: len 7; hex d0000002c40110; asc ;; 3: len 4; hex 80002710; asc ' ;; 4: len 4; hex 80002710; asc ' ;; 5: len 2; hex 6770; asc gp;; RECORD LOCKS space id 501 page no 4 n bits 80 index b of table `test`.`testimp4` trx id 94487 lock_mode X locks rec but not gap Record lock, heap no 2 PHYSICAL RECORD: n_fields 2; compact format; info bits 0 0: len 4; hex 80002710; asc ' ;; 1: len 4; hex 80002710; asc ' ;; RECORD LOCKS space id 501 page no 5 n bits 80 index d of table `test`.`testimp4` trx id 94487 lock_mode X locks rec but not gap Record lock, heap no 2 PHYSICAL RECORD: n_fields 2; compact format; info bits 0 0: len 2; hex 6770; asc gp;; 1: len 4; hex 80002710; asc ' ;;
實際上咱們看到這裏insert語句後主鍵和各個索引都上了隱含鎖只是看不到，經過其餘S2,S3,S4咱們逐步把這些隱式鎖轉換爲了顯示鎖。
4.2 delete語句經過主鍵刪除數據
| TIME | S1 | S2 | S3 |
|---|---|---|---|
| T1 | begin；delete from testimp4 where id=9999;（不提交） | ||
| T2 | select * from testimp4 where b=9999 for update | ||
| T3 | select * from testimp4 where d='a' for update; |
# T1時刻S1鎖狀態： ---TRANSACTION 94493, ACTIVE 3 sec 2 lock struct(s), heap size 1160, 1 row lock(s), undo log entries 1 MySQL thread id 11, OS thread handle 140737089492736, query id 567 localhost root TABLE LOCK table `test`.`testimp4` trx id 94493 lock mode IX RECORD LOCKS space id 501 page no 3 n bits 80 index PRIMARY of table `test`.`testimp4` trx id 94493 lock_mode X locks rec but not gap Record lock, heap no 12 PHYSICAL RECORD: n_fields 6; compact format; info bits 32 0: len 4; hex 8000270f; asc ' ;; 1: len 6; hex 00000001711d; asc q ;; 2: len 7; hex 550000003b071b; asc U ; ;; 3: len 4; hex 8000270f; asc ' ;; 4: len 4; hex 8000270f; asc ' ;; 5: len 1; hex 61; asc a;; # T2時刻S1鎖狀態： ---TRANSACTION 94493, ACTIVE 112 sec 4 lock struct(s), heap size 1160, 3 row lock(s), undo log entries 1 MySQL thread id 11, OS thread handle 140737089492736, query id 567 localhost root TABLE LOCK table `test`.`testimp4` trx id 94493 lock mode IX RECORD LOCKS space id 501 page no 3 n bits 80 index PRIMARY of table `test`.`testimp4` trx id 94493 lock_mode X locks rec but not gap Record lock, heap no 12 PHYSICAL RECORD: n_fields 6; compact format; info bits 32 0: len 4; hex 8000270f; asc ' ;; 1: len 6; hex 00000001711d; asc q ;; 2: len 7; hex 550000003b071b; asc U ; ;; 3: len 4; hex 8000270f; asc ' ;; 4: len 4; hex 8000270f; asc ' ;; 5: len 1; hex 61; asc a;; RECORD LOCKS space id 501 page no 4 n bits 80 index b of table `test`.`testimp4` trx id 94493 lock_mode X locks rec but not gap Record lock, heap no 12 PHYSICAL RECORD: n_fields 2; compact format; info bits 32 0: len 4; hex 8000270f; asc ' ;; 1: len 4; hex 8000270f; asc ' ;; # T3時刻S1鎖狀態： ---TRANSACTION 94493, ACTIVE 133 sec 4 lock struct(s), heap size 1160, 3 row lock(s), undo log entries 1 MySQL thread id 11, OS thread handle 140737089492736, query id 567 localhost root TABLE LOCK table `test`.`testimp4` trx id 94493 lock mode IX RECORD LOCKS space id 501 page no 3 n bits 80 index PRIMARY of table `test`.`testimp4` trx id 94493 lock_mode X locks rec but not gap Record lock, heap no 12 PHYSICAL RECORD: n_fields 6; compact format; info bits 32 0: len 4; hex 8000270f; asc ' ;; 1: len 6; hex 00000001711d; asc q ;; 2: len 7; hex 550000003b071b; asc U ; ;; 3: len 4; hex 8000270f; asc ' ;; 4: len 4; hex 8000270f; asc ' ;; 5: len 1; hex 61; asc a;; RECORD LOCKS space id 501 page no 4 n bits 80 index b of table `test`.`testimp4` trx id 94493 lock_mode X locks rec but not gap Record lock, heap no 12 PHYSICAL RECORD: n_fields 2; compact format; info bits 32 0: len 4; hex 8000270f; asc ' ;; 1: len 4; hex 8000270f; asc ' ;; RECORD LOCKS space id 501 page no 5 n bits 80 index d of table `test`.`testimp4` trx id 94493 lock_mode X locks rec but not gap Record lock, heap no 12 PHYSICAL RECORD: n_fields 2; compact format; info bits 32 0: len 1; hex 61; asc a;; 1: len 4; hex 8000270f; asc ' ;;
實際上咱們看到這裏delete語句後，主鍵加了顯示鎖，這是由於數據查找階段須要加顯示鎖，可是各個二級索引是因爲維護而加的是隱式鎖，咱們經過S2,S3將其轉換爲了顯示鎖。
4.3 delete語句經過二級索引刪除數據
| TIME | S1 | S2 |
|---|---|---|
| T1 | begin；delete from testimp4 where b=9999;（不提交） | |
| T2 | select * from testimp4 where d='a' for update |
#T1時刻S1鎖狀態： ---TRANSACTION 94501, ACTIVE 109 sec 3 lock struct(s), heap size 1160, 3 row lock(s), undo log entries 1 MySQL thread id 11, OS thread handle 140737089492736, query id 576 localhost root TABLE LOCK table `test`.`testimp4` trx id 94501 lock mode IX RECORD LOCKS space id 501 page no 4 n bits 80 index b of table `test`.`testimp4` trx id 94501 lock_mode X Record lock, heap no 1 PHYSICAL RECORD: n_fields 1; compact format; info bits 0 0: len 8; hex 73757072656d756d; asc supremum;; Record lock, heap no 12 PHYSICAL RECORD: n_fields 2; compact format; info bits 32 0: len 4; hex 8000270f; asc ' ;; 1: len 4; hex 8000270f; asc ' ;; RECORD LOCKS space id 501 page no 3 n bits 80 index PRIMARY of table `test`.`testimp4` trx id 94501 lock_mode X locks rec but not gap Record lock, heap no 12 PHYSICAL RECORD: n_fields 6; compact format; info bits 32 0: len 4; hex 8000270f; asc ' ;; 1: len 6; hex 000000017125; asc q%;; 2: len 7; hex 5a0000002518ea; asc Z % ;; 3: len 4; hex 8000270f; asc ' ;; 4: len 4; hex 8000270f; asc ' ;; 5: len 1; hex 61; asc a;; # T2時刻S1鎖狀態： ---TRANSACTION 94501, ACTIVE 119 sec 4 lock struct(s), heap size 1160, 4 row lock(s), undo log entries 1 MySQL thread id 11, OS thread handle 140737089492736, query id 576 localhost root TABLE LOCK table `test`.`testimp4` trx id 94501 lock mode IX RECORD LOCKS space id 501 page no 4 n bits 80 index b of table `test`.`testimp4` trx id 94501 lock_mode X Record lock, heap no 1 PHYSICAL RECORD: n_fields 1; compact format; info bits 0 0: len 8; hex 73757072656d756d; asc supremum;; Record lock, heap no 12 PHYSICAL RECORD: n_fields 2; compact format; info bits 32 0: len 4; hex 8000270f; asc ' ;; 1: len 4; hex 8000270f; asc ' ;; RECORD LOCKS space id 501 page no 3 n bits 80 index PRIMARY of table `test`.`testimp4` trx id 94501 lock_mode X locks rec but not gap Record lock, heap no 12 PHYSICAL RECORD: n_fields 6; compact format; info bits 32 0: len 4; hex 8000270f; asc ' ;; 1: len 6; hex 000000017125; asc q%;; 2: len 7; hex 5a0000002518ea; asc Z % ;; 3: len 4; hex 8000270f; asc ' ;; 4: len 4; hex 8000270f; asc ' ;; 5: len 1; hex 61; asc a;; RECORD LOCKS space id 501 page no 5 n bits 80 index d of table `test`.`testimp4` trx id 94501 lock_mode X locks rec but not gap Record lock, heap no 12 PHYSICAL RECORD: n_fields 2; compact format; info bits 32 0: len 1; hex 61; asc a;; 1: len 4; hex 8000270f; asc ' ;;
實際上咱們看到這裏delete語句後，顯示二級索引加了顯示鎖，而後主鍵加了顯示鎖，這是由於數據查找階段先查找的二級索引而後回表查的主鍵，可是對於二級索引d來說是因爲維護而加的是隱式鎖，咱們經過S2將其轉換爲了顯示鎖。
4.4 update語句經過主鍵修改數據
這裏要特別注意一下，對於二級索引的更新一般是進行了刪除和插入，所以這裏有2行數據都有隱式鎖
| TIME | S1 | S2 | S3 |
|---|---|---|---|
| T1 | begin;update testimp4 set b=10000 where id=9999;（不提交） | ||
| T2 | select * from testimp4 where b=9999 for update | ||
| T3 | select * from testimp4 where b=10000 for update |
# T1時刻S1鎖狀態 ---TRANSACTION 94553, ACTIVE 7 sec 2 lock struct(s), heap size 1160, 1 row lock(s), undo log entries 1 MySQL thread id 11, OS thread handle 140737089492736, query id 730 localhost root TABLE LOCK table `test`.`testimp4` trx id 94553 lock mode IX RECORD LOCKS space id 501 page no 3 n bits 80 index PRIMARY of table `test`.`testimp4` trx id 94553 lock_mode X locks rec but not gap Record lock, heap no 12 PHYSICAL RECORD: n_fields 6; compact format; info bits 0 0: len 4; hex 8000270f; asc ' ;; 1: len 6; hex 000000017159; asc qY;; 2: len 7; hex 770000002a187f; asc w * ;; 3: len 4; hex 8000270f; asc ' ;; 4: len 4; hex 80002710; asc ' ;; 5: len 1; hex 61; asc a;; # T2時刻S1鎖狀態 ---TRANSACTION 94553, ACTIVE 62 sec 3 lock struct(s), heap size 1160, 2 row lock(s), undo log entries 1 MySQL thread id 11, OS thread handle 140737089492736, query id 730 localhost root TABLE LOCK table `test`.`testimp4` trx id 94553 lock mode IX RECORD LOCKS space id 501 page no 3 n bits 80 index PRIMARY of table `test`.`testimp4` trx id 94553 lock_mode X locks rec but not gap Record lock, heap no 12 PHYSICAL RECORD: n_fields 6; compact format; info bits 0 0: len 4; hex 8000270f; asc ' ;; 1: len 6; hex 000000017159; asc qY;; 2: len 7; hex 770000002a187f; asc w * ;; 3: len 4; hex 8000270f; asc ' ;; 4: len 4; hex 80002710; asc ' ;; 5: len 1; hex 61; asc a;; RECORD LOCKS space id 501 page no 4 n bits 80 index b of table `test`.`testimp4` trx id 94553 lock_mode X locks rec but not gap Record lock, heap no 12 PHYSICAL RECORD: n_fields 2; compact format; info bits 32 0: len 4; hex 8000270f; asc ' ;; 1: len 4; hex 8000270f; asc ' ;; # T3時刻S1鎖狀態 ---TRANSACTION 94553, ACTIVE 128 sec 3 lock struct(s), heap size 1160, 3 row lock(s), undo log entries 1 MySQL thread id 11, OS thread handle 140737089492736, query id 730 localhost root TABLE LOCK table `test`.`testimp4` trx id 94553 lock mode IX RECORD LOCKS space id 501 page no 3 n bits 80 index PRIMARY of table `test`.`testimp4` trx id 94553 lock_mode X locks rec but not gap Record lock, heap no 12 PHYSICAL RECORD: n_fields 6; compact format; info bits 0 0: len 4; hex 8000270f; asc ' ;; 1: len 6; hex 000000017159; asc qY;; 2: len 7; hex 770000002a187f; asc w * ;; 3: len 4; hex 8000270f; asc ' ;; 4: len 4; hex 80002710; asc ' ;; 5: len 1; hex 61; asc a;; RECORD LOCKS space id 501 page no 4 n bits 80 index b of table `test`.`testimp4` trx id 94553 lock_mode X locks rec but not gap Record lock, heap no 2 PHYSICAL RECORD: n_fields 2; compact format; info bits 0 0: len 4; hex 80002710; asc ' ;; 1: len 4; hex 8000270f; asc ' ;; Record lock, heap no 12 PHYSICAL RECORD: n_fields 2; compact format; info bits 32 0: len 4; hex 8000270f; asc ' ;; 1: len 4; hex 8000270f; asc ' ;;
這裏因爲對錶的二級索引b經過主鍵進行了修改，那麼二級索引包含了2條數據，一條標記爲del flag，另一條爲插入以下：
(11) normal record offset:266 heapno:12 n_owned 0,delflag:Y minflag:0 rectype:0 (12) normal record offset:126 heapno:2 n_owned 0,delflag:N minflag:0 rectype:0 (13) SUPREMUM record offset:112 heapno:1 n_owned 8,delflag:N minflag:0 rectype:3
所以這兩行都上了隱式鎖，這是因爲二級索引維護而上的，值得注意的是二級索引d不會上隱式鎖，由於update語句的修改不會涉及到d列索引，所以不會維護。若是查詢d列上的值（for update），會獲取d列上的鎖成功，而後會堵塞在主鍵id上以下：
---TRANSACTION 94565, ACTIVE 4 sec starting index read mysql tables in use 1, locked 1 LOCK WAIT 3 lock struct(s), heap size 1160, 2 row lock(s) MySQL thread id 16, OS thread handle 140737086228224, query id 748 localhost root Sending data select * from testimp4 where d='a' for update ------- TRX HAS BEEN WAITING 4 SEC FOR THIS LOCK TO BE GRANTED: RECORD LOCKS space id 501 page no 3 n bits 80 index PRIMARY of table `test`.`testimp4` trx id 94565 lock_mode X locks rec but not gap waiting Record lock, heap no 12 PHYSICAL RECORD: n_fields 6; compact format; info bits 0 0: len 4; hex 8000270f; asc ' ;; 1: len 6; hex 000000017161; asc qa;; 2: len 7; hex 7c0000002d25eb; asc | -% ;; 3: len 4; hex 8000270f; asc ' ;; 4: len 4; hex 80002710; asc ' ;; 5: len 1; hex 61; asc a;; ------------------ TABLE LOCK table `test`.`testimp4` trx id 94565 lock mode IX RECORD LOCKS space id 501 page no 5 n bits 80 index d of table `test`.`testimp4` trx id 94565 lock_mode X Record lock, heap no 12 PHYSICAL RECORD: n_fields 2; compact format; info bits 0 0: len 1; hex 61; asc a;; 1: len 4; hex 8000270f; asc ' ;; RECORD LOCKS space id 501 page no 3 n bits 80 index PRIMARY of table `test`.`testimp4` trx id 94565 lock_mode X locks rec but not gap waiting Record lock, heap no 12 PHYSICAL RECORD: n_fields 6; compact format; info bits 0 0: len 4; hex 8000270f; asc ' ;; 1: len 6; hex 000000017161; asc qa;; 2: len 7; hex 7c0000002d25eb; asc | -% ;; 3: len 4; hex 8000270f; asc ' ;; 4: len 4; hex 80002710; asc ' ;; 5: len 1; hex 61; asc a;;
狀況還有不少不在一一列舉，Innodb行鎖一直都是一個使人頭疼的問題。
5、關於鎖的斷定
5.1 lock_sec_rec_read_check_and_lock函數
主要用於二級索引數據查找段階段加顯示鎖，，對於update/delete而言，首先是須要找到須要修改的數據，加鎖前須要判斷本記錄是否存在隱式鎖，因爲二級索引行數據不包含trx id，所以先用page的max trx id和當前活躍的最小讀寫事務進行比對判斷，若是大於等於則可能存在顯示鎖，而後須要回表經過主鍵進行精細化判斷。而精細化回表判斷行是否存在隱式鎖，那麼代價就比較大了，所以這須要一個判斷流程以下
lock_sec_rec_read_check_and_lock：
if ((page_get_max_trx_id(block->frame) >= trx_rw_min_trx_id()
|| recv_recovery_is_on())
&& !page_rec_is_supremum(rec)) {
lock_rec_convert_impl_to_expl(block, rec, index, offsets);//若是符合前面的條件才調入 lock_rec_convert_impl_to_expl
}
以下調入：
->lock_rec_convert_impl_to_expl ->lock_sec_rec_some_has_impl ->row_vers_impl_x_locked 此處會進行彙集索引的回表，一樣是經過二級索引進行定位返回btr_cur_search_to_nth_level ->row_vers_impl_x_locked_low 最後會調入 row_vers_impl_x_locked_low函數進行核心判斷
棧以下：
#0 row_vers_impl_x_locked_low (clust_rec=0x7fff39a21226 "\200", clust_index=0x7ffeb5092680, rec=0x7fff39a2ac30 "\200", index=0x7ffeb5093610, offsets=0x7fffe8461730, mtr=0x7fffe8460e90) at /home/mysql/soft/percona-server-5.7.29-32/storage/innobase/row/row0vers.cc:101 #1 0x0000000001b2c84e in row_vers_impl_x_locked (rec=0x7fff39a2ac30 "\200", index=0x7ffeb5093610, offsets=0x7fffe8461730) at /home/mysql/soft/percona-server-5.7.29-32/storage/innobase/row/row0vers.cc:390 #2 0x00000000019e8448 in lock_sec_rec_some_has_impl (rec=0x7fff39a2ac30 "\200", index=0x7ffeb5093610, offsets=0x7fffe8461730) at /home/mysql/soft/percona-server-5.7.29-32/storage/innobase/lock/lock0lock.cc:1276 #3 0x00000000019f339a in lock_rec_convert_impl_to_expl (block=0x7fff38d94ca0, rec=0x7fff39a2ac30 "\200", index=0x7ffeb5093610, offsets=0x7fffe8461730) at /home/mysql/soft/percona-server-5.7.29-32/storage/innobase/lock/lock0lock.cc:6124 #4 0x00000000019f3dd2 in lock_sec_rec_read_check_and_lock (flags=0, block=0x7fff38d94ca0, rec=0x7fff39a2ac30 "\200", index=0x7ffeb5093610, offsets=0x7fffe8461730, mode=LOCK_X, gap_mode=1024, thr=0x7ffeb4c89358) at /home/mysql/soft/percona-server-5.7.29-32/storage/innobase/lock/lock0lock.cc:6357 #5 0x0000000001af7271 in sel_set_rec_lock (pcur=0x7ffeb4c887d8, rec=0x7fff39a2ac30 "\200", index=0x7ffeb5093610, offsets=0x7fffe8461730, mode=3, type=1024, thr=0x7ffeb4c89358, mtr=0x7fffe8461a50) at /home/mysql/soft/percona-server-5.7.29-32/storage/innobase/row/row0sel.cc:1278 #6 0x0000000001b00049 in row_search_mvcc (buf=0x7ffeb4977070 "\370\211\037", mode=PAGE_CUR_GE, prebuilt=0x7ffeb4c885b0, match_mode=1, direction=1) at /home/mysql/soft/percona-server-5.7.29-32/storage/innobase/row/row0sel.cc:5710
可是須要注意的是，max trx id只會在二級索引上更新，而且每次更新一行都會更新掉，那麼引發的一個問題就是若是連續屢次刪除同一個二級索引上的記錄**（delete from testimp4 where b=7700;），除第一次之外都會調入row_vers_impl_x_locked_low這個函數，由於這是查詢一行加鎖一行修改一行（每行都會修改page的max trx id）的。可是update卻不一樣，update若是修改本二級索引的值通常會進入（如：update testimp4 set b=1500 where b=1800;）Searching rows for update狀態**，先創建一個臨時文件來先存儲須要更改的行記錄，而後進行批量更改進入updating狀態，那麼則不會出現這種問題，由於這是在數據查找階段進行的判斷，而不是數據修改階段。又好比**（如：update testimp2 set c='a' where b=1800)這樣的語句也不會觸發，這是由於b索引的行記錄一直沒有改變，所以不會修改b索引page的max trx id。所以update很好的規避了這個問題不會頻繁的進入函數row_vers_impl_x_locked_low**進行斷定，可是delete卻不行。
關於row_vers_impl_x_locked_low函數對於二級索引是否存在隱式鎖的斷定，比較複雜分爲好多種狀況，再也不描述。所以最開始咱們看到的問題，這個過程已經進入了row_vers_impl_x_locked_low函數，那麼能夠判斷這個delete語句可能更新了多行（可是從代碼行數上判斷不是這種狀況），或者有可能本語句事務作過修改本語句修改記錄的其餘語句，須要進行精細化判斷。
5.2 lock_sec_rec_modify_check_and_lock
主要用於數據修改階段加隱式鎖，二級索引因爲行數據的修改（update修改了本二級索引字包含段值或者尾部的主鍵）而被動維護的加鎖。注意若是是select for update where條件是主鍵則不會加判斷二級索引是否包含隱含鎖，若是出現衝突會堵塞在主鍵上。
5.3 lock_clust_rec_read_check_and_lock
數據查找階段加顯示鎖，主要用於主鍵查找數據加顯示鎖或者二級索引訪問後的回表主鍵加顯示鎖，加鎖前須要判斷是否存在隱含鎖。因爲主鍵行中包含了trx id僞列，所以能夠簡單的用本行trx id的事務是否還活躍進行斷定了，這個過程代價很小，所以每行加鎖老是會有這個過程，也就是每次都會調用lock_rec_convert_impl_to_expl函數進行判斷，以下：
lock_clust_rec_read_check_and_lock ->lock_rec_convert_impl_to_expl ->lock_clust_rec_some_has_impl (主鍵判斷很是簡單)
棧以下：
#0 lock_clust_rec_some_has_impl (rec=0x7fff05ad40db "\200", index=0x7ffe8802ce70, offsets=0x7fffe8461660) at /home/mysql/soft/percona-server-5.7.29-32/storage/innobase/include/lock0priv.ic:69 #1 0x00000000019f3333 in lock_rec_convert_impl_to_expl (block=0x7fff050a0950, rec=0x7fff05ad40db "\200", index=0x7ffe8802ce70, offsets=0x7fffe8461660) at /home/mysql/soft/percona-server-5.7.29-32/storage/innobase/lock/lock0lock.cc:6118 #2 0x00000000019f418d in lock_clust_rec_read_check_and_lock (flags=0, block=0x7fff050a0950, rec=0x7fff05ad40db "\200", index=0x7ffe8802ce70, offsets=0x7fffe8461660, mode=LOCK_X, gap_mode=1024, thr=0x7ffeb49903c8) at /home/mysql/soft/percona-server-5.7.29-32/storage/innobase/lock/lock0lock.cc:6430 #3 0x0000000001af7193 in sel_set_rec_lock (pcur=0x7ffeb498fe38, rec=0x7fff05ad40db "\200", index=0x7ffe8802ce70, offsets=0x7fffe8461660, mode=3, type=1024, thr=0x7ffeb49903c8, mtr=0x7fffe8461980) at /home/mysql/soft/percona-server-5.7.29-32/storage/innobase/row/row0sel.cc:1263 #4 0x0000000001b00049 in row_search_mvcc (buf=0x7ffeb498f380 "\371\005", mode=PAGE_CUR_GE, prebuilt=0x7ffeb498fc10, match_mode=1, direction=0) at /home/mysql/soft/percona-server-5.7.29-32/storage/innobase/row/row0sel.cc:5710
5.4 lock_clust_rec_modify_check_and_lock
主鍵數據修改階段加隱式鎖，當前發現爲在直接update主鍵值或者delete操做的時候，可是這種狀況下實際上主鍵已經在數據查詢階段加了顯示鎖。
6、update不徹底等同於delete&&insert
直接區分以下：
主鍵更新，接口row_upd_clust_step
row_upd_changes_ord_field_binary 判斷是否更新了彙集索引的值 若是更新了 -> row_upd_clust_rec_by_insert 進行主鍵刪除插入(設置del flag) 若是沒有更新 ->row_upd_clust_rec ->btr_cur_optimistic_update 只考慮樂觀update ->row_upd_changes_field_size_or_external 判斷新記錄是否超過本行現有大小 若是否 ->btr_cur_update_in_place 原地更新 若是是 ->page_cur_delete_rec 則須要進行主鍵刪除(實際刪除非設置del falg) ->btr_cur_insert_if_possible 插入
二級索引更新，接口row_upd_sec_step 始終爲刪除插入(設置del flag)
7、關於History list length 的單位
實際上History list length 就是當一個update undo log （非insert）的計數器，一個事務只有一個undo log 。來源爲trx_sys->rseg_history_len，這個值會在事務提交的時候更新，不管事務大小。可是因爲不少內部事務的存在，這個值會遠大於可觀測的事務個數。棧以下：
#0 trx_purge_add_update_undo_to_history (trx=0x7fffeac7df50, undo_ptr=0x7fffeac7e370, undo_page=0x7fff2837c000 "\373\252\223T", update_rseg_history_len=true, n_added_logs=1, mtr=0x7fffe8399830) at /home/mysql/soft/percona-server-5.7.29-32/storage/innobase/trx/trx0purge.cc:354 #1 0x0000000001b9c064 in trx_undo_update_cleanup (trx=0x7fffeac7df50, undo_ptr=0x7fffeac7e370, undo_page=0x7fff2837c000 "\373\252\223T", update_rseg_history_len=true, n_added_logs=1, mtr=0x7fffe8399830) at /home/mysql/soft/percona-server-5.7.29-32/storage/innobase/trx/trx0undo.cc:1970 #2 0x0000000001b8b639 in trx_write_serialisation_history (trx=0x7fffeac7df50, mtr=0x7fffe8399830) at /home/mysql/soft/percona-server-5.7.29-32/storage/innobase/trx/trx0trx.cc:1684 #3 0x0000000001b8c9b0 in trx_commit_low (trx=0x7fffeac7df50, mtr=0x7fffe8399830) at /home/mysql/soft/percona-server-5.7.29-32/storage/innobase/trx/trx0trx.cc:2184
到這裏，雜七雜八記錄了一大堆，記錄於此以備後用。
附錄1函數接口
一、read view
MVCC::view_open：創建read view
ReadView::prepare：準備read view中的值
ReadView::complete：寫入read view中的值
MVCC::view_close：釋放read view
二、可見性判斷
lock_clust_rec_cons_read_sees：主鍵可見性判斷
lock_sec_rec_cons_read_sees：二級索引可見性判斷
附錄 2具體函數
一、read view
/** The read should not see any transaction with trx id >= this value. In other words, this is the "high water mark". */ trx_id_t m_low_limit_id; /** The read should see all trx ids which are strictly smaller (<) than this value. In other words, this is the low water mark". */ trx_id_t m_up_limit_id; /** trx id of creating transaction, set to TRX_ID_MAX for free views. */ trx_id_t m_creator_trx_id; /** Set of RW transactions that was active when this snapshot was taken */ ids_t m_ids; /** The view does not need to see the undo logs for transactions whose transaction number is strictly smaller (<) than this value: they can be removed in purge if not needed by other views */ trx_id_t m_low_limit_no;
void
ReadView::prepare(trx_id_t id)
{
ut_ad(!m_cloned);
ut_ad(mutex_own(&trx_sys->mutex));
m_creator_trx_id = id;
m_low_limit_no = m_low_limit_id = trx_sys->max_trx_id;
if (!trx_sys->rw_trx_ids.empty()) {
copy_trx_ids(trx_sys->rw_trx_ids);
} else {
m_ids.clear();
}
if (UT_LIST_GET_LEN(trx_sys->serialisation_list) > 0) {
const trx_t* trx;
trx = UT_LIST_GET_FIRST(trx_sys->serialisation_list);
if (trx->no < m_low_limit_no) {
m_low_limit_no = trx->no;
}
}
}
void
ReadView::complete()
{
ut_ad(!m_cloned);
/* The first active transaction has the smallest id. */
m_up_limit_id = !m_ids.empty() ? m_ids.front() : m_low_limit_id;
ut_ad(m_up_limit_id <= m_low_limit_id);
m_closed = false;
}
二、可見性判斷
二級索引回表判斷可見性
Row_sel_get_clust_rec_for_mysql::operator()
->lock_clust_rec_cons_read_sees (回表後根據主鍵判斷其可見性)
->row_sel_build_prev_vers_for_mysql(構建前版本)
->row_vers_build_for_consistent_read
本函數循環構建，直到條件知足，或者前版本爲NULL
if (view->changes_visible(trx_id, index->table->name)) {
/* The view already sees this version: we can copy
it to in_heap and return */
buf = static_cast<byte*>(
mem_heap_alloc(
in_heap, rec_offs_size(*offsets)));
*old_vers = rec_copy(buf, prev_version, *offsets);
rec_offs_make_valid(*old_vers, index, *offsets);
if (vrow && *vrow) {
*vrow = dtuple_copy(*vrow, in_heap);
dtuple_dup_v_fld(*vrow, in_heap);
}
break;
最終會將前版本的主鍵值根據需求取字段返回給MySQL層
關於using index 也須要回表流程
row_search_mvcc:
if (!srv_read_only_mode
&& !lock_sec_rec_cons_read_sees( // 若是二級索引記錄判斷爲不可見
rec, index, trx->read_view)) {
/* We should look at the clustered index.
However, as this is a non-locking read,
we can skip the clustered index lookup if
the condition does not match the secondary
index entry. */
switch (row_search_idx_cond_check(
buf, prebuilt, rec, offsets)) {
case ICP_NO_MATCH:
goto next_rec;
case ICP_OUT_OF_RANGE:
err = DB_RECORD_NOT_FOUND;
goto idx_cond_failed;
case ICP_MATCH:
goto requires_clust_rec; //走這裏就進入了回表判斷流程
}
lock_sec_rec_cons_read_sees：
trx_id_t max_trx_id = page_get_max_trx_id(page_align(rec));//獲取頁的max trx id
ut_ad(max_trx_id > 0);
return(view->sees(max_trx_id));
全文完。
Enjoy MySQL :)
葉老師的「MySQL核心優化」大課已升級到MySQL 8.0，掃碼開啓MySQL 8.0修行之旅吧