MySQL表結構變動，不可不知的Metadata Lock

時間 2019-11-05

標籤 mysql 結構變動不可不知 metadata lock 欄目 MySQL 简体版

原文原文鏈接

在線上進行DDL操做時，相對於其可能帶來的系統負載，其實，咱們最擔憂的仍是MDL其可能致使的阻塞問題。mysql

一旦DDL操做因獲取不到MDL被阻塞，後續其它針對該表的其它操做都會被阻塞。典型以下，如阻塞稍久的話，咱們會看到Threads_running飆升，CPU告警。sql

mysql> show processlist;
+----+-----------------+-----------+-----------+---------+------+---------------------------------+------------------------------------+
| Id | User            | Host      | db        | Command | Time | State                           | Info                               |
+----+-----------------+-----------+-----------+---------+------+---------------------------------+------------------------------------+
|  4 | event_scheduler | localhost | NULL      | Daemon  |  122 | Waiting on empty queue          | NULL                               |
|  9 | root            | localhost | NULL      | Sleep   |   57 |                                 | NULL                               |
| 12 | root            | localhost | employees | Query   |   40 | Waiting for table metadata lock | alter table slowtech.t1 add c1 int |
| 13 | root            | localhost | employees | Query   |   35 | Waiting for table metadata lock | select * from slowtech.t1          |
| 14 | root            | localhost | employees | Query   |   30 | Waiting for table metadata lock | select * from slowtech.t1          |
| 15 | root            | localhost | employees | Query   |   19 | Waiting for table metadata lock | select * from slowtech.t1          |
| 16 | root            | localhost | employees | Query   |   10 | Waiting for table metadata lock | select * from slowtech.t1          |
| 17 | root            | localhost | employees | Query   |    0 | starting                        | show processlist                   |
+----+-----------------+-----------+-----------+---------+------+---------------------------------+------------------------------------+
8 rows in set (0.00 sec)

若是發生在線上，無疑會影響到業務。因此，通常建議將DDL操做放到業務低峯期作，其實有兩方面的考慮，1. 避免對系統負載產生較大影響。2. 減小DDL被阻塞的機率。數據庫

MDL引入的背景session

MDL是MySQL 5.5.3引入的，主要用於解決兩個問題，併發

RR事務隔離級別下不可重複讀的問題app

以下所示，演示環境，MySQL 5.5.0。工具

session1> begin;
Query OK, 0 rows affected (0.00 sec)

session1> select * from t1;
+------+------+
| id  | name |
+------+------+
|    1 | a    |
|    2 | b    |
+------+------+
2 rows in set (0.00 sec)

session2> alter table t1 add c1 int;
Query OK, 2 rows affected (0.02 sec)
Records: 2  Duplicates: 0  Warnings: 0

session1> select * from t1;
Empty set (0.00 sec)

session1> commit;
Query OK, 0 rows affected (0.00 sec)

session1> select * from t1;
+------+------+------+
| id  | name | c1  |
+------+------+------+
|    1 | a    | NULL |
|    2 | b    | NULL |
+------+------+------+
2 rows in set (0.00 sec)

能夠看到，雖然是RR隔離級別，但在開啓事務的狀況下，第二次查詢卻沒有結果。ui

主從複製問題this

包括主從數據不一致，主從複製中斷等。
以下面的主從數據不一致。spa

session1> create table t1(id int,name varchar(10)) engine=innodb;
Query OK, 0 rows affected (0.00 sec)

session1> begin;
Query OK, 0 rows affected (0.00 sec)

session1> insert into t1 values(1,'a');
Query OK, 1 row affected (0.00 sec)

session2> truncate table t1;
Query OK, 0 rows affected (0.46 sec)

session1> commit;
Query OK, 0 rows affected (0.35 sec)

session1> select * from t1;
Empty set (0.00 sec)

再來看看從庫的結果

session1> select * from slowtech.t1;
+------+------+------+
| id   | name | c1   |
+------+------+------+
|    1 | a    | NULL |
+------+------+------+
1 row in set (0.00 sec)

看看binlog的內容，能夠看到，truncate操做記錄在前，insert操做記錄在後。

# at 7140
#180714 19:32:14 server id 1  end_log_pos 7261    Query    thread_id=31    exec_time=0    error_code=0
SET TIMESTAMP=1531567934/*!*/;
create table t1(id int,name varchar(10)) engine=innodb
/*!*/;

# at 7261
#180714 19:32:30 server id 1  end_log_pos 7333    Query    thread_id=32    exec_time=0    error_code=0
SET TIMESTAMP=1531567950/*!*/;
BEGIN
/*!*/;
# at 7333
#180714 19:32:30 server id 1  end_log_pos 7417    Query    thread_id=32    exec_time=0    error_code=0
SET TIMESTAMP=1531567950/*!*/;
truncate table t1
/*!*/;
# at 7417
#180714 19:32:30 server id 1  end_log_pos 7444    Xid = 422
COMMIT/*!*/;

# at 7444
#180714 19:32:34 server id 1  end_log_pos 7516    Query    thread_id=31    exec_time=0    error_code=0
SET TIMESTAMP=1531567954/*!*/;
BEGIN
/*!*/;
# at 7516
#180714 19:32:24 server id 1  end_log_pos 7611    Query    thread_id=31    exec_time=0    error_code=0
SET TIMESTAMP=1531567944/*!*/;
insert into t1 values(1,'a')
/*!*/;
# at 7611
#180714 19:32:34 server id 1  end_log_pos 7638    Xid = 421
COMMIT/*!*/;

若是會話2執行的是drop table操做，還會致使主從中斷。

有意思的是，若是會話2執行的是alter table操做，其依舊會被阻塞，阻塞時間受innodb_lock_wait_timeout參數限制。

mysql> show processlist;
+----+------+-----------+----------+---------+------+-------------------+---------------------------+
| Id | User | Host      | db       | Command | Time | State             | Info                      |
+----+------+-----------+----------+---------+------+-------------------+---------------------------+
| 54 | root | localhost | NULL     | Query   |    0 | NULL              | show processlist          |
| 58 | root | localhost | slowtech | Sleep   | 1062 |                   | NULL                      |
| 60 | root | localhost | slowtech | Query   |   11 | copy to tmp table | alter table t1 add c1 int |
+----+------+-----------+----------+---------+------+-------------------+---------------------------+
3 rows in set (0.00 sec)

MDL的基本概念

首先，看看官方的說法，

To ensure transaction serializability, the server must not permit one session to perform a data definition language (DDL) statement on a table that is used in an uncompleted explicitly or implicitly started transaction in another session.

The server achieves this by acquiring metadata locks on tables used within a transaction and deferring release of those locks until the transaction ends.

A metadata lock on a table prevents changes to the table's structure.

This locking approach has the implication that a table that is being used by a transaction within one session cannot be used in DDL statements by other sessions until the transaction ends.

從上面的描述能夠看到，

1. MDL出現的初衷就是爲了保護一個處於事務中的表的結構不被修改。

2. 這裏提到的事務包括兩類，顯式事務和AC-NL-RO（auto-commit non-locking read-only）事務。顯式事務包括兩類：1. 關閉AutoCommit下的操做，2. 以begin或start transaction開始的操做。AC-NL-RO可理解爲AutoCommit開啓下的select操做。

3. MDL是事務級別的，只有在事務結束後纔會釋放。在此以前，其實也有相似的保護機制，只不過是語句級別的。

須要注意的是，MDL不單單適用於表，一樣也適用於其它對象，以下表所示，其中，"等待狀態"對應的是"show processlist"中的State。

爲了提升數據庫的併發度，MDL被細分爲了11種類型。

MDL_INTENTION_EXCLUSIVE
MDL_SHARED
MDL_SHARED_HIGH_PRIO
MDL_SHARED_READ
MDL_SHARED_WRITE
MDL_SHARED_WRITE_LOW_PRIO
MDL_SHARED_UPGRADABLE
MDL_SHARED_READ_ONLY
MDL_SHARED_NO_WRITE
MDL_SHARED_NO_READ_WRITE
MDL_EXCLUSIVE

經常使用的有MDL_SHARED_READ,MDL_SHARE D_WRITE及MDL_EXCLUSIVE，其分別用於SELECT操做，DML操做及DDL操做。其它類型的對應操做可參考源碼sql/mdl.h。

對於MDL_EXCLUSIVE，官方的解釋是，

  /*
    An exclusive metadata lock.
    A connection holding this lock can modify both table's metadata and data.
    No other type of metadata lock can be granted while this lock is held.
    To be used for CREATE/DROP/RENAME TABLE statements and for execution of
    certain phases of other DDL statements.
  */

簡而言之，MDL_EXCLUSIVE是獨佔鎖，在其持有期間是不容許其它類型的MDL被授予，天然也包括SELECT和DML操做。

這也就是爲何DDL操做被阻塞時，後續其它操做也會被阻塞。

關於MDL的補充

1. MDL的最大等待時間由lock_wait_timeout參數決定，其默認值爲31536000（365天）。在使用工具進行DDL操做時，這個值就不太合理。事實上，pt-online-schema-change和gh-ost對其就進行了相應的調整，其中，前者60s，後者3s。

2. 若是一個SQL語法上有效，但執行時報錯，如，列名不存在，其一樣會獲取MDL鎖，直到事務結束才釋放。