golng mysql庫鏈接池分析

golng mysql庫鏈接池分析

0x1 背景

golang的協程是好用，可是有時候瓶頸並不在語言上，而是在後面的數據源上面，例如咱們常見的mysql，redis等，當一個後端服務不少請求的時候，語言是能hold得住，可是 mysql產生錯誤，好比 too many connection, too many time_wait 等等這些，今天咱們就分析一下怎麼解決這種問題

0x2 代碼範例

請查看main.go， halokid （有幫忙的話請start或者follow一下哦，謝謝）

0x3 分析

只執行ini函數， 檢查mysql的進程顯示爲（原有的mysql是沒有進程在處理的）

沒執行前

mysql> show processlist;
+----+-----------------+------------------+------+---------+------+------------------------+------------------+
| Id | User            | Host             | db   | Command | Time | State                  | Info             |
+----+-----------------+------------------+------+---------+------+------------------------+------------------+
|  4 | event_scheduler | localhost        | NULL | Daemon  | 2304 | Waiting on empty queue | NULL             |
|  9 | root            | 10.244.1.1:64000 | test | Sleep   | 1315 |                        | NULL             |
| 10 | root            | 10.244.1.1:64022 | test | Query   |    0 | starting               | show processlist |
+----+-----------------+------------------+------+---------+------+------------------------+------------------+
3 rows in set (0.01 sec)

執行後

mysql> show processlist;
+----+-----------------+------------------+------+---------+------+------------------------+------------------+
| Id | User            | Host             | db   | Command | Time | State                  | Info             |
+----+-----------------+------------------+------+---------+------+------------------------+------------------+
|  4 | event_scheduler | localhost        | NULL | Daemon  | 2284 | Waiting on empty queue | NULL             |
|  9 | root            | 10.244.1.1:64000 | test | Sleep   | 1295 |                        | NULL             |
| 10 | root            | 10.244.1.1:64022 | test | Query   |    0 | starting               | show processlist |
| 13 | root            | 10.244.1.1:52134 | test | Sleep   |   20 |                        | NULL             |
+----+-----------------+------------------+------+---------+------+------------------------+------------------+
4 rows in set (0.00 sec)

可見執行 db.Ping() 以後， process多了一個 Sleep 的鏈接，就是放了一個鏈接進 鏈接池

運行

db.SetMaxOpenConns(10)
db.SetMaxIdleConns(5)

兩句以後， 鏈接池並無改變， 可見上面的邏輯是在 數據庫處理邏輯真實執行的時候才生效的

執行協程查詢

mysql> show processlist;
+----+-----------------+------------------+------+---------+------+------------------------+------------------+
| Id | User            | Host             | db   | Command | Time | State                  | Info             |
+----+-----------------+------------------+------+---------+------+------------------------+------------------+
|  4 | event_scheduler | localhost        | NULL | Daemon  | 4397 | Waiting on empty queue | NULL             |
|  9 | root            | 10.244.1.1:64000 | test | Sleep   | 3408 |                        | NULL             |
| 10 | root            | 10.244.1.1:64022 | test | Query   |    0 | starting               | show processlist |
| 19 | root            | 10.244.1.1:54823 | test | Sleep   |  952 |                        | NULL             |
| 20 | root            | 10.244.1.1:54824 | test | Sleep   | 1104 |                        | NULL             |
| 47 | root            | 10.244.1.1:57906 | test | Sleep   |    0 |                        | NULL             |
| 48 | root            | 10.244.1.1:57909 | test | Sleep   |    0 |                        | NULL             |
| 49 | root            | 10.244.1.1:57912 | test | Sleep   |    0 |                        | NULL             |
| 50 | root            | 10.244.1.1:57907 | test | Sleep   |    0 |                        | NULL             |
| 51 | root            | 10.244.1.1:57908 | test | Sleep   |    0 |                        | NULL             |
| 52 | root            | 10.244.1.1:57913 | test | Sleep   |    0 |                        | NULL             |
| 53 | root            | 10.244.1.1:57911 | test | Sleep   |    0 |                        | NULL             |
| 54 | root            | 10.244.1.1:57910 | test | Sleep   |    0 |                        | NULL             |
| 55 | root            | 10.244.1.1:57915 | test | Sleep   |    0 |                        | NULL             |
| 56 | root            | 10.244.1.1:57914 | test | Sleep   |    0 |                        | NULL             |
+----+-----------------+------------------+------+---------+------+------------------------+------------------+
15 rows in set (0.00 sec)

執行完在等待

mysql> show processlist;
+----+-----------------+------------------+------+---------+------+------------------------+------------------+
| Id | User            | Host             | db   | Command | Time | State                  | Info             |
+----+-----------------+------------------+------+---------+------+------------------------+------------------+
|  4 | event_scheduler | localhost        | NULL | Daemon  | 3931 | Waiting on empty queue | NULL             |
|  9 | root            | 10.244.1.1:64000 | test | Sleep   | 2942 |                        | NULL             |
| 10 | root            | 10.244.1.1:64022 | test | Query   |    0 | starting               | show processlist |
| 19 | root            | 10.244.1.1:54823 | test | Sleep   |  486 |                        | NULL             |
| 20 | root            | 10.244.1.1:54824 | test | Sleep   |  638 |                        | NULL             |
| 32 | root            | 10.244.1.1:56588 | test | Sleep   |   22 |                        | NULL             |
| 33 | root            | 10.244.1.1:56591 | test | Sleep   |   22 |                        | NULL             |
| 34 | root            | 10.244.1.1:56589 | test | Sleep   |   22 |                        | NULL             |
| 35 | root            | 10.244.1.1:56590 | test | Sleep   |   22 |                        | NULL             |
| 36 | root            | 10.244.1.1:56592 | test | Sleep   |   22 |                        | NULL             |
+----+-----------------+------------------+------+---------+------+------------------------+------------------+
10 rows in set (0.00 sec)

協程執行完以後

mysql> show processlist;
+----+-----------------+------------------+------+---------+------+------------------------+------------------+
| Id | User            | Host             | db   | Command | Time | State                  | Info             |
+----+-----------------+------------------+------+---------+------+------------------------+------------------+
|  4 | event_scheduler | localhost        | NULL | Daemon  | 3941 | Waiting on empty queue | NULL             |
|  9 | root            | 10.244.1.1:64000 | test | Sleep   | 2952 |                        | NULL             |
| 10 | root            | 10.244.1.1:64022 | test | Query   |    0 | starting               | show processlist |
| 19 | root            | 10.244.1.1:54823 | test | Sleep   |  496 |                        | NULL             |
| 20 | root            | 10.244.1.1:54824 | test | Sleep   |  648 |                        | NULL             |
+----+-----------------+------------------+------+---------+------+------------------------+------------------+
5 rows in set (0.00 sec)

咱們發現最大鏈接控制在了10個， 執行完以後還有5個鏈接在保持着

這裏有一個很重要的問題，就是鏈接池的過時時間

0x4 深刻分析 咱們把 db.SetConnMaxLifetime(15 * time.Second)， 鏈接池鏈接的生命週期設置爲 15秒, 咱們會發現15秒以後，鏈接池的鏈接都會斷掉

mysql> show processlist;
+----+-----------------+------------------+------+---------+------+------------------------+------------------+
| Id | User            | Host             | db   | Command | Time | State                  | Info             |
+----+-----------------+------------------+------+---------+------+------------------------+------------------+
|  4 | event_scheduler | localhost        | NULL | Daemon  | 4987 | Waiting on empty queue | NULL             |
|  9 | root            | 10.244.1.1:64000 | test | Sleep   | 3998 |                        | NULL             |
| 10 | root            | 10.244.1.1:64022 | test | Query   |    0 | starting               | show processlist |
| 19 | root            | 10.244.1.1:54823 | test | Sleep   | 1542 |                        | NULL             |
| 20 | root            | 10.244.1.1:54824 | test | Sleep   | 1694 |                        | NULL             |
+----+-----------------+------------------+------+---------+------+------------------------+------------------+
5 rows in set (0.00 sec)

30秒以後再次查詢數據庫

time.Sleep(30 * time.Second)
  rows, err := db.Query("select name from users")
  fmt.Println("err -----", err)
  defer rows.Close()
  for rows.Next(){
    var name string
    rows.Scan(&name)
    fmt.Println("name---", name)
  }

這個時候發現程序會從新發起新的db鏈接

總結：

mysql服務端的鏈接生命週期

還有一種請況就是，咱們的程序的鏈接池生命週期設置大於mysql服務器的生命週期設置， 這個時候就會有一種請況，假如咱們重複用鏈接池的鏈接，會產生 鏈接錯誤的問題，解決方法有兩種：

1. 能夠在程序裏面設置生命週期時間小於mysql服務端的鏈接生命週期時間就能夠了
2. 增長程序的重連(keepalive）機制，就是定時發送一個鏈接包服務端 關於第2點咱們咱們之後能夠再發散來講，通常若是容許的話，用第一種方式便可。

mysql> show variables like 'mysqlx_wait_timeout';
+---------------------+-------+
| Variable_name       | Value |
+---------------------+-------+
| mysqlx_wait_timeout | 28800 |
+---------------------+-------+
1 row in set (0.00 sec)