如何在MySQL中分配innodb_buffer_pool_size

時間  2019-11-06
標籤 如何 mysql 分配 innodb buffer pool size 欄目 MySQL 简体版
原文   原文鏈接

如何在MySQL中分配innodb_buffer_pool_size

innodb_buffer_pool_size是整個MySQL服務器最重要的變量。mysql

1. 爲何須要innodb buffer pool？

在MySQL5.5以前，普遍使用的和默認的存儲引擎是MyISAM。MyISAM使用操做系統緩存來緩存數據。InnoDB須要innodb buffer pool中處理緩存。因此很是須要有足夠的InnoDB buffer pool空間。sql

2. MySQL InnoDB buffer pool 裏包含什麼？

  • 數據緩存
    InnoDB數據頁面數據庫

  • 索引緩存
    索引數據緩存

  • 緩衝數據
    髒頁（在內存中修改還沒有刷新(寫入)到磁盤的數據）服務器

  • 內部結構
    如自適應哈希索引，行鎖等。架構

3. 如何設置innodb_buffer_pool_size?

innodb_buffer_pool_size默認大小爲128M。最大值取決於CPU的架構。在32-bit平臺上，最大值爲2**32 -1,在64-bit平臺上最大值爲2**64-1。當緩衝池大小大於1G時，將innodb_buffer_pool_instances設置大於1的值能夠提升服務器的可擴展性。dom

大的緩衝池能夠減少屢次磁盤I/O訪問相同的表數據。在專用數據庫服務器上，能夠將緩衝池大小設置爲服務器物理內存的80%。性能

3.1 配置緩衝池大小時，請注意如下潛在問題

  • 物理內存爭用可能致使操做系統頻繁的pagingspa

  • InnoDB爲緩衝區和control structures保留了額外的內存，所以總分配空間比指定的緩衝池大小大約大10％。操作系統

  • 緩衝池的地址空間必須是連續的，這在帶有在特定地址加載的DLL的Windows系統上多是一個問題。

  • 初始化緩衝池的時間大體與其大小成比例。在具備大緩衝池的實例上，初始化時間可能很長。要減小初始化時間，能夠在服務器關閉時保存緩衝池狀態，並在服務器啓動時將其還原。

    • innodb_buffer_pool_dump_pct：指定每一個緩衝池最近使用的頁面讀取和轉儲的百分比。 範圍是1到100。默認值是25。例如，若是有4個緩衝池，每一個緩衝池有100個page，而且innodb_buffer_pool_dump_pct設置爲25，則dump每一個緩衝池中最近使用的25個page。
    • innodb_buffer_pool_dump_at_shutdown：默認啓用。指定在MySQL服務器關閉時是否記錄在InnoDB緩衝池中緩存的頁面，以便在下次從新啓動時縮短預熱過程。
    • innodb_buffer_pool_load_at_startup：默認啓用。指定在MySQL服務器啓動時，InnoDB緩衝池經過加載以前保存的相同頁面自動預熱。 一般與innodb_buffer_pool_dump_at_shutdown結合使用。

增大或減少緩衝池大小時，將以chunk的形式執行操做。chunk大小由innodb_buffer_pool_chunk_size配置選項定義，默認值爲128 MB。

緩衝池大小必須始終等於或者是innodb_buffer_pool_chunk_size * innodb_buffer_pool_instances的倍數。
若是將緩衝池大小更改成不等於或等於innodb_buffer_pool_chunk_size * innodb_buffer_pool_instances的倍數的值，
則緩衝池大小將自動調整爲等於或者是innodb_buffer_pool_chunk_size * innodb_buffer_pool_instances的倍數的值。

innodb_buffer_pool_size能夠動態設置，容許在不從新啓動服務器的狀況下調整緩衝池的大小。 能夠經過狀態變量Innodb_buffer_pool_resize_status報告在線調整緩衝池大小操做的狀態。

mysql> show status like 'Innodb_buffer_pool_resize%';
+----------------------------------+-------+
| Variable_name                    | Value |
+----------------------------------+-------+
| Innodb_buffer_pool_resize_status |       |
+----------------------------------+-------+

 

3.2 配置示例

在如下示例中，innodb_buffer_pool_size設置爲3G，innodb_buffer_pool_instances設置爲8。innodb_buffer_pool_chunk_size默認值爲128M。

3G是有效的innodb_buffer_pool_size值，由於3G是innodb_buffer_pool_instances = 8 * innodb_buffer_pool_chunk_size = 128M的倍數

# mysqld --innodb_buffer_pool_size=3G --innodb_buffer_pool_instances=8 &

mysql> show variables like 'innodb_buffer_pool%';

+-------------------------------------+----------------+
| Variable_name                       | Value          |
+-------------------------------------+----------------+
| innodb_buffer_pool_chunk_size       | 134217728      |
| innodb_buffer_pool_dump_at_shutdown | ON             |
| innodb_buffer_pool_dump_now         | OFF            |
| innodb_buffer_pool_dump_pct         | 25             |
| innodb_buffer_pool_filename         | ib_buffer_pool |
| innodb_buffer_pool_instances        | 8              |
| innodb_buffer_pool_load_abort       | OFF            |
| innodb_buffer_pool_load_at_startup  | ON             |
| innodb_buffer_pool_load_now         | OFF            |
| innodb_buffer_pool_size             | 3221225472     |
+-------------------------------------+----------------+
10 rows in set (0.01 sec)

 

在如下示例中，innodb_buffer_pool_size設置爲3G，innodb_buffer_pool_instances設置爲16. innodb_buffer_pool_chunk_size爲128M。

3G不是有效的innodb_buffer_pool_size值，由於3G不是innodb_buffer_pool_instances = 16 * innodb_buffer_pool_chunk_size = 128M的倍數，能夠看出innodb_buffer_pool_size的值自動調整到4GB

# mysqld --innodb_buffer_pool_size=3G --innodb_buffer_pool_instances=16 &

mysql> show variables like '%innodb_buffer_pool%';
+-------------------------------------+----------------+
| Variable_name                       | Value          |
+-------------------------------------+----------------+
| innodb_buffer_pool_chunk_size       | 134217728      |
| innodb_buffer_pool_dump_at_shutdown | ON             |
| innodb_buffer_pool_dump_now         | OFF            |
| innodb_buffer_pool_dump_pct         | 25             |
| innodb_buffer_pool_filename         | ib_buffer_pool |
| innodb_buffer_pool_instances        | 16             |
| innodb_buffer_pool_load_abort       | OFF            |
| innodb_buffer_pool_load_at_startup  | ON             |
| innodb_buffer_pool_load_now         | OFF            |
| innodb_buffer_pool_size             | 4294967296     |
+-------------------------------------+----------------+
10 rows in set (0.01 sec)

 

3.3 在線調整InnoDB緩衝池大小

mysql> SET GLOBAL innodb_buffer_pool_size = 3221225472

 

3.4 監控在線緩衝池調整進度

mysql> SHOW STATUS WHERE Variable_name='InnoDB_buffer_pool_resize_status';
+----------------------------------+----------------------------------------------------+
| Variable_name                    | Value                                              |
+----------------------------------+----------------------------------------------------+
| Innodb_buffer_pool_resize_status | Completed resizing buffer pool at 180824 15:05:03. |
+----------------------------------+----------------------------------------------------+

 

緩衝池大小調整進度也記錄在服務器錯誤日誌中。如下爲增大時，記錄的日誌

2018-08-24T07:05:03.819049Z 2 [Note] InnoDB: Requested to resize buffer pool. (new size: 3221225472 bytes)
2018-08-24T07:05:03.819141Z 0 [Note] InnoDB: Resizing buffer pool from 2684354560 to 3221225472 (unit=134217728).
2018-08-24T07:05:03.819155Z 0 [Note] InnoDB: Disabling adaptive hash index.
2018-08-24T07:05:03.824902Z 0 [Note] InnoDB: disabled adaptive hash index.
2018-08-24T07:05:03.824933Z 0 [Note] InnoDB: Withdrawing blocks to be shrunken.
2018-08-24T07:05:03.824940Z 0 [Note] InnoDB: Latching whole of buffer pool.
2018-08-24T07:05:03.824959Z 0 [Note] InnoDB: buffer pool 0 : resizing with chunks 5 to 6.
2018-08-24T07:05:03.839564Z 0 [Note] InnoDB: buffer pool 0 : 1 chunks (8192 blocks) were added.
2018-08-24T07:05:03.839594Z 0 [Note] InnoDB: buffer pool 1 : resizing with chunks 5 to 6.
2018-08-24T07:05:03.848910Z 0 [Note] InnoDB: buffer pool 1 : 1 chunks (8192 blocks) were added.
2018-08-24T07:05:03.849046Z 0 [Note] InnoDB: buffer pool 2 : resizing with chunks 5 to 6.
2018-08-24T07:05:03.856711Z 0 [Note] InnoDB: buffer pool 2 : 1 chunks (8192 blocks) were added.
2018-08-24T07:05:03.856741Z 0 [Note] InnoDB: buffer pool 3 : resizing with chunks 5 to 6.
2018-08-24T07:05:03.864867Z 0 [Note] InnoDB: buffer pool 3 : 1 chunks (8192 blocks) were added.
2018-08-24T07:05:03.864902Z 0 [Note] InnoDB: Completed to resize buffer pool from 2684354560 to 3221225472.
2018-08-24T07:05:03.864915Z 0 [Note] InnoDB: Re-enabled adaptive hash index.
2018-08-24T07:05:03.864935Z 0 [Note] InnoDB: Completed resizing buffer pool at 180824 15:05:03.

 

如下爲減少時，記錄的日誌

2018-08-24T07:10:20.666816Z 2 [Note] InnoDB: Requested to resize buffer pool. (new size: 2684354560 bytes)
2018-08-24T07:10:20.666880Z 0 [Note] InnoDB: Resizing buffer pool from 3221225472 to 2684354560 (unit=134217728).
2018-08-24T07:10:20.666889Z 0 [Note] InnoDB: Disabling adaptive hash index.
2018-08-24T07:10:20.673416Z 0 [Note] InnoDB: disabled adaptive hash index.
2018-08-24T07:10:20.673508Z 0 [Note] InnoDB: Withdrawing blocks to be shrunken.
2018-08-24T07:10:20.673519Z 0 [Note] InnoDB: buffer pool 0 : start to withdraw the last 8192 blocks.
2018-08-24T07:10:20.678441Z 0 [Note] InnoDB: buffer pool 0 : withdrawing blocks. (8192/8192)
2018-08-24T07:10:20.678521Z 0 [Note] InnoDB: buffer pool 0 : withdrew 8192 blocks from free list. Tried to relocate 0 pages (8192/8192).
2018-08-24T07:10:20.678919Z 0 [Note] InnoDB: buffer pool 0 : withdrawn target 8192 blocks.
2018-08-24T07:10:20.678977Z 0 [Note] InnoDB: buffer pool 1 : start to withdraw the last 8192 blocks.
2018-08-24T07:10:20.681644Z 0 [Note] InnoDB: buffer pool 1 : withdrawing blocks. (8192/8192)
2018-08-24T07:10:20.682168Z 0 [Note] InnoDB: buffer pool 1 : withdrew 8192 blocks from free list. Tried to relocate 0 pages (8192/8192).
2018-08-24T07:10:20.682235Z 0 [Note] InnoDB: buffer pool 1 : withdrawn target 8192 blocks.
2018-08-24T07:10:20.682254Z 0 [Note] InnoDB: buffer pool 2 : start to withdraw the last 8192 blocks.
2018-08-24T07:10:20.686560Z 0 [Note] InnoDB: buffer pool 2 : withdrawing blocks. (8192/8192)
2018-08-24T07:10:20.686917Z 0 [Note] InnoDB: buffer pool 2 : withdrew 8192 blocks from free list. Tried to relocate 0 pages (8192/8192).
2018-08-24T07:10:20.687002Z 0 [Note] InnoDB: buffer pool 2 : withdrawn target 8192 blocks.
2018-08-24T07:10:20.687010Z 0 [Note] InnoDB: buffer pool 3 : start to withdraw the last 8192 blocks.
2018-08-24T07:10:20.690038Z 0 [Note] InnoDB: buffer pool 3 : withdrawing blocks. (8192/8192)
2018-08-24T07:10:20.690373Z 0 [Note] InnoDB: buffer pool 3 : withdrew 8192 blocks from free list. Tried to relocate 0 pages (8192/8192).
2018-08-24T07:10:20.690433Z 0 [Note] InnoDB: buffer pool 3 : withdrawn target 8192 blocks.
2018-08-24T07:10:20.690479Z 0 [Note] InnoDB: Latching whole of buffer pool.
2018-08-24T07:10:20.690498Z 0 [Note] InnoDB: buffer pool 0 : resizing with chunks 6 to 5.
2018-08-24T07:10:20.693293Z 0 [Note] InnoDB: buffer pool 0 : 1 chunks (8192 blocks) were freed.
2018-08-24T07:10:20.693357Z 0 [Note] InnoDB: buffer pool 1 : resizing with chunks 6 to 5.
2018-08-24T07:10:20.695947Z 0 [Note] InnoDB: buffer pool 1 : 1 chunks (8192 blocks) were freed.
2018-08-24T07:10:20.696011Z 0 [Note] InnoDB: buffer pool 2 : resizing with chunks 6 to 5.
2018-08-24T07:10:20.698977Z 0 [Note] InnoDB: buffer pool 2 : 1 chunks (8192 blocks) were freed.
2018-08-24T07:10:20.699288Z 0 [Note] InnoDB: buffer pool 3 : resizing with chunks 6 to 5.
2018-08-24T07:10:20.702088Z 0 [Note] InnoDB: buffer pool 3 : 1 chunks (8192 blocks) were freed.
2018-08-24T07:10:20.702398Z 0 [Note] InnoDB: Completed to resize buffer pool from 3221225472 to 2684354560.
2018-08-24T07:10:20.702413Z 0 [Note] InnoDB: Re-enabled adaptive hash index.
2018-08-24T07:10:20.703896Z 0 [Note] InnoDB: Completed resizing buffer pool at 180824 15:10:20.

 

4. 配置的innodb_buffer_pool_size是否合適？

當前配置的innodb_buffer_pool_size是否合適，能夠經過分析InnoDB緩衝池的性能來驗證。

可使用如下公式計算InnoDB緩衝池性能：

Performance = innodb_buffer_pool_reads / innodb_buffer_pool_read_requests * 100

innodb_buffer_pool_reads：表示InnoDB緩衝池沒法知足的請求數。須要從磁盤中讀取。

innodb_buffer_pool_read_requests：表示從內存中讀取邏輯的請求數。

例如，在個人服務器上，檢查當前InnoDB緩衝池的性能：

root@localhost [(none)] 15:35:31>show status like 'innodb_buffer_pool_read%';
+---------------------------------------+-------------+
| Variable_name                         | Value       |
+---------------------------------------+-------------+
| Innodb_buffer_pool_read_ahead_rnd     | 0           |
| Innodb_buffer_pool_read_ahead         | 0           |
| Innodb_buffer_pool_read_ahead_evicted | 0           |
| Innodb_buffer_pool_read_requests      | 4029033624  |
| Innodb_buffer_pool_reads              | 91661       |
+---------------------------------------+-------------+
5 rows in set (0.00 sec)


Performance = 91661 / 4029033624 * 100 = 0.0022750120389663

 

意味着InnoDB能夠知足緩衝池自己的大部分請求。從磁盤完成讀取的百分比很是小。所以無需增長innodb_buffer_pool_size值。

4.1 何時減少innodb_buffer_pool_size？

在專用MySQL服務器上，多餘的innodb_buffer內存不會有問題，可是當使用共享服務器時，可能會有性能影響。由於空閒內存對其餘程序和操做系統頗有用。

可使用SHOW ENGINE INNODB STATUS\G命令檢查內存狀態：

mysql> show engine innodb status\G

...
Total large memory allocated 26386366464
Dictionary memory allocated 23826297
Buffer pool size   1572672
Free buffers       8192
Database pages     1553364
Old database pages 573246
Modified db pages  36
Pending reads      0
Pending writes: LRU 0, flush list 0, single page 0
Pages made young 881819, not young 18198964
0.02 youngs/s, 0.05 non-youngs/s
Pages read 681064, created 2749237, written 3988300
0.02 reads/s, 0.12 creates/s, 11.50 writes/s
Buffer pool hit rate 1000 / 1000, young-making rate 0 / 1000 not 0 / 1000
Pages read ahead 0.00/s, evicted without access 0.00/s, Random read ahead 0.00/s
LRU len: 1553364, unzip_LRU len: 0
I/O sum[5152]:cur[0], unzip sum[0]:cur[0]
...

 

Free buffers :表示有多少空閒buffer。若是 此值長時間都較高，則能夠考慮減少InnoDB緩衝池大小。

InnoDB buffer pool 命中率：

InnoDB buffer pool 命中率 = innodb_buffer_pool_read_requests / (innodb_buffer_pool_read_requests + innodb_buffer_pool_reads ) * 100

此值低於99%，則能夠考慮增長innodb_buffer_pool_size。

5. InnoDB緩衝池狀態變量有哪些？

能夠運行如下命令進行查看：

root@localhost [(none)] 16:00:31>show global status like '%innodb_buffer_pool_pages%';
+----------------------------------+--------+
| Variable_name                    | Value  |
+----------------------------------+--------+
| Innodb_buffer_pool_pages_data    | 457    |
| Innodb_buffer_pool_pages_dirty   | 0      |
| Innodb_buffer_pool_pages_flushed | 36     |
| Innodb_buffer_pool_pages_free    | 163363 |
| Innodb_buffer_pool_pages_misc    | 0      |
| Innodb_buffer_pool_pages_total   | 163820 |
+----------------------------------+--------+
6 rows in set (0.00 sec)

 

說明：

  • Innodb_buffer_pool_pages_data
    InnoDB緩衝池中包含數據的頁數。 該數字包括髒頁面和乾淨頁面。 使用壓縮表時，報告的Innodb_buffer_pool_pages_data值可能大於Innodb_buffer_pool_pages_total（Bug＃59550）。

  • Innodb_buffer_pool_pages_dirty
    顯示在內存中修改但還沒有寫入數據文件的InnoDB緩衝池數據頁的數量（髒頁刷新）。

  • Innodb_buffer_pool_pages_flushed
    表示從InnoDB緩衝池中刷新髒頁的請求數。

  • Innodb_buffer_pool_pages_free
    顯示InnoDB緩衝池中的空閒頁面

  • Innodb_buffer_pool_pages_misc
    InnoDB緩衝池中的頁面數量不少，由於它們已被分配用於管理開銷，例如行鎖或自適應哈希索引。此值也能夠計算爲Innodb_buffer_pool_pages_total - Innodb_buffer_pool_pages_free - Innodb_buffer_pool_pages_data。

  • Innodb_buffer_pool_pages_total
    InnoDB緩衝池的總大小，以page爲單位。

  • innodb_buffer_pool_reads
    表示InnoDB緩衝池沒法知足的請求數。須要從磁盤中讀取。

  • innodb_buffer_pool_read_requests
    它表示從內存中邏輯讀取的請求數。

  • innodb_buffer_pool_wait_free
    一般，對InnoDB緩衝池的寫入發生在後臺。 當InnoDB須要讀取或建立頁面而且沒有可用的乾淨頁面時，InnoDB首先刷新一些髒頁並等待該操做完成。 此計數器計算這些等待的實例。 若是已正確設置innodb_buffer_pool_size，則此值應該很小。若是大於0，則表示InnoDb緩衝池過小。

  • innodb_buffer_pool_write_request
    表示對緩衝池執行的寫入次數。

6. InnoDB緩衝池當前使用了多少實際GB內存？

經過將緩衝池中可用的數據與InnoDB頁面（InnoDB緩衝池單位）大小相乘，咱們能夠發現InnoDB緩衝池此時正在使用的實際內存。

set @ibpdata = (select variable_value from information_schema.global_status where variable_name = 'innodb_buffer_pool_pages_data');

ERROR 3167 (HY000): The 'INFORMATION_SCHEMA.GLOBAL_STATUS' feature is disabled; see the documentation for 'show_compatibility_56'

#從MySQL 5.7.6開始，GLOBAL_STATUS表中提供的信息從Performance Schema獲取

mysql> set @ibpdata = (select variable_value from performance_schema.global_status where variable_name = 'innodb_buffer_pool_pages_data');

mysql> select @ibpdata;
+----------+
| @ibpdata |
+----------+
| 568      |
+----------+
1 row in set (0.00 sec)


mysql> set @idbpgsize = (select variable_value from performance_schema.global_status where variable_name = 'innodb_page_size');

mysql> select @idbpgsize;
+------------+
| @idbpgsize |
+------------+
| 16384      |
+------------+
1 row in set (0.00 sec)

mysql> set @ibpsize = @ibpdata * @idbpgsize / (1024*1024*1024);
Query OK, 0 rows affected (0.00 sec)

mysql> select @ibpsize;
+-----------------+
| @ibpsize        |
+-----------------+
| 0.0086669921875 |
+-----------------+
1 row in set (0.00 sec)
相關文章
相關標籤/搜索
每日一句
    每一个你不满意的现在，都有一个你没有努力的曾经。
本站公眾號
   歡迎關注本站公眾號,獲取更多信息
聯繫我們 最近搜索 最新文章 沪ICP备13005482号-10 MyBatis教程 SQL 教程 MySQL教程 Java 教程 Thymeleaf 教程 Hibernate教程 Spring教程 Redis教程