使用innodb_ruby探查Innodb索引結構

　　innodb_ruby 是使用 Ruby 編寫的 InnoDB 文件格式解析器。innodb_ruby 的目的是暴露一些其餘隱藏的 InnoDB 原理。

　　innodb_ruby不適合使用於生產環境，但能夠做爲學習工具來使用。

• ①、安裝

　　如下安裝參考MySQL大師知數堂吳炳錫老師的blog.

　　　　下載

[root@MySQL56_L1 mysql]# wget https://cache.ruby-china.org/pub/ruby/ruby-1.9.3-p551.tar.gz
--2017-01-19 14:06:35--  https://cache.ruby-china.org/pub/ruby/ruby-1.9.3-p551.tar.gz
Resolving cache.ruby-china.org... 183.61.64.66, 121.201.98.27, 2405:fd80:110:0:d63d:7eff:fe73:c46, ...
Connecting to cache.ruby-china.org|183.61.64.66|:443... connected.
ERROR: certificate common name 「*.b0.upaiyun.com」 doesn’t match requested host name 「cache.ruby-china.org」.
To connect to cache.ruby-china.org insecurely, use ‘--no-check-certificate’.

[root@MySQL56_L1 mysql]# wget --no-check-certificate https://cache.ruby-china.org/pub/ruby/ruby-1.9.3-p551.tar.gz
--2017-01-19 14:07:31--  https://cache.ruby-china.org/pub/ruby/ruby-1.9.3-p551.tar.gz
Resolving cache.ruby-china.org... 121.201.98.27, 183.61.64.66, 2405:fd80:110:0:d63d:7eff:fe73:165a, ...
Connecting to cache.ruby-china.org|121.201.98.27|:443... connected.
WARNING: certificate common name 「*.b0.upaiyun.com」 doesn’t match requested host name 「cache.ruby-china.org」.
HTTP request sent, awaiting response... 200 OK
Length: 12605119 (12M) [application/octet-stream]
Saving to: 「ruby-1.9.3-p551.tar.gz」

100%[=================================================================================================================>] 12,605,119   287K/s   in 47s     

2017-01-19 14:08:23 (262 KB/s) - 「ruby-1.9.3-p551.tar.gz」 saved [12605119/12605119]

　　　　安裝依賴

[root@MySQL56_L1 mysql]# yum -y install zlib-devel curl-devel openssl-devel httpd-devel apr-devel apr-util-devel

　　　　解壓

[root@MySQL56_L1 mysql]# tar -zxvf ruby-1.9.3-p551.tar.gz 

　　　　配置&安裝

[root@MySQL56_L1 ruby-1.9.3-p551]# ./configure
[root@MySQL56_L1 ruby-1.9.3-p551]# make && make install
[root@MySQL56_L1 ruby-1.9.3-p551]# gem install innodb_ruby

• ②、innodb_ruby的使用說明

　　　　工欲善其事，必先利其器，使用以前要先查看幫助，知數堂吳炳錫老師的blog上也有介紹做者的github地址。但更詳細的使用方法仍是使用--help進行查看

[root@MySQL56_L1 data]# innodb_space --help

Usage: innodb_space <options> <mode> Invocation examples: innodb_space -s ibdata1 [-T tname [-I iname]] [options] <mode> Use ibdata1 as the system tablespace and load the tname table (and the iname index for modes that require it) from data located in the system tablespace data dictionary. This will automatically generate a record describer for any indexes. innodb_space -f tname.ibd [-r ./desc.rb -d DescClass] [options] <mode> Use the tname.ibd table (and the DescClass describer where required). The following options are supported: --help, -? Print this usage text. --trace, -t Enable tracing of all data read. Specify twice to enable even more tracing (including reads during opening of the tablespace) which can be quite noisy. --system-space-file, -s <arg> Load the system tablespace file or files <arg>: Either a single file e.g. "ibdata1", a comma-delimited list of files e.g. "ibdata1,ibdata1", or a directory name. If a directory name is provided, it will be scanned for all files named "ibdata?" which will then be sorted alphabetically and used to load the system tablespace. --table-name, -T <name> Use the table name <name>. --index-name, -I <name> Use the index name <name>. --space-file, -f <file> Load the tablespace file <file>. --page, -p <page> Operate on the page <page>. --level, -l <level> Operate on the level <level>. --list, -L <list> Operate on the list <list>. --require, -r <file> Use Ruby's "require" to load the file <file>. This is useful for loading  classes with record describers. --describer, -d <describer> Use the named record describer to parse records in index pages. The following modes are supported: system-spaces Print a summary of all spaces in the system. data-dictionary-tables Print all records in the SYS_TABLES data dictionary table. data-dictionary-columns Print all records in the SYS_COLUMNS data dictionary table. data-dictionary-indexes Print all records in the SYS_INDEXES data dictionary table. data-dictionary-fields Print all records in the SYS_FIELDS data dictionary table. space-summary Summarize all pages within a tablespace. A starting page number can be provided with the --page/-p argument. space-index-pages-summary Summarize all "INDEX" pages within a tablespace. This is useful to analyze page fill rates and record counts per page. In addition to "INDEX" pages, "ALLOCATED" pages are also printed and assumed to be completely empty. A starting page number can be provided with the --page/-p argument. space-index-pages-free-plot Use Ruby's gnuplot module to produce a scatterplot of page free space for all "INDEX" and "ALLOCATED" pages in a tablespace. More aesthetically pleasing plots can be produced with space-index-pages-summary output, but this is a quick and easy way to produce a passable plot. A starting page number can be provided with the --page/-p argument. space-page-type-regions Summarize all contiguous regions of the same page type. This is useful to provide an overall view of the space and allocations within it. A starting page number can be provided with the --page/-p argument. space-page-type-summary Summarize all pages by type. A starting page number can be provided with the --page/-p argument. space-indexes Summarize all indexes (actually each segment of the indexes) to show the number of pages used and allocated, and the segment fill factor. space-lists Print a summary of all lists in a space. space-list-iterate Iterate through the contents of a space list. space-extents Iterate through all extents, printing the extent descriptor bitmap. space-extents-illustrate Iterate through all extents, illustrating the extent usage using ANSI color and Unicode box drawing characters to show page usage throughout the space. space-extents-illustrate-svg Iterate through all extents, illustrating the extent usage in SVG format printed to stdout to show page usage throughout the space. space-lsn-age-illustrate Iterate through all pages, producing a heat map colored by the page LSN using ANSI color and Unicode box drawing characters, allowing the user to get an overview of page modification recency. space-lsn-age-illustrate-svg Iterate through all pages, producing a heat map colored by the page LSN producing SVG format output, allowing the user to get an overview of page modification recency. space-inodes-summary Iterate through all inodes, printing a short summary of each FSEG. space-inodes-detail Iterate through all inodes, printing a detailed report of each FSEG. index-recurse Recurse an index, starting at the root (which must be provided in the first --page/-p argument), printing the node pages, node pointers (links), leaf pages. A record describer must be provided with the --describer/-d argument to recurse indexes (in order to parse node pages). index-record-offsets Recurse an index as index-recurse does, but print the offsets of each record within the page. index-digraph Recurse an index as index-recurse does, but print a dot-compatible digraph instead of a human-readable summary. index-level-summary Print a summary of all pages at a given level (provided with the --level/-l argument) in an index. index-fseg-internal-lists index-fseg-leaf-lists Print a summary of all lists in an index file segment. Index root page must be provided with --page/-p. index-fseg-internal-list-iterate index-fseg-leaf-list-iterate Iterate the file segment list (whose name is provided in the first --list/-L argument) for internal or leaf pages for a given index (whose root page is provided in the first --page/-p argument). The lists used for each index are "full", "not_full", and "free". index-fseg-internal-frag-pages index-fseg-leaf-frag-pages Print a summary of all fragment pages in an index file segment. Index root page must be provided with --page/-p. page-dump Dump the contents of a page, using the Ruby pp ("pretty-print") module. page-account Account for a page's usage in FSEGs.  page-validate Validate the contents of a page. page-directory-summary Summarize the record contents of the page directory in a page. If a record describer is available, the key of each record will be printed. page-records Summarize all records within a page. page-illustrate Produce an illustration of the contents of a page. record-dump Dump a detailed description of a record and the data it contains. A record offset must be provided with -R/--record. record-history Summarize the history (undo logs) for a record. A record offset must be provided with -R/--record. undo-history-summary Summarize all records in the history list (undo logs). undo-record-dump Dump a detailed description of an undo record and the data it contains. A record offset must be provided with -R/--record. [root@MySQL56_L1 data]# 

• ③、使用innodb_ruby進行測試

　　　　建立測試表

CREATE TABLE `t2` (
  `id` int(11) NOT NULL AUTO_INCREMENT,
  `name` varchar(50) DEFAULT NULL,
  `remark` varchar(50) DEFAULT NULL,
  `add_time` datetime DEFAULT NULL,
  PRIMARY KEY (`id`),
  KEY `ix_t2_name` (`name`)
) ENGINE=InnoDB AUTO_INCREMENT=3829 DEFAULT CHARSET=utf8

　　　　插入測試數據(請移步到百度網盤)

　　　　使用innodb_space來查看t2表的索引結構、數據分配狀況

　　　　space-indexes：Summarize all indexes (actually each segment of the indexes) to show the number of pages used and allocated, and the segment fill factor（）

[root@MySQL56_L1 data]# innodb_space -s ibdata1 -T zeno3376/t2 space-indexes
id          name                            root        fseg        used        allocated   fill_factor
42          PRIMARY                         3           internal    1           1           100.00%     
42          PRIMARY                         3           leaf        9           9           100.00%     
43          ix_t2_name                      4           internal    1           1           100.00%     
43          ix_t2_name                      4           leaf        4           4           100.00%   

　　　　name:索引的名稱；PRIMARY表明的就是彙集索引，由於InnoDB表是彙集因此組織表，行記錄就是彙集索引；ix_t2_name就是輔助索引的名稱

　　　　root:索引中根節點的page number；能夠看出彙集索引的根節點是第3個page（爲何是從第三個page開始，看下文space-page-type-regions），輔助索引的根節點是第4個page

　　　　fseg:page的說明

　　　　used:索引使用了多少個page；能夠看出彙集索引的根幾點使用了1個page，葉子節點使用了9個page；輔助索引ix_t2_name的葉子節點使用了4個page

　　　　allocated: 索引分配了多少個page；能夠看出彙集索引的根幾點分配了1個page，葉子節點分配了9個page；輔助索引ix_t2_name的葉子節點分配了4個page

　　　　fill_factor:索引的填充度；全部的填充度都是100%

　　　　space-page-type-regions：Summarize all contiguous regions of the same page type. This is useful to  provide an overall view of the space and allocations within it. A starting  page number can be provided with the --page/-p argument.（統計每一個類型的頁共佔用了多少頁）

[root@MySQL56_L1 data]# innodb_space -s ibdata1 -T zeno3376/t2  space-page-type-regions
start       end         count       type               
0           0           1           FSP_HDR             
1           1           1           IBUF_BITMAP         
2           2           1           INODE               
3           17          15          INDEX               
18          18          1           FREE (ALLOCATED)   

　　　　start：從第幾個page開始

　　　　end：從第幾個page結束

　　　　count：佔用了多少個page；

　　　　type: page的類型

　　　　從上面的結果能夠看出：「FSP_HDR」、「IBUF_BITMAP」、「INODE」是分別佔用了0,1,2號的page，從3號page開始纔是存放數據和索引的頁（Index），佔用了3~17號的page，共15個page。

　　　　接下來，根據獲得的彙集索引和輔助索引的根節點來獲取索引上的其餘page的信息

　　　　page-records:Summarize all records within a page.

# 解析表（彙集索引組織表，這裏不須要加-I primary，不然會報錯）
[root@MySQL56_L1 data]# innodb_space -s ibdata1 -T zeno3376/t2 -I primary -p 3 page-records           
/usr/local/lib/ruby/gems/1.9.1/gems/innodb_ruby-0.9.15/lib/innodb/system.rb:213:in `index_by_name': undefined method `[]' for nil:NilClass (NoMethodError)
        from /usr/local/lib/ruby/gems/1.9.1/gems/innodb_ruby-0.9.15/bin/innodb_space:1913:in `<top (required)>'
        from /usr/local/bin/innodb_space:23:in `load'
        from /usr/local/bin/innodb_space:23:in `<main>'

[root@MySQL56_L1 data]# innodb_space -s ibdata1 -T zeno3376/t2 -p 3 page-records           
Record 126: (id=1782) → #5
Record 140: (id=1890) → #6
Record 154: (id=2101) → #7
Record 168: (id=2317) → #10
Record 182: (id=2531) → #11
Record 196: (id=2747) → #12
Record 210: (id=2964) → #15
Record 224: (id=3179) → #16
Record 238: (id=3394) → #17

# -p 3 就是解析3號page的意思

　　　　上面的結果是解析彙集索引根節點頁的信息，1行就表明使用了1個page，因此，葉子節點共使用了9個page，根節點使用了1個page，跟space_indexes的解析結果一致。

　　　　Record 126: (id=1782) → #5 

　　　   id = 1782 表明的就是表中id爲1782的記錄，由於id是主鍵

　　　　-> #5 表明的是指向5號page

　　　　Record 126: (id=1782) → #5： 整行的意思就是5號page的id最小值是1782，包含了1782~1889的行記錄。

　　　　注意：page number並非連續的

　　　　根據解析root獲得的信息，繼續解析第一個葉子節點的信息

[root@MySQL56_L1 data]# innodb_space -s ibdata1 -T zeno3376/t2 -p 5 page-records
Record 128: (id=1782) → (name="zeno", remark="mysql", add_time=:NULL)

Record 162: (id=1783) → (name="KIK91QJET1FCZ46EJKML", remark="H4HJO5F7W5GSSDORT8AAT", add_time="184524556-49-63 92:14:08")

Record 233: (id=1784) → (name="XQZJ08164WSB2EI9M3HCWCEZZOXNB6", remark="8878ASA5AW", add_time="184524556-50-65 04:03:84")

Record 303: (id=1785) → (name="XAXK7RVVTYWEXB2ZFB", remark="TVZNZPW150ZNNJAC1", add_time="184524556-50-65 16:99:20")

Record 368: (id=1786) → (name="G0BZFYV26V14", remark="CYYVCNQJVDQ4OLO6YBZ", add_time="184524556-50-65 63:27:68")

…… # 注：已截斷部分數據

Record 7187: (id=1885) → (name="XQ2E35QOX32I5GL0TH", remark="SZ4QTI116S3ISRZOJL0M", add_time="184524556-52-49 52:32:00")

Record 7255: (id=1886) → (name="S127FSHO2IPIE2", remark="2EX67306JBI7AL9Z", add_time="184524556-52-49 72:18:56")

Record 7315: (id=1887) → (name="2XKN9VXB5561923IPKVMBW", remark="6ZBU7PRXNDUHR4DV2PB", add_time="184524556-52-49 19:62:88")

Record 7386: (id=1888) → (name="42R60NM6IMTNHRB1L", remark="UG3GLX6ONU5", add_time="184524556-52-49 45:81:76")

Record 7444: (id=1889) → (name="0O2S6OCUC99MQKM1", remark="1K5GJEQ5QU83T3F", add_time="184524556-52-49 32:71:04")

　　　　從上面能夠看出，彙集索引的葉子節點是包含了行記錄的全部數據。

　　　　同理，解析輔助索引ix_t2_name，可是須要注意的是，在解析輔助索引是，須要加上「-I ix_t2_name」

[root@MySQL56_L1 data]# innodb_space -s ibdata1 -T zeno3376/t2 -I ix_t2_name -p 4 page-records
Record 127: (name="01EE2CCYUW35K0LVT5DAG2044NW") → #8
Record 196: (name="8WCS36CV56KGA8NE6OG23QFS") → #13
Record 169: (name="HQVX6ZX7H2XI") → #9
Record 235: (name="QXS8RUJF6FY") → #14

　　　　從上面能夠出，輔助索引ix_t2_name的key是name列，葉子節點共使用了4個page，加上根節點，那麼輔助索引ix_t2_name共使用了5個page，跟使用space_indexes解析出來的結果一致。

　　　　Record 127: (name="01EE2CCYUW35K0LVT5DAG2044NW") → #8 這條記錄表明的意思是輔助索引的第1個葉子節點的page number是8，8號page的第一個key值是"01EE2CCYUW35K0LVT5DAG2044NW"

　　　　Record 196: (name="8WCS36CV56KGA8NE6OG23QFS") → #13 這條記錄表明的意思是輔助索引的第2個葉子節點的page number是13，13號page的第一個key值是"8WCS36CV56KGA8NE6OG23QFS"

　　　　其它的記錄如此類推……

　　　　接下來看看輔助索引的葉子節點的結構

[root@MySQL56_L1 data]# innodb_space -s ibdata1 -T zeno3376/t2 -I ix_t2_name -p 8 page-records
Record 127: (name="01EE2CCYUW35K0LVT5DAG2044NW") → (id=1855)

Record 165: (name="02RFY8SJLQ879F2CYHI") → (id=2132)

Record 10829: (name="04FNKNM16R7U27A3") → (id=3152)

Record 195: (name="06WM2Q51B0D8L76VM2") → (id=2184)

Record 224: (name="0739V9NMP4") → (id=1843)

……# 注：已截斷部分信息

Record 8197: (name="8U4049BA2TAAY7A89SDG") → (id=2003)

Record 10591: (name="8UQOOOU7X5AYE75GU") → (id=3111)

Record 8228: (name="8V5C6OGK4NGAHE6") → (id=2247)

Record 12607: (name="8V6SFJ0P8E1XKIF005QD3NTCI") → (id=3435)

Record 9595: (name="8VK9HHEN3G") → (id=2972)

　　　　從上面能夠看到葉子節點中包含可輔助索引和主鍵列

　　　　Record 127: (name="01EE2CCYUW35K0LVT5DAG2044NW") → (id=1855) 表明的意思就是name值爲"01EE2CCYUW35K0LVT5DAG2044NW"的記錄指向主鍵id=1855的行記錄。

　　　　其餘的記錄同理。

 

　　　　以上，若有錯謬，請不吝指出。