pg_lightool-postgres數據分佈查看工具

時間 2019-12-15

標籤 lightool postgres 數據分佈查看工具简体版

原文原文鏈接

1、前言

數據庫在通過長時間的運行以後，有效數據不規則的分散在數據文件的各個地方。如何能得知每一個page，每一個segment到底存儲了多少數據呢？做者DIY了這個功能'datadis'到pg_lightool工具中。html

2、源碼下載

下載地址：http://gitee.com/movead/pg_lightoolnode

目前只有源碼編譯安裝版本。git

3、代碼編譯安裝

1.首先須要配置postgres的PATH路徑sql

2.進入下載的代碼的pg_lightool目錄數據庫

3.make;make install;centos

這樣就pg_lightool工具就成功安裝到postgres的bin目錄下了數組

4、工具使用

現有表

postgres=# select relfilenode from pg_class where relname ='t2';
 relfilenode 
-------------
       16904
(1 row)

postgres=# \d t2
\                      Table "public.t2"
 Column |       Type        | Collation | Nullable | Default 
--------+-------------------+-----------+----------+---------
 i      | integer           |           |          | 
 j      | integer           |           |          | 
 k      | character varying |           |          | 

postgres=#\q
  
[lchch@yfslcentos71 13211]$ ll 16904*
-rw------- 1 lchch lchch 1073741824 Aug 26 19:39 16904
-rw------- 1 lchch lchch 1073741824 Aug 26 20:24 16904.1
-rw------- 1 lchch lchch       8192 Aug 26 20:22 16904.2
-rw------- 1 lchch lchch     548864 Aug 26 20:22 16904_fsm
-rw------- 1 lchch lchch      40960 Aug 26 19:50 16904_vm
[lchch@yfslcentos71 13211]$

執行命令

[lchch@yfslcentos71 ~]$ pg_lightool datadis -f 0/13211/16904 -D /work/opt/postgres/20180728/data -p /home/lchch -g 1 -d
Start Datadis Analyse...
Datadis Analyse Success.
[lchch@yfslcentos71 ~]$

解析結果

某些解析會致使大量的輸出，因此輸出結果存儲在-p指定的目錄下，電影下載，目前格式不是很美觀，各位看官不要介意。工具

[lchch@yfslcentos71 ~]$ cat datadis.txt 
---------------------------------------------------
FILE:16904.0 use ratio:98%
Free Items Num:       0
Use Items Num:        20578304
Items Size:           164626432
Page Head Size:       3145728
Page Special Size:    0
Maybe Tuple Use Size: 0
Sure Tuple Use Size:  884867072
Free Size:            21102592
---------------------------------------------------
---------------------------------------------------
FILE:16904.1 use ratio:98%
Free Items Num:       0
Use Items Num:        20578304
Items Size:           164626432
Page Head Size:       3145728
Page Special Size:    0
Maybe Tuple Use Size: 0
Sure Tuple Use Size:  884867072
Free Size:            21102592
---------------------------------------------------
---------------------------------------------------
FILE:16904.2 use ratio:98%
Free Items Num:       0
Use Items Num:        157
Items Size:           1256
Page Head Size:       24
Page Special Size:    0
Maybe Tuple Use Size: 0
Sure Tuple Use Size:  6751
Free Size:            161
---------------------------------------------------
###################################################

改變表內數據分佈

postgres=# delete from t2 where i % 2 = 1;
DELETE 20578383
postgres=# update t2 set k = 'https://www.oschina.net/' where i < 1000;
UPDATE 1076
postgres=# checkpoint;
CHECKPOINT
postgres=#

再次解析的結果

# 執行命令
[lchch@yfslcentos71 ~]$ pg_lightool datadis -f 0/13211/16904 -D /work/opt/postgres/20180728/data -p /home/lchch -g 1 -d
Start Datadis Analyse...
Datadis Analyse Success.
[lchch@yfslcentos71 ~]$ 

# 結果查詢
[lchch@yfslcentos71 ~]$ cat datadis.txt 
---------------------------------------------------
FILE:16904.0 use ratio:45%
Free Items Num:       10289230
Use Items Num:        10289432
Items Size:           82314648
Page Head Size:       3145728
Page Special Size:    0
Maybe Tuple Use Size: 18103
Sure Tuple Use Size:  442432485
Free Size:            545830860
---------------------------------------------------
---------------------------------------------------
FILE:16904.1 use ratio:45%
Free Items Num:       10289153
Use Items Num:        10289564
Items Size:           82314868
Page Head Size:       3145728
Page Special Size:    0
Maybe Tuple Use Size: 24811
Sure Tuple Use Size:  442432223
Free Size:            545824194
---------------------------------------------------
---------------------------------------------------
FILE:16904.2 use ratio:68%
Free Items Num:       78
Use Items Num:        384
Items Size:           1848
Page Head Size:       96
Page Special Size:    0
Maybe Tuple Use Size: 0
Sure Tuple Use Size:  20782
Free Size:            10042
---------------------------------------------------
###################################################
RELFILENODE:16904 Use Ratio:45%
Free Items Num:       20578461
Use Items Num:        20579380
Items Size:           164631364
Page Head Size:       6291552
Page Special Size:    0
Maybe Tuple Use Size: 42914
Sure Tuple Use Size:  884885490
Free Size:            1091665096
###################################################
[lchch@yfslcentos71 ~]$

其餘的查詢方式

# 命令執行
[lchch@yfslcentos71 ~]$ pg_lightool datadis -f 0/13211/16904 -D /work/opt/postgres/20180728/data -p /home/lchch -g 1
Start Datadis Analyse...
Datadis Analyse Success.
[lchch@yfslcentos71 ~]$ 

# 解析結果
[lchch@yfslcentos71 ~]$ cat datadis.txt 
FILE:16904.0 use ratio:45%
FILE:16904.1 use ratio:45%
FILE:16904.2 use ratio:68%
###################################################
RELFILENODE:16904 Use Ratio:45%
###################################################
[lchch@yfslcentos71 ~]$

# 執行命令
[lchch@yfslcentos71 ~]$ pg_lightool datadis -f 0/13211/16904 -D /work/opt/postgres/20180728/data -p /home/lchch -g 2
Start Datadis Analyse...
Datadis Analyse Success.
[lchch@yfslcentos71 ~]$ 

# 解析結果
vi datadis.txt 
PAGE:0(0) OF FILE:16904 use ratio:41%
PAGE:1(1) OF FILE:16904 use ratio:45%
PAGE:2(2) OF FILE:16904 use ratio:45%
PAGE:3(3) OF FILE:16904 use ratio:45%
...
PAGE:131069(262141) OF FILE:16904.1 use ratio:45%
PAGE:131070(262142) OF FILE:16904.1 use ratio:45%
PAGE:131071(262143) OF FILE:16904.1 use ratio:45%
PAGE:0(262144) OF FILE:16904.2 use ratio:45%
PAGE:1(262145) OF FILE:16904.2 use ratio:89%
PAGE:2(262146) OF FILE:16904.2 use ratio:89%
PAGE:3(262147) OF FILE:16904.2 use ratio:48%
###################################################
RELFILENODE:16904 Use Ratio:45%
###################################################
[lchch@yfslcentos71 ~]$

注意：以下，執行的爲pageinspect命令，這個命令對單獨的page就行數據統計，站長站，並打印每一個元組的詳細狀況，再也不是datadis命令。post

# 執行命令
[lchch@yfslcentos71 ~]$ pg_lightool pageinspect -f 0/13211/16904 -D /work/opt/postgres/20180728/data -p /home/lchch -b 0
Start Page Inspect...
Page Inspect Success.
[lchch@yfslcentos71 ~]$ 

# 結果解析
vi pageinspect.txt 
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
PAGE:0(0) OF FILE:16904 use ratio:41%
Free Items Num:       157
Use Items Num:        55
Items Size:           848
Page Head Size:       24
Page Special Size:    0
Maybe Tuple Use Size: 0
Sure Tuple Use Size:  3135
Free Size:            4185
----------------------------------------------------------------------------
LP   STATE       XMIN      XMAX      TUPLELEN  HOFF      OFFSET RLP
----------------------------------------------------------------------------
0    INVALID     0         0         0         0         0      (0,0)
1    INVALID     0         0         0         0         158    (0,0)
2    INVALID     0         0         0         0         0      (0,0)
3    INVALID     0         0         0         0         159    (0,0)
4    INVALID     0         0         0         0         0      (0,0)
5    INVALID     0         0         0         0         160    (0,0)
6    INVALID     0         0         0         0         0      (0,0)
7    INVALID     0         0         0         0         161    (0,0)
...
202  NOMAL       757       0         57        24        5248   (0,202)
203  NOMAL       757       0         57        24        5184   (0,203)
204  NOMAL       757       0         57        24        5120   (0,204)
205  NOMAL       757       0         57        24        5056   (0,205)
206  NOMAL       757       0         57        24        4992   (0,206)
207  NOMAL       757       0         57        24        4928   (0,207)
208  NOMAL       757       0         57        24        4864   (0,208)
209  NOMAL       757       0         57        24        4800   (0,209)
210  NOMAL       757       0         57        24        4736   (0,210)
211  NOMAL       757       0         57        24        4672   (0,211)

-g -d -s各類不一樣的參數組合含有更多不一樣的輸出結果。centos7

參數說明

pg_lightool -?
pg_lightool is a light tool of postgres

Usage:
  pg_lightool OPTION blockrecover
  pg_lightool OPTION walshow
  pg_lightool OPTION datadis      //數據分佈狀況查詢
  pg_lightool OPTION pageinspect  //page詳情查詢

Common Options:
  -V, --version                         output version information, then exit

For blockrecover:
  -l, --log                             whether to write a debug info
  -f, --relnode=spcid/dbid/relfilenode specify files to repair
  -b, --block=n1[,n2,n3]                specify blocks to repair(10 limit)
  -w, --walpath=walpath                 wallog read from
  -D, --pgdata=datapath                 data dir of database
  -i, --immediate			            does not do a backup for old file

For datadis:
  -f, --relnode=spcid/dbid/relfilenode  /指定要查詢分佈的表
  -D, --pgdata=datapath                 //數據文件路徑
  -p, --place=outPtah                   //存放解析結果的路徑
  -g, --grade=level                     1 顯示segfile級別的使用率(默認);
                                        2 顯示page級別的使用率;
                                        3 所有顯示;
  -d, --detail		                    是否顯示詳細使用信息
  -s, --small		                    顯示使用率比此項小的結果

For pageinspect:
  -f, --relnode=spcid/dbid/relfilenode  //page存在的表
  -D, --pgdata=datapath                 //數據文件路徑
  -p, --place=outPtah                   //存放解析結果的路徑
  -b, --block=blkno                     //指定要查詢的page
[lchch@yfslcentos71 ~]$