pt-table-checksum校驗與pt-table-sync,前者主要用於數據的校驗,驗證主從是否一致,後者主要用來修復數據,二者通常狀況結合起來用能夠修復數據不一致的問題。mysql
下載工具包 的最新地址以下:sql
https://www.percona.com/downloads/percona-toolkit/LATEST/
安裝pt-table-checksum 和pt-table-sync命令。須要先安裝percona-toolkit 工具集數據庫
1.1percona-toolkit 工具集安裝方式介紹:
percona-toolkit 工具安裝有3種方式:
第一種是下載二進制包,直接解壓包,就可使用。
第二種是下載源碼包,進行編譯安裝
第三種是rpm包安裝
不論是二進制安裝仍是源碼包編譯安裝,仍是rpm包安裝,安裝前要下載相關的依賴軟件包ide
yum ×××tall perl perl-devel perl-CPAN perl-DBD-MySQL perl-Time-HiRes perl-DBI perl-Digest-MD5 -y
1.2下載源碼包編譯安裝:工具
yum ×××tall perl perl-devel perl-CPAN perl-DBD-MySQL perl-Time-HiRes perl-DBI perl-Digest-MD5 -y tar xf percona-toolkit-3.0.13.tar.gz [root@localhost ~]# cd percona-toolkit-3.0.13 [root@localhost percona-toolkit-3.0.13]# perl Makefile.PL Checking if your kit is complete... Looks good Writing Makefile for percona-toolkit make && make ×××tall
1.3下載二進制包直接解壓安裝:學習
yum ×××tall perl perl-devel perl-CPAN perl-DBD-MySQL perl-Time-HiRes perl-DBI perl-Digest-MD5 -y tar xf percona-toolkit-3.0.11_x86_64.tar.gz -C /usr/local/ mv percona-toolkit-3.0.11_x86_64 /usr/local/percona-toolkit [root@ks-es11 local]# cat /etc/profile.d/percona-toolkit.sh export PATH=$PATH:/usr/local/percona-toolkit/bin [root@ks-es11 local]# cat /etc/profile.d/mysql.sh export PATH=$PATH:/usr/local/mysql7/bin
master 192.168.0.39
slave 192.168.0.11測試
2.一、master上操做:this
mysql -uroot -p'jiawu256789' -e "ggrant replication slave on *.* to rept@'192.168.0.11' identified by 'JuwoSdk21TbUser'; flush privileges;" mysqldump -uroot -p'jiawu256789' -B -A -F --master-data=2 --single-transaction --set-gtid-purged=OFF --events|gzip >/opt/juwo_$(date +%F).sql.gz scp -rp -P10239 /opt/juwo_$(date +%F).sql.gz root@192.168.0.11:/root
2.二、slave上操做:日誌
gzip -d /root/juwo_$(date +%F).sql.gz mysql -uroot -p'jiawu256789' -e "source /root/juwo_$(date +%F).sql;" mysql -uroot -p'jiawu256789' -e "CHANGE MASTER TO MASTER_HOST='192.168.0.39',MASTER_PORT=3306,MASTER_USER='rept',MASTER_PASSWORD='JuwoSdk21TbUser',MASTER_AUTO_POSITION = 1;start slave;show slave status\G"
建立pt-table-checksum 和pt-table-sync所須要的操做mysql的帳戶以及修復數據時,所須要的權限code
登錄slave和master(兩邊都要執行以下SQL)
master(192.168.0.18)
slave(192.168.0.22)
grant update,×××ert,select,create,drop,delete,index,execute,super,process,replication slave on *.* to ptsum@'192.168.0.%' identified by 'ptchecksums'; flush privileges;
master庫上新建測試庫和測試表,SQL指令以下:
create database test01; use test01; CREATE TABLE `frame01` ( `id` int(11) NOT NULL AUTO_INCREMENT,`parent_id` int(11) DEFAULT NULL,`dsn` varchar(255) NOT NULL,PRIMARY KEY (`id`)); ×××ert into `frame01` values(1,1,'192.168.0.22,u=pt22,p=ptchecksums,P=3307'); ×××ert into `frame01` values(2,2,'192.168.0.33,u=pt33,p=ptchecksums,P=3308');
slave查看數據已經同步過來了
mysql> select * from frame01; +----+-----------+------------------------------------------+ | id | parent_id | dsn | +----+-----------+------------------------------------------+ | 1 | 1 | 192.168.0.22,u=pt22,p=ptchecksums,P=3307 | | 2 | 2 | 192.168.0.33,u=pt33,p=ptchecksums,P=3308 | +----+-----------+------------------------------------------+ 2 rows in set (0.00 sec)
slave庫修改frame01表數,模擬數據不一致
update frame01 set dsn='192.168.0.55,u=umaaa,p=ptchsyeudew,P=3310' where id=1; update frame01 set dsn='192.168.0.66,u=umbbb,p=ptchsyeudew,P=3311' where id=2; mysql> select * from frame01; +----+-----------+-------------------------------------------+ | id | parent_id | dsn | +----+-----------+-------------------------------------------+ | 1 | 1 | 192.168.0.55,u=umaaa,p=ptchsyeudew,P=3310 | | 2 | 2 | 192.168.0.66,u=umbbb,p=ptchsyeudew,P=3311 | +----+-----------+-------------------------------------------+
在新版本的pt-table-ckecksum不用本身添加checksums表,在pt-table-checksum 表時,會在master庫上默認生成percona庫和表checksums
[root@kusou-es11 ~]# pt-table-checksum h=192.168.0.39,u=ptsum,p='ptchecksums',P=3306 --tables=test01.frame01 --no-check-binlog-format --nocheck-replication-filters --recursion-method="processlist" Checking if all tables can be checksummed ... Starting checksum ... TS ERRORS DIFFS ROWS DIFF_ROWS CHUNKS SKIPPED TIME TABLE 06-15T10:37:41 0 1 2 0 1 0 0.314 test01.frame01
默認會在master庫上生成庫percona和表checksums
mysql> select * from percona.checksums; +--------+---------+-------+------------+-------------+----------------+----------------+----------+----------+------------+------------+---------------------+ | db | tbl | chunk | chunk_time | chunk_index | lower_boundary | upper_boundary | this_crc | this_cnt | master_crc | master_cnt | ts | +--------+---------+-------+------------+-------------+----------------+----------------+----------+----------+------------+------------+---------------------+ | test01 | frame01 | 1 | 0.001818 | NULL | NULL | NULL | 2eedbb7c | 2 | 2eedbb7c | 2 | 2019-06-15 11:08:00 | +--------+---------+-------+------------+-------------+----------------+----------------+----------+----------+------------+------------+---------------------+ 1 row in set (0.00 sec)
[root@kusou-es11 ~]# pt-table-checksum h=192.168.0.39,u=ptsum,p='ptchecksums',P=3306 --databases=test01 --nocheck-replication-filters --replicate=test01.checksums --no-check-binlog-format --recursion-method="processlist" Checking if all tables can be checksummed ... Starting checksum ... TS ERRORS DIFFS ROWS DIFF_ROWS CHUNKS SKIPPED TIME TABLE 06-15T10:40:18 0 1 2 0 1 0 0.317 test01.frame01
--nocheck-replication-filters 表示不檢查複製過濾器; --replicate=test01.checksums 把檢查結果寫到mysql中test01庫中的checksums表; --databases=test01 檢查的庫爲test01, 此處能夠寫多個庫,採用逗號進行分割; --no-check-binlog-format 不檢查binlog日誌格式, pt-table-checksum在運行的時候是基於statement複製格式的,若是binlog日誌格式爲row的話會報錯,經過上面--no-check-binlog-format來去除binlog日誌檢查,以防報錯; --recursion-method="proccesslist" 這時工具會從show processlist中在主庫中自動尋找從庫,固然不加此參數,默認也是採用的這種方式,還有另一種方式那就是dns,這個後面會介紹演示。
會把checksums表寫入到master 192.168.0.39 的test01庫中。登陸主庫查看
mysql> select * from test01.checksums; +--------+---------+-------+------------+-------------+----------------+----------------+----------+----------+------------+------------+---------------------+ | db | tbl | chunk | chunk_time | chunk_index | lower_boundary | upper_boundary | this_crc | this_cnt | master_crc | master_cnt | ts | +--------+---------+-------+------------+-------------+----------------+----------------+----------+----------+------------+------------+---------------------+ | test01 | frame01 | 1 | 0.001907 | NULL | NULL | NULL | 2eedbb7c | 2 | 2eedbb7c | 2 | 2019-06-15 11:28:02 | +--------+---------+-------+------------+-------------+----------------+----------------+----------+----------+------------+------------+---------------------+ 1 row in set (0.00 sec)
TS :完成檢查的時間。
ERRORS :檢查時候發生錯誤和警告的數量。
DIFFS :0表示一致,1表示不一致。當指定--no-replicate-check時,會一直爲0,當指定--replicate-check-only會顯示不一樣的信息。
驗證演示以下:
[root@kusou-es11 ~]# pt-table-checksum h=192.168.0.39,u=ptsum,p='ptchecksums',P=3306 --databases=test01 --no-check-binlog-format --nocheck-replication-filters --no-replicate-check --recursion-method="processlist" Checking if all tables can be checksummed ... Starting checksum ... TS ERRORS DIFFS ROWS DIFF_ROWS CHUNKS SKIPPED TIME TABLE 06-15T11:08:00 0 0 2 0 1 0 0.012 test01.frame01
本來master 上 test01庫和slave上test01庫上的數據是不一致的,可是 pt-table-checksum 檢測時,加上參數-no-replicate-check後,DIFFS 會始終是0
當指定--replicate-check-only會顯示不一樣的信息:
[root@kusou-es11 ~]# pt-table-checksum h=192.168.0.39,u=ptsum,p='ptchecksums',P=3306 --databases=test01 --no-check-binlog-format --nocheck-replication-filters --replicate-check-only --recursion-method="processlist" Checking if all tables can be checksummed ... Starting checksum ... Differences on kusou-es11 TABLE CHUNK CNT_DIFF CRC_DIFF CHUNK_INDEX LOWER_BOUNDARY UPPER_BOUNDARY test01.frame01 1 0 1
ROWS :表的行數。
CHUNKS :被劃分到表中的塊的數目。
SKIPPED :因爲錯誤或警告或過大,則跳過塊的數目。
TIME :執行的時間。
TABLE :被檢查的表名
pt-table-checksum +dsn方式指令參數說明以下:
--nocheck-replication-filters :不檢查複製過濾器,建議啓用。後面能夠用--databases來指定須要檢查的數據庫。 --no-check-binlog-format : 不檢查複製的binlog模式,要是binlog模式是ROW,則會報錯。 --replicate-check-only :只顯示不一樣步的信息。 --replicate= :把checksums的信息寫入到指定表中,建議直接寫到被檢查的數據庫當中。 --databases= :指定須要被檢查的數據庫,多個則用逗號隔開。 --tables= :指定須要被檢查的表,多個用逗號隔開 h=192.168.0.39 :Master的內網地址 u=ptsum :用戶名 p=ptchecksums :密碼 P=3306 :端口 dsn=D :表示用dsn方式進行檢查,D爲database t=dsns :表示用這張表
配置dsn方式校驗數據:
登錄192.168.0.39 的·master庫建立連接slave庫的路由表
mysql> use percona; mysql>CREATE TABLE `dsns` ( `id` int(11) NOT NULL AUTO_INCREMENT,`parent_id` int(11) DEFAULT NULL,`dsn` varchar(255) NOT NULL,PRIMARY KEY (`id`)); mysql>######×××ert into dsns values(1,1,'h=slave的IP,u=username,p=password,P=3306'); mysql>×××ert into dsns values(1,1,'h=192.168.0.11,u=ptsum,p=ptchecksums,P=3306');
[root@kusou-es11 ~]# pt-table-checksum --nocheck-replication-filters --replicate=percona.checksums --databases=test01 --tables=frame01 --no-check-binlog-format h=192.168.0.39,u=ptsum,p=ptchecksums,P=3306 --recursion-method dsn=D=percona,t=dsns,h=192.168.0.11,u=ptsum,p=ptchecksums,P=3306 Checking if all tables can be checksummed ... Starting checksum ... TS ERRORS DIFFS ROWS DIFF_ROWS CHUNKS SKIPPED TIME TABLE 06-15T11:45:39 0 1 2 0 1 0 0.317 test01.frame01
DIFFS =1 也是能夠看到;master 和slave的test01庫中frame01表的數據是不一致的
指令參數簡單說明:
第一段的ip,帳戶,密碼和端口,指的是鏈接主庫的帳戶信息。
h=192.168.0.39,u=ptsum,p=ptchecksums,P=3306
--recursion-method dsn= 採用dsn的連接方式進行校驗數據
D=percona,t=dsns, 從 percona庫中讀取 dsns表中的連接從庫的帳戶信息
第二段的ip,帳戶,密碼和端口,指的是鏈接從庫的帳戶信息。
h=192.168.0.11,u=ptsum,p=ptchecksums,P=3306
提示:此條pt-table-checksum指令是從 從庫192.168.0.11上的percona.dsns表讀取連接從庫的帳戶信息,和主庫192.168.0.39中的test01.frame01表的數據進行比對的。
若是此時登陸192.168.0.11從庫,刪掉slave庫percona.dsns表中的數據,而後再進行pt-table-checksum比對的話,會提示鏈接不上從庫,致使檢查失敗
mysql> select * from dsns; +----+-----------+---------------------------------------------+ | id | parent_id | dsn | +----+-----------+---------------------------------------------+ | 1 | 1 | h=192.168.0.11,u=ptsum,p=ptchecksums,P=3306 | +----+-----------+---------------------------------------------+ 1 row in set (0.00 sec) mysql> delete from dsns where id=1; Query OK, 1 row affected (0.00 sec) mysql> select * from dsns; [root@kusou-es11 ~]# pt-table-checksum --nocheck-replication-filters --replicate=percona.checksums --databases=test01 --tables=frame01 --no-check-binlog-format h=192.168.0.39,u=ptsum,p=ptchecksums,P=3306 --recursion-method dsn=D=percona,t=dsns,h=192.168.0.11,u=ptsum,p=ptchecksums,P=3306 Checking if all tables can be checksummed ... Starting checksum ... Diffs cannot be detected because no slaves were found. Please read the --recursion-method documentation for information. TS ERRORS DIFFS ROWS DIFF_ROWS CHUNKS SKIPPED TIME TABLE 06-15T12:23:40 0 0 2 0 1 0 0.009 test01.frame01
報錯提示:Diffs cannot be detected because no slaves were found. Please read the --recursion-method documentation for information.
DIFFS =0 ,然而實際上,master和slave庫的test01.frame01表的數據是不一致的,檢測比對失敗。
[root@kusou-es11 ~]# pt-table-checksum --nocheck-replication-filters --replicate=percona.checksums --databases=test01 --tables=frame01 --no-check-binlog-format h=192.168.0.39,u=ptsum,p=ptchecksums,P=3306 --recursion-method dsn=D=percona,t=dsns Checking if all tables can be checksummed ... Starting checksum ... TS ERRORS DIFFS ROWS DIFF_ROWS CHUNKS SKIPPED TIME TABLE 06-15T12:53:54 0 1 2 0 1 0 0.316 test01.frame01
DIFFS =1 說明master和slave數據存在不一致,檢測成功
執行如上的命令,是能夠檢測成功的,緣由是:上述的命令是直接從主庫192.168.0.39的percona.dsns表中讀取的連接slave庫的帳戶信息的
slave庫上操做下面的指令:
報錯1: [root@localhost local]# /usr/local/percona-toolkit/bin/pt-table-checksum h=192.168.0.39,u=ptsum,p='ptchecksums',P=3306 --databases=mysql --no-check-binlog-format --nocheck-replication-filters Can't locate Digest/MD5.pm in @INC (@INC conta×××: /usr/local/lib64/perl5 /usr/local/share/perl5 /usr/lib64/perl5/vendor_perl /usr/share/perl5/vendor_perl /usr/lib64/perl5 /usr/share/perl5 .) at /usr/local/percona-toolkit/bin/pt-table-checksum line 789. BEGIN failed--compilation aborted at /usr/local/percona-toolkit/bin/pt-table-checksum line 789.
解決辦法:
緣由是:缺乏perl-Digest-MD5包, 安裝perl-Digest-MD5便可解決。 [root@localhost ~]# yum -y ×××tall perl-Digest-MD5
pt-table-checksum的數據校驗工具到此簡單介紹完成。有疑問的,歡迎留言一塊兒交流學習。