沒有備份怎麼恢復被drop的表(利用undrop-for-innodb)
介紹:
也許你們都難以理解,這麼重要的數據爲啥不備份(或者備份不可用)?並且還任性的drop table了。顯然有備份是最好的，可是它們並不老是可用的。這種狀況使人恐懼，但並不是毫無但願。在許多狀況下，能夠恢復數據庫或表中的幾乎全部數據。恢復計劃取決於InnoDB是否將全部數據保存在一個ibdata1中，仍是每一個表都有本身的表空間。本文將考慮innodb_file_per_table=OFF的狀況。html
drop 表恢復其餘方法:
本文重要部分:
環境:
- 時間:#Sat Aug 4 19:37:24 CST 2018
- CentOS Linux release 7.2.1511 (Core)
- MySQL:5.7.23-log
- 使用工具:undrop-for-innodb
1.工具安裝
- 依賴包安裝
yum install -y make gcc flex bison
- 下載工具包(github)
#cd /opt/ [root@db13_19:55:25 /opt] #git clone https://github.com/twindb/undrop-for-innodb.git Cloning into 'undrop-for-innodb'... Resolving deltas: 100% (77/77), done.
- make
#cd /opt/undrop-for-innodb #make .... .... [root@db13_20:39:43 /opt/undrop-for-innodb] #ll total 2920 -rw-r--r-- 1 root root 6271 Aug 4 19:55 check_data.c -rw-r--r-- 1 root root 66128 Aug 4 20:39 check_data.o -rwxr-xr-x 1 root root 727801 Aug 4 20:39 c_parser -rw-r--r-- 1 root root 28587 Aug 4 19:55 c_parser.c -rw-r--r-- 1 root root 1030296 Aug 4 20:39 c_parser.o drwxr-xr-x 2 root root 92 Aug 4 19:55 dictionary -rw-r--r-- 1 root root 1978 Aug 4 19:55 fetch_data.sh drwxr-xr-x 2 root root 4096 Aug 4 19:55 include -rw-r--r-- 1 root root 8936 Aug 4 19:55 innochecksum.c -rwxr-xr-x 1 root root 36343 Aug 4 20:39 innochecksum_changer -rw-r--r-- 1 root root 154459 Aug 4 20:39 lex.yy.c -rw-r--r-- 1 root root 18047 Aug 4 19:55 LICENSE -rw-r--r-- 1 root root 1942 Aug 4 19:55 Makefile -rw-r--r-- 1 root root 16585 Aug 4 19:55 print_data.c -rw-r--r-- 1 root root 127176 Aug 4 20:39 print_data.o -rw-r--r-- 1 root root 3464 Aug 4 19:55 README.md -rwxr-xr-x 1 root root 1536 Aug 4 19:55 recover_dictionary.sh drwxr-xr-x 2 root root 4096 Aug 4 19:55 sakila -rw-r--r-- 1 root root 103506 Aug 4 20:39 sql_parser.c -rw-r--r-- 1 root root 8462 Aug 4 19:55 sql_parser.l -rw-r--r-- 1 root root 296840 Aug 4 20:39 sql_parser.o -rw-r--r-- 1 root root 26355 Aug 4 19:55 sql_parser.y -rwxr-xr-x 1 root root 61725 Aug 4 20:39 stream_parser -rw-r--r-- 1 root root 23103 Aug 4 19:55 stream_parser.c -rw-r--r-- 1 root root 109304 Aug 4 20:39 stream_parser.o -rw-r--r-- 1 root root 14764 Aug 4 19:55 sys_parser.c -rw-r--r-- 1 root root 2182 Aug 4 19:55 tables_dict.c -rw-r--r-- 1 root root 40264 Aug 4 20:39 tables_dict.o -rwxr-xr-x 1 root root 6629 Aug 4 19:55 test.sh drwxr-xr-x 3 root root 42 Aug 4 19:55 vagrant [root@db13_20:39:57 /opt/undrop-for-innodb]
- 增長用於恢復表結構的工具sys_parser(官方文檔未使用):
#gcc `/usr/local/mysql57/bin/mysql_config --cflags` `/usr/local/mysql57/bin/mysql_config --libs` -o sys_parser sys_parser.c 注:mysql_basedir: /usr/local/mysql57/bin/
2.表數據生成和drop表
- 初始化一個新實例:
[root@db212_20:58:44 /data/57mysql]
#mkdir mysql3507/{data,logs,tmp} -p
#chown -R mysql:mysql mysql3507
[root@db212_21:07:34 /3507]
\\複製my3507.cnf到mysql3507下
#ln -s /data/57mysql/mysql3507/ /3507
#/usr/local/mysql57/bin/mysqld --defaults-file=/3507/my3507.cnf --initialize-insecure
#/usr/local/mysql57/bin/mysqld --defaults-file=/3507/my3507.cnf&
[1] 11669
//取初始密碼並登陸:
/usr/local/mysql57/bin/mysql -S /tmp/mysql3507.sock -uroot
(unknown)@localhost [(none)]>alter user user() identified by '*****';
root@localhost [(none)]>CREATE DATABASE wenyz;
root@localhost [(none)]>use wenyz;
Database changed
//建立表
root@localhost [wenyz]>CREATE TABLE `t2` (
-> `id` int(10) unsigned NOT NULL AUTO_INCREMENT,
-> `ti` varchar(100) NOT NULL,
-> `date` date DEFAULT NULL,
-> PRIMARY KEY (`id`)
-> ) ENGINE=InnoDB AUTO_INCREMENT=4079859 DEFAULT CHARSET=utf8;
Query OK, 0 rows affected (0.01 sec)
//造數據
root@localhost [wenyz]>insert into t2(ti,date) values(substring(MD5(RAND()),floor(RAND()*26)+1,15),now()) ;
Records: 448 Duplicates: 0 Warnings: 0
//查看數據行數和checksum值
root@localhost [wenyz]>select count(*) from t2;
+----------+
| count(*) |
+----------+
| 896 |
+----------+
root@localhost [wenyz]>checksum table t2;
+----------+------------+
| Table | Checksum |
+----------+------------+
| wenyz.t2 | 3458542072 |
+----------+------------+
//DROP 表
root@localhost [wenyz]>drop table t2;
Query OK, 0 rows affected (0.01 sec)
3.利用stream_parser將ibdata1文件導出成page文件
#cd /opt/undrop-for-innodb/ [root@db212_21:25:52 /opt/undrop-for-innodb] #./stream_parser -f /3507/data/ibdata1 Opening file: /3507/data/ibdata1 File information: .... Size to process: 104857600 (100.000 MiB) Size to process: 104857600 (100.000 MiB) time of last access: 1533388916 Sat Aug 4 21:21:56 2018 time of last modification: 1533388917 Sat Aug 4 21:21:57 2018 time of last status change: 1533388917 Sat Aug 4 21:21:57 2018 total size, in bytes: 104857600 (100.000 MiB) Size to process: 104857600 (100.000 MiB) All workers finished in 0 sec
4.恢復表結構 [ Top ]
這裏引入官方的一段描述:mysql
InnoDB stores all data in B+tree indexes. A table has one clustered index PRIMARY, all fields are stored there. Thus, if the table has secondary keys, each key has an index. Each index is identified by index_id.linux
Consequently, if we want to recover a table, we have to find all pages that belong to a particular index_id.git
接下來咱們先來看看怎麼找到table id和INDEX_ID(page文件編號)的，理解原理後再用程序跑一次，導入到臨時數據庫中github
- 手工查找table id
觀察如下結果中,wenyz/t2後的40,即爲table id
[root@db212_21:25:55 /opt/undrop-for-innodb] #./c_parser -4Df pages-ibdata1/FIL_PAGE_INDEX/0000000000000001.page -t dictionary/SYS_TABLES.sql | grep 'wenyz/t2' 000000000521 3B00000149047E SYS_TABLES "wenyz/t2" 40 3 33 0 64 "" 0 SET FOREIGN_KEY_CHECKS=0; LOAD DATA LOCAL INFILE '000000000521 3B00000149047E SYS_TABLES "wenyz/t2" 40 3 33 0 64 "" 0
- 經過table id查看page文件編號
觀察如下結果中40後的41,即爲INDEX_ID(page文件編號)
[root@db212_21:29:54 /opt/undrop-for-innodb] #./c_parser -4Df pages-ibdata1/FIL_PAGE_INDEX/0000000000000003.page -t dictionary/SYS_INDEXES.sql | grep '40' 000000000521 3B0000014903A2 SYS_INDEXES 40 41 "PRIMARY" 1 3 0 4294967295 SET FOREIGN_KEY_CHECKS=0; 000000000521 3B0000014903A2 SYS_INDEXES 40 41 "PRIMARY" 1 3 0 4294967295
- 用程序恢復字典信息:編輯mysql登陸信息.
[root@db212_21:57:04 /opt/undrop-for-innodb] //將文件中三處mysql替換成/usr/local/mysql57/bin/mysql --login-path=p3507 #vi recover_dictionary.sh 43 /usr/local/mysql57/bin/mysql --login-path=p3507 -e "CREATE DATABASE IF NOT EXISTS test" ... 50 /usr/local/mysql57/bin/mysql --login-path=p3507 test < dictionary/$t.sql ... 58 /usr/local/mysql57/bin/mysql --login-path=p3507 test < dumps/default/$t.sql
- 執行/recover_dictionary.sh,恢復字典信息
[root@db212_22:13:57 /opt/undrop-for-innodb] #./recover_dictionary.sh Generating dictionary tables dumps... OK Creating test database ... OK Creating dictionary tables in database test: SYS_TABLES ... OK SYS_COLUMNS ... OK SYS_INDEXES ... OK SYS_FIELDS ... OK All OK Loading dictionary tables data: SYS_TABLES ... 52 recs OK SYS_COLUMNS ... 284 recs OK SYS_INDEXES ... 68 recs OK SYS_FIELDS ... 90 recs OK All OK [root@db212_22:14:02 /opt/undrop-for-innodb]
- 登陸mysql查看信息字典信息:
mysql --login-path=p3507 root@localhost [(none)]>use test; Database changed root@localhost [test]>select * from SYS_TABLES where name like 'wenyz/t2%'; +----------+----+--------+------+--------+---------+--------------+-------+ | NAME | ID | N_COLS | TYPE | MIX_ID | MIX_LEN | CLUSTER_NAME | SPACE | +----------+----+--------+------+--------+---------+--------------+-------+ | wenyz/t2 | 40 | 3 | 33 | 0 | 64 | | 0 | +----------+----+--------+------+--------+---------+--------------+-------+ 1 row in set (0.00 sec) root@localhost [test]>select * from SYS_INDEXES where table_id=40; +----------+----+---------+----------+------+-------+------------+ | TABLE_ID | ID | NAME | N_FIELDS | TYPE | SPACE | PAGE_NO | +----------+----+---------+----------+------+-------+------------+ | 40 | 41 | PRIMARY | 1 | 3 | 0 | 4294967295 | +----------+----+---------+----------+------+-------+------------+ //注意記錄上表中id,此ID爲INDEX_ID(page文件編號)等會表數據恢復要使用 1 row in set (0.00 sec) root@localhost [test]>
- 表恢復t2表結構 [ Top ]
./sys_parser -h 127.0.0.1 -u root -p xxxx -d test wenyz/t2
./sys_parser: error while loading shared libraries: libmysqlclient.so.20: cannot open shared object file: No such file or directory
#ln -s /opt/mysql-5.7.23-linux-glibc2.12-x86_64/lib/libmysqlclient.so.20 /usr/lib64/libmysqlclient.so.20
[root@db212_22:23:25 /opt/undrop-for-innodb]
#./sys_parser -h 127.0.0.1 -u root -p zstzst -d test wenyz/t2
CREATE TABLE `t2`(
`id` INT UNSIGNED NOT NULL,
`ti` VARCHAR(100) CHARACTER SET 'utf8' COLLATE 'utf8_general_ci' NOT NULL,
`date` DATE,
PRIMARY KEY (`id`)
) ENGINE=InnoDB;
[root@db212_22:23:30 /opt/undrop-for-innodb]
- 將恢復的表結構存到/tmp/t2.sql
#cat /tmp/t2.sql
CREATE TABLE `t2`(
`id` INT UNSIGNED NOT NULL,
`ti` VARCHAR(100) CHARACTER SET 'utf8' COLLATE 'utf8_general_ci' NOT NULL,
`date` DATE,
PRIMARY KEY (`id`)
) ENGINE=InnoDB;
[root@db212_22:26:20 /opt/undrop-for-innodb]
5.表數據恢復 [ Top ]
- 查看數據是否存在
如下命令中使用的0000000000000041.page爲上面提到的INDEX_ID(page文件編號),/tmp/t2.sql爲上面恢復的表結構.sql
[root@db212_22:33:44 /opt/undrop-for-innodb] #./c_parser -6f pages-ibdata1/FIL_PAGE_INDEX/0000000000000041.page -t /tmp/t2.sql |head -2 -- Page id: 459, Format: COMPACT, Records list: Valid, Expected records: (180 180) 000000000507 A7000001210110 t2 4079859 "d553635af1a3b" "2018-08-04" 000000000508 A8000001230110 t2 4079860 "44d64b99fc30d1b" "2018-08-04" [root@db212_22:33:44 /opt/undrop-for-innodb]
- 利用c_parser將0000000000000041.page導出成可執行sql
//注意:此處幾個文件名程序把導出的兩個數據文件的文件名關係是寫死的,如下dumps/default/t2中的t2是須要和表名一致,在t2_load.sql中會引用此文件路經.
./c_parser -6f pages-ibdata1/FIL_PAGE_INDEX/0000000000000041.page -t /tmp/t2.sql > dumps/default/t2 2> dumps/default/t2_load.sql
#cd dumps/default/
[root@db212_22:41:01 /opt/undrop-for-innodb/dumps/default]
#ll
total 132
-rw-r--r-- 1 root root 21232 Aug 4 22:14 SYS_COLUMNS
...
-rw-r--r-- 1 root root 62923 Aug 4 22:40 t2
-rw-r--r-- 1 root root 308 Aug 4 22:40 t2_load.sql
[root@db212_22:41:04 /opt/undrop-for-innodb/dumps/default]
#cat t2_load.sql
SET FOREIGN_KEY_CHECKS=0;
LOAD DATA LOCAL INFILE '/opt/undrop-for-innodb/dumps/default/t2' REPLACE INTO TABLE `t2` FIELDS TERMINATED BY '\t' OPTIONALLY ENCLOSED BY '"' LINES STARTING BY 't2\t' (`id`, `ti`, `date`);
-- STATUS {"records_expected": 896, "records_dumped": 896, "records_lost": false} STATUS END
- 將表結構t2.sql和表數據t2和t2_load.sql導入數據庫
root@localhost [test]>source /tmp/t2.sql Query OK, 0 rows affected (0.02 sec) root@localhost [test]>source /opt/undrop-for-innodb/dumps/default/t2_load.sql Query OK, 0 rows affected (0.00 sec) Query OK, 896 rows affected (0.01 sec) Records: 896 Deleted: 0 Skipped: 0 Warnings: 0 root@localhost [test]>select count(*) from t2; +----------+ | count(*) | +----------+ | 896 | +----------+ 1 row in set (0.00 sec) root@localhost [test]>checksum table t2; +---------+------------+ | Table | Checksum | +---------+------------+ | test.t2 | 3458542072 | +---------+------------+ 1 row in set (0.00 sec) root@localhost [test]> //drop前數據信息對比: root@localhost [wenyz]>select count(*) from t2; +----------+ | count(*) | +----------+ | 896 | +----------+ 1 row in set (0.00 sec) root@localhost [wenyz]>checksum table t2; +----------+------------+ | Table | Checksum | +----------+------------+ | wenyz.t2 | 3458542072 | +----------+------------+ 1 row in set (0.00 sec)
- 至此數據已經徹底恢復.
6.未解決的問題 [ Top ]
問題1.
innodb(非獨立表空間)狀況,drop表後,用工具讀ibdata1對應數據頁文件,若是是這個表大於350行左右的數據,頁文件是存在的,但小於350行左右,頁文件就不存在(drop前頁文件是存在的).這是個什麼緣由呢
腳本輸出信息:數據庫
如下爲精簡信息,完整輸出信息請點擊下載文本(因爲不能上傳txt,因此在文件名後面加了sh,下載後請刪除.sh):
320行,沒有drop記錄:
320行,drop
640行,dropbash
select count(*) from t2 640 Table Checksum wenyz.t2 1273189789 ... -rw-r--r-- 1 root root 16384 Aug 4 23:09 0000000000000040.page -rw-r--r-- 1 root root 98304 Aug 4 23:09 0000000000000041.page /被drop表空間文件 -rw-r--r-- 1 root root 16384 Aug 4 23:09 18446744069414584320.page count(*)320(行) Table Checksum wenyz.t2 3018070873 .... -rw-r--r-- 1 root root 16384 Aug 4 23:12 0000000000000038.page -rw-r--r-- 1 root root 32768 Aug 4 23:12 0000000000000039.page -rw-r--r-- 1 root root 16384 Aug 4 23:12 0000000000000040.page \\這裏缺41.page -rw-r--r-- 1 root root 16384 Aug 4 23:12 18446744069414584320.page
把drop命令在腳本里註釋後(仍是320行)ide
如下爲精簡信息,完整輸出信息附後: #/tmp/init3507.sh BrI?Zu>o=1uN mysql: [Warning] Using a password on the command line interface can be insecure. root@localhost [(none)]>alter user user() identified by 'xxxxx'; Query OK, 0 rows affected (0.00 sec) root@localhost [(none)]>exit Bye mysql: [Warning] Using a password on the command line interface can be insecure. count(*) ---------------------------------------------------------------------------------320行數據 320 Table Checksum wenyz.t2 2368041617 ... -rw-r--r-- 1 root root 32768 Aug 4 23:16 0000000000000039.page -rw-r--r-- 1 root root 16384 Aug 4 23:16 0000000000000040.page -rw-r--r-- 1 root root 32768 Aug 4 23:16 0000000000000041.page //320行時不drop是有此表空間文件的 -rw-r--r-- 1 root root 16384 Aug 4 23:16 18446744069414584320.page [root@db211_23:16:21 /opt/undrop-for-innodb] #
- 爲了快速測試數據恢復,使用的腳本
init3507.sh
/tmp/init3507.sh
pkill mysqld rm -rf /3507/data/* /3507/logs/* /usr/local/mysql57/bin/mysqld --defaults-file=/3507/my3507.cnf --initialize-insecure /usr/local/mysql57/bin/mysqld --defaults-file=/3507/my3507.cnf& sleep 2 /usr/local/mysql57/bin/mysql -S /tmp/mysql3507.sock -uroot </tmp/create_t2.sql rm -rf /opt/undrop-for-innodb/pages-ibdata1 #ls -l cd /opt/undrop-for-innodb/ sleep 5 rm -rf /opt/undrop-for-innodb/pages-ibdata1 cd /opt/undrop-for-innodb/ /opt/undrop-for-innodb/stream_parser -f /3507/data/ibdata1 ls -l /opt/undrop-for-innodb/pages-ibdata1/FIL_PAGE_INDEX
/tmp/create_t2.sql (因爲不能上傳sql文件,因此只能改成sh)工具
alter user user() identified by 'xxxx'; CREATE DATABASE wenyz; use wenyz; CREATE TABLE `t2` ( `id` int(10) unsigned NOT NULL AUTO_INCREMENT, `ti` varchar(100) NOT NULL, `date` date DEFAULT NULL, PRIMARY KEY (`id`) ) ENGINE=InnoDB AUTO_INCREMENT=4079859 DEFAULT CHARSET=utf8; insert into t2(ti,date) values(substring(MD5(RAND()),floor(RAND()*26)+1,15),now()) ; insert into t2(ti,date) values(substring(MD5(RAND()),floor(RAND()*26)+1,15),now()) ; insert into t2(ti,date) values(substring(MD5(RAND()),floor(RAND()*26)+1,15),now()) ; insert into t2(ti,date) values(substring(MD5(RAND()),floor(RAND()*26)+1,15),now()) ; insert into t2(ti,date) values(substring(MD5(RAND()),floor(RAND()*26)+1,15),now()) ; insert into t2(ti,date) values(substring(MD5(RAND()),floor(RAND()*26)+1,15),now()) ; insert into t2(ti,date) values(substring(MD5(RAND()),floor(RAND()*26)+1,15),now()) ; insert into t2(ti,date) values(substring(MD5(RAND()),floor(RAND()*26)+1,15),now()) ; insert into t2(ti,date) values(substring(MD5(RAND()),floor(RAND()*26)+1,15),now()) ; insert into t2(ti,date) values(substring(MD5(RAND()),floor(RAND()*26)+1,15),now()) ; insert into t2(ti,date) select ti,date from t2; insert into t2(ti,date) select ti,date from t2; insert into t2(ti,date) select ti,date from t2; insert into t2(ti,date) select ti,date from t2; insert into t2(ti,date) select ti,date from t2; select count(*) from t2; checksum table t2; drop table t2;
7.參考過的資料 [ Top ]
相關文章
- 1. iphone沒有備份的數據怎麼恢復
- 2. icloud備份qq數據怎麼恢復
- 3. tomcat恢復備份之後，功能沒有恢復的緣由
- 4. 經過全備+主從同步恢復被drop的庫或表
- 5. 經過全備+binlog_server同步恢復被drop的庫或表
- 6. 經過全備+relaylog同步恢復被drop的庫或表
- 7. 利用增量備份恢復standby
- 8. 利用binlog實現備份-恢復
- 9. 利用mydumper和myloader備份恢復mysql
- 10. MySQL異常恢復-drop表恢復
- 更多相關文章...
- • MySQL數據庫恢復（LOAD DATA） - MySQL教程
- • MySQL數據庫備份（INTO OUTFILE） - MySQL教程
- • IDEA下SpringBoot工程配置文件沒有提示
- • C# 中 foreach 遍歷的用法
相關標籤/搜索
每日一句
-
每一个你不满意的现在，都有一个你没有努力的曾经。
最新文章
- 1. 在windows下的虛擬機中,安裝華爲電腦的deepin操作系統
- 2. 強烈推薦款下載不限速解析神器
- 3. 【區塊鏈技術】孫宇晨：區塊鏈技術帶來金融服務的信任變革
- 4. 搜索引起的鏈接分析-計算網頁的重要性
- 5. TiDB x 微衆銀行 | 耗時降低 58%，分佈式架構助力實現普惠金融
- 6. 《數字孿生體技術白皮書》重磅發佈（附完整版下載）
- 7. 雙十一「避坑」指南：區塊鏈電子合同爲電商交易保駕護航！
- 8. 區塊鏈產業，怎樣「鏈」住未來？
- 9. OpenglRipper使用教程
- 10. springcloud請求一次好用一次不好用zuul Name or service not known
歡迎關注本站公眾號,獲取更多信息
