MySQL物理備份

Ⅰ、xtrabackup介紹

• xtrabackup只能備份innodb引擎的數據,不能備份表結構,percona開源的,強烈推薦最新版本(舊版本bug多)
• innobackupex能夠備份myisam和innodb兩種引擎的數據和表結構,通常用這個
• 備份時,默認讀取MySQL配置文件(datadir)

Ⅱ、xtrabackup安裝使用

2.1 安裝

[root@VM_0_5_centos src]# yum install perl-DBD-MySQL
不安裝這個備份會報錯：Failed to connect to MySQL server: DBI connect

[root@VM_0_5_centos src]# cd /usr/local/src
[root@VM_0_5_centos src]# wget https://www.percona.com/downloads/XtraBackup/Percona-XtraBackup-2.4.7/binary/tarball/percona-xtrabackup-2.4.7-Linux-x86_64.tar.gz
[root@VM_0_5_centos src]# tar zxvf percona-xtrabackup-2.4.7-Linux-x86_64.tar.gz -C ..
添加環境變量
[root@VM_0_5_centos src]# cd ..
[root@VM_0_5_centos src]# ln -s percona-xtrabackup-2.4.7-Linux-x86_64/ xtrabackup
[root@VM_0_5_centos src]# echo "PATH=/usr/local/xtrabackup/bin:$PATH" >> /etc/profile
[root@VM_0_5_centos src]# source /etc/profile

2.2 玩一手

[root@VM_0_5_centos src]# innobackupex --compress --compress-threads=8 --stream=xbstream -S /tmp/mysql.sock --parallel=4  /data/backup/ > /data/backup/backup.xbstream
建議用-S鏈接,默認走socket,不用-S可能報連不上

經常使用參數：throttle
指定備份時用到的iops是多少,限制速度

8個壓縮線程,4個備份線程

輸出內容（簡化）

190620 19:47:53 innobackupex: Starting the backup operation

IMPORTANT: Please check that the backup run completes successfully.
           At the end of a successful backup run innobackupex
           prints "completed OK!".

190620 19:47:53  version_check Connecting to MySQL server with DSN 'dbi:mysql:;mysql_read_default_group=xtrabackup;mysql_socket=/tmp/mysql.sock' as 'root'  (using password: YES).
190620 19:47:53  version_check Connected to MySQL server
190620 19:47:53  version_check Executing a version check against the server...
190620 19:47:53  version_check Done.
190620 19:47:53 Connecting to MySQL server host: localhost, user: root, password: set, port: not set, socket: /tmp/mysql.sock
Using server version 5.7.20-log
innobackupex version 2.4.7 based on MySQL server 5.7.13 Linux (x86_64) (revision id: 05f1fcf)
xtrabackup: uses posix_fadvise().

# 鏈接數據庫並作兩次版本檢查

xtrabackup: cd to /mdata/mysql_test_data
xtrabackup: open files limit requested 0, set to 100001
xtrabackup: using the following InnoDB configuration:
xtrabackup:   innodb_data_home_dir = .
xtrabackup:   innodb_data_file_path = ibdata1:12M:autoextend
xtrabackup:   innodb_log_group_home_dir = ./
xtrabackup:   innodb_log_files_in_group = 2
xtrabackup:   innodb_log_file_size = 50331648
InnoDB: Number of pools: 1
190620 19:47:53 >> log scanned up to (10304795)

# 讀取配置文件,尋找對應的文件及日誌位置

xtrabackup: Generating a list of tablespaces
InnoDB: Allocated tablespace ID 51 for dump_test/dump_inno, old maximum was 0
xtrabackup: Starting 4 threads for parallel data files transfer
190620 19:47:53 [04] Compressing and streaming ./ibdata1
190620 19:47:53 [03] Compressing and streaming ./dump_test/dump_inno.ibd
190620 19:47:53 [03]        ...done
190620 19:47:53 [03] Compressing and streaming ./test/test.ibd
190620 19:47:53 [02] Compressing and streaming ./test/sbtest1.ibd
190620 19:47:53 [03]        ...done

...

190620 19:47:54 >> log scanned up to (10304795)
190620 19:47:54 Executing FLUSH NO_WRITE_TO_BINLOG TABLES...
190620 19:47:54 Executing FLUSH TABLES WITH READ LOCK...
190620 19:47:54 Starting to backup non-InnoDB tables and files
190620 19:47:54 [01] Compressing and streaming ./dump_test/dump_inno.frm to <STDOUT>
190620 19:47:54 [01]        ...done
190620 19:47:54 [01] Compressing and streaming ./dump_test/db.opt to <STDOUT>
190620 19:47:54 [01]        ...done

...

190620 19:47:55 Finished backing up non-InnoDB tables and files

# 拷貝數據

190620 19:47:55 [00] Compressing and streaming xtrabackup_binlog_info
190620 19:47:55 [00]        ...done

# 獲取二進制文件日誌點

190620 19:47:55 Executing FLUSH NO_WRITE_TO_BINLOG ENGINE LOGS...
xtrabackup: The latest check point (for incremental): '10304786'
190620 19:47:55 >> log scanned up to (10304795)
xtrabackup: Stopping log copying thread.
190620 19:47:55 Executing UNLOCK TABLES
190620 19:47:55 All tables unlocked

# 中止拷貝,釋放鎖

190620 19:47:55 [00] Compressing and streaming ib_buffer_pool to <STDOUT>
190620 19:47:55 [00]        ...done
190620 19:47:55 Backup created in directory '/data/backup'
MySQL binlog position: filename 'bin.000006', position '154'
190620 19:47:55 [00] Compressing and streaming backup-my.cnf
190620 19:47:55 [00]        ...done
190620 19:47:55 [00] Compressing and streaming xtrabackup_info
190620 19:47:55 [00]        ...done
xtrabackup: Transaction log of lsn (10304786) to (10304795) was copied.
190620 19:47:55 completed OK!
190620 19:47:55 [00]        ...done
190620 19:47:55 Backup created in directory '/data/backup'
MySQL binlog position: filename 'bin.000006', position '154'
190620 19:47:55 [00] Compressing and streaming backup-my.cnf
190620 19:47:55 [00]        ...done
190620 19:47:55 [00] Compressing and streaming xtrabackup_info
190620 19:47:55 [00]        ...done
xtrabackup: Transaction log of lsn (10304786) to (10304795) was copied.
190620 19:47:55 completed OK!

# 生成各類文件,備份結束

Ⅲ、xtrabackup原理分析

3.1 xtrabackup全備步驟

-	操做	解析
step1	Connecting to MySQL server host	鏈接登陸
step2	using the following InnoDB configuration	讀相關配置文件
step3	start xtrabackup_log	啓用日誌文件,記錄redo的lsn,同時持續掃描redo log,將新產生的redo拷貝到xtrabackup_logfile
step4	copy innodb tables .ibd、.ibdata一、undo logs	拷貝innodb表的獨立表空間、共享表空間、undo日誌
step5	flush no_write_to_binlog tables、flush tables with read lock	強制將commit log刷入redo防止數據丟失(5.6以前沒有),鎖表
step6	copy non-innodb tables .MYD、.MYI、.opt、misc files和innodb tables .frm、.opt、misc files	拷貝myisam表相關內容和innodb表的表結構文件
step7	Get binary log position	獲取二進制日誌位置點,寫入到xtrabackup_binlog_info文件
step8	flush no_write_to_binlog engine logs	將redo刷盤
step9	stopping log copying thread	中止拷貝
step10	unlock tables	釋放鎖
step11	completed OK	生成各類文件,備份結束

tips：
①簡單點說：一個線程備份redo,貫穿整個過程始終,另外的線程備份表空間文件,直到completed OK,備份成功

②5.6以前的xtrabackup有丟數據的風險,強烈建議使用最新版本

③和mysqldump、mydumper相比,xtrabackup備份的是結束時間點的數據(二進制文件位置點不同),因此物理備份除了自己恢復塊以外,同步也快,由於不用拉數據,作一個一小時的備份,邏輯備份須要作一個小時的數據同步,物理備份不須要

④備份過程當中遇到myisam仍是會阻塞,數據一致性需求

Ⅳ、xtrabackup備份恢復

4.1 查看備份文件

因爲我這裏用的是流文件的方式備份的,因此要先打開流文件

[root@VM_0_5_centos backup]# xbstream -x < backup.xbstream
[root@VM_0_5_centos backup]# ll
total 2792
drwxr-x--- 2 root root    4096 July 20 19:47 abc
-rw-r----- 1 root root     417 July 20 19:47 backup-my.cnf.qp
-rw-r--r-- 1 root root 1822257 July 20 19:46 backup.xbstream
drwxr-x--- 2 root root    4096 July 20 19:47 dump_test
-rw-r----- 1 root root     370 July 20 19:47 ib_buffer_pool.qp
-rw-r----- 1 root root  969374 July 20 19:47 ibdata1.qp
drwxr-x--- 2 root root    4096 July 20 19:47 mysql
drwxr-x--- 2 root root    4096 July 20 19:47 performance_schema
drwxr-x--- 2 root root   12288 July 20 19:47 sys
drwxr-x--- 2 root root    4096 July 20 19:47 test
-rw-r----- 1 root root     102 July 20 19:47 xtrabackup_binlog_info.qp
-rw-r----- 1 root root     115 July 20 19:47 xtrabackup_checkpoints
-rw-r----- 1 root root     494 July 20 19:47 xtrabackup_info.qp
-rw-r----- 1 root root     391 July 20 19:47 xtrabackup_logfile.qp

看到不少qp文件,是由於備份時作了壓縮,咱們須要將其解壓
[root@VM_0_5_centos backup]# for f in `find ./ -iname "*\.qp"`; do qpress -dT4 $f  $(dirname $f) && rm -f $f; done
[root@VM_0_5_centos backup]# ll
drwxr-x--- 2 root root     4096 July 20 19:57 abc
-rw-r--r-- 1 root root      427 July 20 19:57 backup-my.cnf
-rw-r--r-- 1 root root  1822257 July 20 19:46 backup.xbstream
drwxr-x--- 2 root root     4096 July 20 19:57 dump_test
-rw-r--r-- 1 root root      413 July 20 19:57 ib_buffer_pool
-rw-r--r-- 1 root root 12582912 July 20 19:57 ibdata1
drwxr-x--- 2 root root     4096 July 20 19:57 mysql
drwxr-x--- 2 root root    12288 July 20 19:57 performance_schema
drwxr-x--- 2 root root    12288 July 20 19:57 sys
drwxr-x--- 2 root root     4096 July 20 19:57 test
-rw-r--r-- 1 root root       15 July 20 19:57 xtrabackup_binlog_info
-rw-r----- 1 root root      115 July 20 19:47 xtrabackup_checkpoints
-rw-r--r-- 1 root root      521 July 20 19:57 xtrabackup_info
-rw-r--r-- 1 root root     2560 July 20 19:57 xtrabackup_logfile

能夠看到,除了備份表空間等,還生成了4個文件

看下4個文件

[root@VM_0_5_centos backup]# cat xtrabackup_binlog_info           # 記錄binlog文件名和position
bin.000006    154
------
[root@VM_0_5_centos backup]# cat xtrabackup_checkpoints           # 記錄備份過程當中checkpoint、lsn信息
backup_type = full-backuped
from_lsn = 0
to_lsn = 10304786
last_lsn = 10304795
compact = 0
recover_binlog_info = 0
------
[root@VM_0_5_centos backup]# cat xtrabackup_info                  # 整個備份過程當中的信息
uuid = 48febc78-0012-11e8-b724-525400a4dac1
name =
tool_name = innobackupex
tool_command = --compress --compress-threads=8 --stream=xbstream -S /tmp/mysql.sock --parallel=4 ./
tool_version = 2.4.7
ibbackup_version = 2.4.7
server_version = 5.7.20-log
start_time = 2019-06-20 19:47:51
end_time = 2019-06-20 19:47:56
lock_time = 0
binlog_pos = filename 'bin.000006', position '154'
innodb_from_lsn = 0
innodb_to_lsn = 10304786
partial = N
incremental = N
format = xbstream
compact = N
compressed = compressed
encrypted = N
------
xtrabackup_logfile                                                # 持續備份的redo,直接看不了

4.2 恢復一手瞅瞅

step1: 應用日誌,將backup恢復
[root@VM_0_5_centos mdata]# innobackupex --apply-log backup

step2：將恢復好的數據拷貝到datadir,直接move也行
[root@VM_0_5_centos mdata]# innobackupex --copy-back backup

step3：修改文件屬主
[root@VM_0_5_centos mdata]# chown -R mysql:mysql mysql_test_data

step4：啓動數據庫
/etc/init.d/mysql.server start
Starting MySQL. SUCCESS!

tips：

• 日誌應用完成後,backup文件中會多出一個文件：xtrabackup_binlog_pos_innodb,記錄的是用於innodb的binlog的當前position,而xtrabackup_binlog_info記錄的是整個實例當前的binlog position
• 般狀況下,這兩個位置點是同樣的,但備份時兩種引擎都存在時,則有可能出現xtrabackup_binlog_info.pos > xtrabackup_binlog_pos_innodb.pos
• 因此咱們通常用xtrabackup_binlog_info中的binlog position

Ⅴ、其餘相關問題

5.1 增量備份

--incremental-history-name=name 可以使用改參數作增量備份

但很是不建議用這個增量備份功能,性能特別差

若昨天全備100G,今天更新了30G,作增量要掃描100G文件才知道哪些頁改動了,再去備份,線上很難接受

percona有個參數能夠監控哪些頁改動了,因此不用去掃以前的全部備份的表空間,但用的也比較少

要作增量,用二進制日誌的機制來作便可

5.2 指定庫表備份

一樣不推薦這種玩法,強烈建議完整備份

若是實例使用的是共享表空間(ibdata),不完整備份的話,可能會遇到各類問題

好比備份了a庫,沒備份b庫,用這個備份恢復後在b庫下面建立一個和以前同名的表就建立不了

5.3 遠程備份

innobackupex --compress --compress-threads=8 --stream=xbstream --user=root --parallel=4 ./ | ssh root@192.168.1.192 "xbstream -x -C /data/www/mysql/backup"