PostgreSQL備份-pg_rman備份

時間 2019-11-10

標籤 postgresql 備份 rman 欄目 Postgre SQL 简体版

原文原文鏈接

介紹

pg_rman是一款專門爲postgresql設計的在線備份恢復的工具。其支持在線和基於時間點備份方式，還能夠經過建立backup catalog來維護DB cluster備份信息。node

pg_rman特色：

使用簡單.一個命令便可完成備份和恢復.
支持在線全備,增量備份，歸檔備份.
支持備份壓縮.經過gzip工具實現頁內壓縮.
自動備份維護.自動刪除過時的WAL備份文件.
支持備份驗證.
恢復期間無事務丟失.支持基於PITR的配置文件生成器.

pg_rman的用法

pg_rman manage backup/recovery of PostgreSQL database.

Usage:
  pg_rman OPTION init
  pg_rman OPTION backup
  pg_rman OPTION restore
  pg_rman OPTION show [DATE]
  pg_rman OPTION show detail [DATE]
  pg_rman OPTION validate [DATE]
  pg_rman OPTION delete DATE
  pg_rman OPTION purge

Common Options:
  -D, --pgdata=PATH         location of the database storage area
  -A, --arclog-path=PATH    location of archive WAL storage area
  -S, --srvlog-path=PATH    location of server log storage area
  -B, --backup-path=PATH    location of the backup storage area
  -c, --check               show what would have been done
  -v, --verbose             show what detail messages
  -P, --progress            show progress of processed files

Backup options:
  -b, --backup-mode=MODE    full, incremental, or archive
  -s, --with-serverlog      also backup server log files
  -Z, --compress-data       compress data backup with zlib
  -C, --smooth-checkpoint   do smooth checkpoint before backup
  -F, --full-backup-on-error   switch to full backup mode
                               if pg_rman cannot find validate full backup
                               on current timeline
      NOTE: this option is only used in --backup-mode=incremental or archive.
  --keep-data-generations=NUM keep NUM generations of full data backup
  --keep-data-days=NUM        keep enough data backup to recover to N days ago
  --keep-arclog-files=NUM   keep NUM of archived WAL
  --keep-arclog-days=DAY    keep archived WAL modified in DAY days
  --keep-srvlog-files=NUM   keep NUM of serverlogs
  --keep-srvlog-days=DAY    keep serverlog modified in DAY days
  --standby-host=HOSTNAME   standby host when taking backup from standby
  --standby-port=PORT       standby port when taking backup from standby

Restore options:
  --recovery-target-time    time stamp up to which recovery will proceed
  --recovery-target-xid     transaction ID up to which recovery will proceed
  --recovery-target-inclusive whether we stop just after the recovery target
  --recovery-target-timeline  recovering into a particular timeline
  --hard-copy                 copying archivelog not symbolic link

Catalog options:
  -a, --show-all            show deleted backup too

Delete options:
  -f, --force               forcibly delete backup older than given DATE

Connection options:
  -d, --dbname=DBNAME       database to connect
  -h, --host=HOSTNAME       database server host or socket directory
  -p, --port=PORT           database server port
  -U, --username=USERNAME   user name to connect as
  -w, --no-password         never prompt for password
  -W, --password            force password prompt

Generic options:
  -q, --quiet               don't show any INFO or DEBUG messages
  --debug                   show DEBUG messages
  --help                    show this help, then exit
  --version                 output version information, then exit

使用前提

開啓歸檔
配置csvlog

安裝

下載地址：
https://github.com/ossc-db/pg_rman/releases
https://yum.postgresql.org/9.6/redhat/rhel-6.5-x86_64/git

[root@racnode1 ~]# rpm --force -ivh openssl10-libs-1.0.1e-1.ius.el6.x86_64.rpm 
warning: openssl10-libs-1.0.1e-1.ius.el6.x86_64.rpm: Header V4 DSA/SHA1 Signature, key ID 9cd4953f: NOKEY
Preparing...                ########################################### [100%]
   1:openssl10-libs         ########################################### [100%]
[root@racnode1 ~]# rpm -ivh postgresql96-libs-9.6.4-1PGDG.rhel6.x86_64.rpm 
warning: postgresql96-libs-9.6.4-1PGDG.rhel6.x86_64.rpm: Header V4 DSA/SHA1 Signature, key ID 442df0f8: NOKEY
Preparing...                ########################################### [100%]
   1:postgresql96-libs      ########################################### [100%]
[root@racnode1 ~]# rpm -ivh pg_rman-1.3.5-1.pg96.el6.x86_64.rpm 
Preparing...                ########################################### [100%]
   1:pg_rman                ########################################### [100%]

安裝完成後，pg_rman會安裝在默認的/usr/psql-9.6/目錄下
切換用戶到postgres用戶下 github

配置環境變量

export PG_RMAN=/usr/pgsql-9.6
export PATH=$PATH:$HOME/bin:/opt/pgsql95/bin:$PG_RMAN/bin
export BACKUP_PATH=/data/pg_rman

初始化備份目錄

首先須要初始化一個backup catalog，實際上就是須要一個目錄，這個目錄將用於存放備份的文件。同時這個目錄也會存放一些元數據，例如備份的配置文件，數據庫的systemid，時間線文件歷史等等。初始化命令須要兩個參數，分別爲備份目標目錄，以及數據庫的$PGDATAsql

[postgres@racnode1 ~]$ pg_rman init -B /data/pg_rman
INFO: ARCLOG_PATH is set to '/home/postgres/arch'
INFO: SRVLOG_PATH is set to '/home/postgres/data/pg_log'

[root@racnode1 pg_rman]# ll
total 16
drwx------. 4 postgres postgres 4096 Aug 30 00:06 backup
-rw-rw-r--. 1 postgres postgres   76 Aug 30 00:06 pg_rman.ini
-rw-rw-r--. 1 postgres postgres   40 Aug 30 00:06 system_identifier
drwx------. 2 postgres postgres 4096 Aug 30 00:06 timeline_history

你能夠把未來要使用的配置寫在這個配置文件中，或者寫在pg_rman的命令行中。數據庫

[postgres@racnode1 pg_rman]$ cat pg_rman.ini 
ARCLOG_PATH='/home/postgres/arch'
SRVLOG_PATH='/home/postgres/data/pg_log'

COMPRESS_DATA = YES  
KEEP_ARCLOG_FILES = 10  
KEEP_ARCLOG_DAYS = 10  
KEEP_DATA_GENERATIONS = 3  
KEEP_DATA_DAYS = 10  
KEEP_SRVLOG_FILES = 10  
KEEP_SRVLOG_DAYS = 10

備份操做

全量備份
[postgres@racnode1 ~]$ pg_rman backup -b full
INFO: copying database files
INFO: copying archived WAL files
INFO: backup complete
INFO: Please execute 'pg_rman validate' to verify the files are correctly copied.
INFO: start deleting old archived WAL files from ARCLOG_PATH (keep files = 10, keep days = 10)
INFO: the threshold timestamp calculated by keep days is "2017-08-20 00:00:00"
INFO: start deleting old backup (keep generations = 3 AND keep after = 2017-08-20 00:00:00)
INFO: does not include the backup just taken
WARNING: backup "2017-08-30 17:09:31" is not taken into account
DETAIL: This is not a valid backup.

增量備份
[postgres@racnode1 ~]$ psql
psql (9.6.0)
Type "help" for help.

postgres=# \c lottu lottu
You are now connected to database "lottu" as user "lottu".
lottu=# create table lottu as select generate_series(1,10) id;
SELECT 10

[postgres@racnode1 ~]$ pg_rman backup -b incremental
INFO: copying database files
INFO: copying archived WAL files
INFO: backup complete
INFO: Please execute 'pg_rman validate' to verify the files are correctly copied.
...
DETAIL: This is not valid backup.

備份集校驗

每次備份完，必需要作一次校驗，不然備份集不可用用來恢復，增量備份時也不會用它來作增量比較
[postgres@racnode1 ~]$ pg_rman validate  
INFO: validate: "2017-08-30 17:34:18" backup and archive log files by CRC
INFO: backup "2017-08-30 17:34:18" is valid
INFO: validate: "2017-08-30 17:37:36" backup and archive log files by CRC
INFO: backup "2017-08-30 17:37:36" is valid
[postgres@racnode1 ~]$ pg_rman show
=====================================================================
 StartTime           EndTime              Mode    Size   TLI  Status 
=====================================================================
2017-08-30 17:37:36  2017-08-30 17:37:45  FULL  3710kB     3  OK
2017-08-30 17:34:18  2017-08-30 17:34:27  FULL  3656kB     3  OK
2017-08-30 17:33:33  2017-08-30 17:33:45  FULL  3820kB     3  OK
2017-08-30 17:27:49  2017-08-30 17:27:58  FULL  3766kB     3  OK
2017-08-30 17:27:33  2017-08-30 17:27:43  FULL  3712kB     3  OK

查看備份集

[postgres@racnode1 ~]$ pg_rman show
=====================================================================
 StartTime           EndTime              Mode    Size   TLI  Status 
=====================================================================
2017-08-30 17:37:36  2017-08-30 17:37:45  FULL  3710kB     3  OK
2017-08-30 17:34:18  2017-08-30 17:34:27  FULL  3656kB     3  OK
2017-08-30 17:33:33  2017-08-30 17:33:45  FULL  3820kB     3  OK
2017-08-30 17:27:49  2017-08-30 17:27:58  FULL  3766kB     3  OK
2017-08-30 17:27:33  2017-08-30 17:27:43  FULL  3712kB     3  OK

刪除備份集

1. 按指定時間從catalog刪除備份集
例如我只須要個人備份集能恢復到2017-08-30 17:27:49，在這個時間點之前，不須要用來恢復到這個時間點的備份全刪掉。
[postgres@racnode1 ~]$ pg_rman delete "2017-08-30 17:27:49"
WARNING: cannot delete backup with start time "2017-08-30 17:27:49"
DETAIL: This is the latest full backup necessary for successful recovery.
INFO: delete the backup with start time: "2017-08-30 17:27:33"

2. 根據備份策略來刪除備份集
KEEP_DATA_GENERATIONS = 3   -- 備份集冗餘度是3
KEEP_DATA_DAYS = 10         -- 備份集保留日期是10d
在備份時；假如備份時間是2017-08-30。二者條件是「和」
INFO: start deleting old backup (keep generations = 3 AND keep after = 2017-08-20 00:00:00)

清除備份集

物理刪除已從catalog刪除的備份集
上面從catalog刪除的備份集;備份集文件夾並無一塊兒刪除。只是把arclog/ database/ srvlog/刪除了。
[postgres@racnode1 ~]$ pg_rman purge
INFO: DELETED backup "2017-08-30 17:27:33" is purged
INFO: DELETED backup "2017-08-30 17:26:22" is purged
INFO: DELETED backup "2017-08-30 17:15:26" is purged

恢復操做

pg_rman restore [options:]  
命令的選項也很簡單，甚至能夠不指定任何option
Restore options:  
The parameters which are started with –recovery are same as parameters in recovery.conf. See also 「Recovery Configuration」 for details.  
接下來的幾個配置，與recovery.conf的意思對齊。
若是不指定時間線，則使用$PGDATA/global/pg_control，若是沒有$PGDATA/global/pg_control，則使用最新的全量備份集的時間線。
--recovery-target-timeline TIMELINE  
Specifies recovering into a particular timeline. If not specified, the current timeline from ($PGDATA/global/pg_control) is used.  
若是不指定，則恢復到最新時間
--recovery-target-time TIMESTAMP  
This parameter specifies the time stamp up to which recovery will proceed. If not specified, continue recovery to the latest time.  
若是不指定，則恢復到最新xid
--recovery-target-xid XID  
This parameter specifies the transaction ID up to which recovery will proceed. If not specified, continue recovery to the latest xid.  
若是不指定，則默認使用true，即恢復到包含恢復目標XID的commit record爲止，或者第一筆commit record ts>指定ts的 commit redo record爲止； 
若是是false則不apply恢復目標XID的commit record，或者不apply第一筆commit record ts>=指定ts的 commit redo record。
--recovery-target-inclusive  
Specifies whether we stop just after the specified recovery target (true), or just before the recovery target (false). Default is true.  
是否使用硬連接複製archive log，而不須要拷貝文件
The following parameter determines the behavior of restore.  
--hard-copy  
The archive WAL are copied to archive WAL storage area. If not specified, pg_rman makes symbolic link to archive WAL where are in the backup catalog directory. 

恢復時，須要注意，你能夠選擇原地恢復（覆蓋式），或者使用新的$PGDATA做爲恢復目標。
可是請注意，無論是哪一種恢復方式，若是在本機恢復的話，pg_rman可能會覆蓋原有的數據文件，arch, pg_xlog目錄中的文件，因此，若是你要保留原數據，建議先將原數據目錄重命名。 
1. 原地恢復 
2. 使用新的$PGDATA恢復
[postgres@racnode1 ~]$ pg_rman restore --recovery-target-time "2017-08-30 19:48:00";
INFO: the recovery target timeline ID is not given
INFO: use timeline ID of current database cluster as recovery target: 5
INFO: calculating timeline branches to be used to recovery target point
INFO: searching latest full backup which can be used as restore start point
INFO: found the full backup can be used as base in recovery: "2017-08-30 17:37:36"
INFO: copying online WAL files and server log files
INFO: clearing restore destination
INFO: validate: "2017-08-30 17:37:36" backup and archive log files by SIZE
INFO: backup "2017-08-30 17:37:36" is valid
INFO: restoring database files from the full mode backup "2017-08-30 17:37:36"
INFO: searching incremental backup to be restored
INFO: validate: "2017-08-30 17:59:34" backup and archive log files by SIZE
INFO: backup "2017-08-30 17:59:34" is valid
INFO: restoring database files from the incremental mode backup "2017-08-30 17:59:34"
INFO: searching backup which contained archived WAL files to be restored
INFO: backup "2017-08-30 17:59:34" is valid
INFO: restoring WAL files from backup "2017-08-30 17:59:34"
INFO: restoring online WAL files and server log files
INFO: generating recovery.conf
INFO: restore complete
HINT: Recovery will start automatically when the PostgreSQL server is started.

附上備份腳本

##===========================================================  
##   pg_rman.sh                
##   created by lottu           
##   2017/08/07    
##   usage: pg_rman.sh                            
##============================================================  
#!/bin/bash

source /home/postgres/.bash_profile

DATE=`date +%Y%m%d`;
PG_HOME=/home/postgres
BACK_LOG=/home/postgres/log/pg_rman_${DATE}.log

#START BACKUP
echo "START BACKUP............................................" > $BACK_LOG
#執行備份命令
pg_rman backup -b full >> $BACK_LOG
#備份集校驗
pg_rman validate   >>  $BACK_LOG
#檢查備份是否成功
error_num=`pg_rman show | awk 'BEGIN{n=0}{if(NR > 3 && $8 != "OK")n++}END{print n}'`
if [ $error_num > 0 ];then
    # message="Postgres 數據庫服務器${hostname}在${DATE}備份失敗" 
    # 報警方式：微信，短信，郵件.....
    # wechat.sh $message
    echo "0"
fi
#清理無效備份集
pg_rman purge   >> $BACK_LOG

echo "BACKUP  END............................................" >> $BACK_LOG

參考文獻：--https://yq.aliyun.com/articles/71929?spm=5176.100239.blogcont2274.11.N0xZJ5bash

相關標籤/搜索

每日一句

每一个你不满意的现在，都有一个你没有努力的曾经。