前言
上次分享了barman備份的安裝部署過程，此次將繼續分享一下barman備份恢復工具的使用過程。
barman是目前postgreSQL數據庫的比較成熟的一款備份工具，是基於postgresql的一款物理備份工具。與oracle數據庫中的rman較爲相似。
其簡單的工做原理即爲：
物理全備+歸檔備份（WAL日誌備份），從而達到任意時間點的恢復。即恢復全備後，經過指定WAL日誌的時間點、LSN等便可恢復到數據庫的任意時間點。
barman備份主要分爲三類：
1.僅流複製；
2.ssh/rsync;
3.流複製+ssh/rsync
node
上一次，已經完成了barman工具的安裝部署工做。本文中，將對postgresql和barman進行配置並測試備份恢復。算法
1、postgreSQL配置
參數配置（具體參數能夠根據實際狀況配置）：
listen_addresses = '*' port = 18801 max_connections = 100 superuser_reserved_connections = 3 tcp_keepalives_idle = 7200 tcp_keepalives_interval = 130 tcp_keepalives_count = 3 shared_buffers = 256MB work_mem = 16MB maintenance_work_mem = 128MB wal_level = logical full_page_writes = on max_wal_size = 1GB min_wal_size = 80MB wal_keep_segments = 32 logging_collector = on log_directory = '/PgData/pg_log' log_filename = 'postgresql-%Y-%m-%d_%H%M%S.log' log_truncate_on_rotation = on log_rotation_age = 1d log_rotation_size = 0 log_min_messages = info log_min_error_statement = info shared_preload_libraries = 'pg_stat_statements' #加載pg_stat_statements模塊 track_io_timing = on #跟蹤IO消耗的時間 pg_stat_statements.max = 10000 #最多保留多少條統計信息，經過LRU算法，覆蓋老的記錄。 pg_stat_statements.track = all #all：全部SQL包括函數內嵌套的SQL,top：直接執行的SQL(函數內的sql不被跟蹤),none：不跟蹤 pg_stat_statements.track_utility=on # 是否跟蹤非DML語句 (例如DDL，DCL)，on表示跟蹤, off表示不跟蹤 pg_stat_statements.save = on # 重啓後是否保留統計信息 #track_activity_query_size = 2048 #設置單條SQL的最長長度，超過被截斷顯示
修改pg_hba.conf
host all barman 0.0.0.0/0 trust host replication barmstr 0.0.0.0/0 trust
postgresql數據庫上建立用戶（barman須要superuser權限，barmanStr須要replication權限）：
create user barman superuser password 'barman'; create user barmanStr replication password 'barman';
重啓數據庫：
pg_ctl -D /PgData start
鏈接嘗試：
在備份服務器上嘗試鏈接驗證：sql
psql -h 192.168.138.141 -U barman -d wangxin -p 18801 psql -h 192.168.138.141 -U barmstr -d wangxin -p 18801
2、barman配置：
建立用戶：
useradd barman
修改barman的配置文件：
vi /etc/barman.conf數據庫
[barman] barman_user = barman configuration_files_directory = /etc/barman.d ;指定barman_home，即備份文件的存放路徑 barman_home = /home/barman log_file = /home/barman/barman.log log_level = INFO compression = gzip parallel_jobs = 1 immediate_checkpoint = true basebackup_retry_times = 3 basebackup_retry_sleep = 60 check_timeout = 120 retention_policy = REDUNDANCY 2 retention_policy = RECOVERY WINDOW OF 4 WEEKS
修改服務配置文件：
cp streaming-server.conf-template pgsql-streaming.conf vi pgsql-streaming.conf
內容以下：json
[pgsql_streaming] description = "pgsql11-streaming (Streaming-Only)" conninfo = host=pgsql11 user=barman dbname=wangxin port=18801 streaming_conninfo = host=pgsql11 user=barmanstr port=18801 backup_method = postgres streaming_backup_name = barman_streaming_backup streaming_archiver = on archiver = on slot_name = pgsql_streaming create_slot = auto streaming_archiver_name = barman_receive_wal streaming_archiver_batch_size = 50 path_prefix = "/pg_client/bin"
配置服務端和備份端的ssh免密登陸：
pgsql:
服務器
ssh-keygen -t rsa cat id_rsa.pub >> authorized_keys scp authorized_keys >> barman@barman:/home/barman/.ssh/
barman:oracle
ssh-keygen -t rsa cat id_rsa.pub >> authorized_keys scp authorized_keys >> pgsql@pgsql11:/home/pgsql/.ssh/
pgsql:app
chmod -R 600 /home/pgsql/.ssh/authorized_keys
barman:ssh
chmod -R 600 /home/barman/.ssh/authorized_keys
建立slot槽：
barman receive-wal --create-slot pgsql_streaming
進行check檢查：
barman check pgsql_streaming
查看鏈接服務的信息
[barman@barman ~]$ barman show-server pgsql_streaming
Server pgsql_streaming:
active: True
archive_command: (disabled)
archive_mode: off
archive_timeout: 0
archived_count: 0
archiver: True
archiver_batch_size: 0
backup_directory: /home/barman/pgsql_streaming
backup_method: postgres
backup_options: BackupOptions({'concurrent_backup'})
bandwidth_limit: None
barman_home: /home/barman
barman_lock_directory: /home/barman
basebackup_retry_sleep: 60
basebackup_retry_times: 3
basebackups_directory: /home/barman/pgsql_streaming/base
check_timeout: 120
checkpoint_timeout: 300
compression: gzip
config_file: /PgData/postgresql.conf
connection_error: None
conninfo: host=pgsql11 user=barman dbname=wangxin port=18801
create_slot: auto
current_archived_wals_per_second: 0.0
current_lsn: 0/669C3780
current_size: 1134777182
current_xlog: 000000010000000000000066
custom_compression_filter: None
custom_decompression_filter: None
data_checksums: off
data_directory: /PgData
description: pgsql11-streaming (Streaming-Only)
disabled: False
errors_directory: /home/barman/pgsql_streaming/errors
failed_count: 0
has_backup_privileges: True
hba_file: /PgData/pg_hba.conf
hot_standby: on
ident_file: /PgData/pg_ident.conf
immediate_checkpoint: True
**incoming_wals_directory: /home/barman/pgsql_streaming/incoming**
is_archiving: False
is_in_recovery: False
is_superuser: True
last_archived_time: None
last_archived_wal: None
last_backup_maximum_age: None
last_failed_time: None
last_failed_wal: None
max_incoming_wals_queue: None
max_replication_slots: 10
max_wal_senders: 10
minimum_redundancy: 0
msg_list: []
name: pgsql_streaming
network_compression: False
parallel_jobs: 1
passive_node: False
path_prefix: /pg_client/bin
pg_basebackup_bwlimit: True
pg_basebackup_compatible: True
pg_basebackup_installed: True
pg_basebackup_path: /pg_client/bin/pg_basebackup
pg_basebackup_tbls_mapping: True
pg_basebackup_version: 11.4
pg_receivexlog_compatible: True
pg_receivexlog_installed: True
pg_receivexlog_path: /pg_client/bin/pg_receivewal
pg_receivexlog_supports_slots: True
pg_receivexlog_synchronous: False
pg_receivexlog_version: 11.4
pgespresso_installed: False
post_archive_retry_script: None
post_archive_script: None
post_backup_retry_script: None
post_backup_script: None
post_delete_retry_script: None
post_delete_script: None
post_recovery_retry_script: None
post_recovery_script: None
post_wal_delete_retry_script: None
post_wal_delete_script: None
postgres_systemid: 6901481653023568481
pre_archive_retry_script: None
pre_archive_script: None
pre_backup_retry_script: None
pre_backup_script: None
pre_delete_retry_script: None
pre_delete_script: None
pre_recovery_retry_script: None
pre_recovery_script: None
pre_wal_delete_retry_script: None
pre_wal_delete_script: None
primary_ssh_command: None
recovery_options: RecoveryOptions()
replication_slot: Record(slot_name='pgsql_streaming', active=False, restart_lsn=None)
replication_slot_support: True
retention_policy: RECOVERY WINDOW OF 4 WEEKS
retention_policy_mode: auto
reuse_backup: None
server_txt_version: 11.4
slot_name: pgsql_streaming
ssh_command: None
stats_reset: 2020-12-02 10:33:56.983254+08:00
streaming: True
streaming_archiver: True
streaming_archiver_batch_size: 50
streaming_archiver_name: barman_receive_wal
streaming_backup_name: barman_streaming_backup
streaming_conninfo: host=pgsql11 user=barmanstr port=18801
streaming_supported: True
streaming_systemid: 6901481653023568481
streaming_wals_directory: /home/barman/pgsql_streaming/streaming
synchronous_standby_names: ['']
tablespace_bandwidth_limit: None
timeline: 1
wal_compression: off
wal_keep_segments: 0
wal_level: logical
wal_retention_policy: MAIN
wals_directory: /home/barman/pgsql_streaming/wals
xlog_segment_size: 16777216
xlogpos: 0/669C3780
檢查barman是否知足備份要求：tcp
[barman@barman ~]$ barman check pgsql_streaming
Server pgsql_streaming:
WAL archive: FAILED (please make sure WAL shipping is setup)
PostgreSQL: OK
superuser or standard user with backup privileges: OK
PostgreSQL streaming: OK
wal_level: OK
replication slot: FAILED (slot 'pgsql_streaming' not initialised: is 'receive-wal' running?)
directories: OK
retention policy settings: OK
backup maximum age: OK (no last_backup_maximum_age provided)
compression settings: OK
failed backups: OK (there are 0 failed backups)
minimum redundancy requirements: OK (have 0 backups, expected at least 0)
pg_basebackup: OK
pg_basebackup compatible: OK
pg_basebackup supports tablespaces mapping: OK
systemid coherence: OK (no system Id stored on disk)
pg_receivexlog: OK
pg_receivexlog compatible: OK
receive-wal running: FAILED (See the Barman log file for more details)
archive_mode: FAILED (please set it to 'on' or 'always')
archive_command: FAILED (please set it accordingly to documentation)
archiver errors: OK
能夠關注到一個錯誤：
replication slot: FAILED (slot ‘pgsql_streaming’ not initialised: is ‘receive-wal’ running?)
解決該錯誤，咱們須要執行如下命令，讓barman服務器來強制接受源端的wal日誌：
nohup barman receive-wal pgsql_streaming &
再次進行check：
[barman@barman ~]$ barman check pgsql_streaming
Server pgsql_streaming:
WAL archive: FAILED (please make sure WAL shipping is setup)
PostgreSQL: OK
superuser or standard user with backup privileges: OK
PostgreSQL streaming: OK
wal_level: OK
replication slot: OK
directories: OK
retention policy settings: OK
backup maximum age: OK (no last_backup_maximum_age provided)
compression settings: OK
failed backups: OK (there are 0 failed backups)
minimum redundancy requirements: OK (have 0 backups, expected at least 0)
pg_basebackup: OK
pg_basebackup compatible: OK
pg_basebackup supports tablespaces mapping: OK
systemid coherence: OK (no system Id stored on disk)
pg_receivexlog: OK
pg_receivexlog compatible: OK
receive-wal running: OK
archive_mode: FAILED (please set it to 'on' or 'always')
archive_command: FAILED (please set it accordingly to documentation)
archiver errors: OK
此時，能夠發現，錯誤只有如下內容：
WAL archive: FAILED (please make sure WAL shipping is setup)
archive_mode: FAILED (please set it to ‘on’ or ‘always’)
archive_command: FAILED (please set it accordingly to documentation)
可是這三個錯誤是經過ssh/rsync的方式，若是咱們只用streaming的方式能夠忽略，而後直接發起備份，固然若是想要處理這些問題，能夠這樣作：
1.問題一
執行barman cron: Starting WAL archiving for server pgsql_streaming 執行barman switch-xlog barman switch-xlog pgsql_streaming
2.問題二
根據提示在server端設置archive_mod和archive_command參數： vi postgresql.conf archive_mod = on archive_command = 'rsync -a %p barman@barman:/home/barman/pgsql_streaming/incoming/%f' #該路徑就是咱們剛在barman show-server pg_streaming中查詢的
重啓數據庫
此時再次執行check能夠看到狀態都正常了：
[root@barman barman.d]# barman check pgsql_streaming
Server pgsql_streaming:
PostgreSQL: OK
superuser or standard user with backup privileges: OK
PostgreSQL streaming: OK
wal_level: OK
replication slot: OK
directories: OK
retention policy settings: OK
backup maximum age: OK (no last_backup_maximum_age provided)
compression settings: OK
failed backups: OK (there are 0 failed backups)
minimum redundancy requirements: OK (have 0 backups, expected at least 0)
pg_basebackup: OK
pg_basebackup compatible: OK
pg_basebackup supports tablespaces mapping: OK
systemid coherence: OK (no system Id stored on disk)
pg_receivexlog: OK
pg_receivexlog compatible: OK
receive-wal running: OK
archive_mode: OK
archive_command: OK
continuous archiving: OK
archiver errors: OK
所有正常之後，咱們則能夠開始進行備份。
3、barman備份：
進行備份：
barman backup pgsql_streaming
查看備份：
barman list-backup pgsql_streaming
查看備份詳細信息
barman show-backup pgsql_streaming 20201204T164310
Backup 20201204T164310:
Server Name : pgsql_streaming
System Id : 6901481653023568481
Status : DONE
PostgreSQL Version : 110004
PGDATA directory : /PgData
Base backup information:
Disk usage : 1.1 GiB (1.1 GiB with WALs)
Incremental size : 1.1 GiB (-0.00%)
Timeline : 1
Begin WAL : 00000001000000000000006F
End WAL : 00000001000000000000006F
WAL number : 1
WAL compression ratio: 99.90%
Begin time : 2020-12-04 16:43:10+08:00
End time : 2020-12-04 16:43:39.799061+08:00
Copy time : 29 seconds
Estimated throughput : 37.4 MiB/s
Begin Offset : 40
End Offset : 0
Begin LSN : 0/6F000028
End LSN : 0/70000000
WAL information:
No of files : 0
Disk usage : 0 B
Last available : 00000001000000000000006F
Catalog information:
Retention Policy : VALID
Previous Backup : - (this is the oldest base backup)
Next Backup : - (this is the latest base backup)
實時查看備份狀況:
[root@barman .ssh]# barman status pgsql_streaming
Server pgsql_streaming:
Description: pgsql11-streaming (Streaming-Only)
Active: True
Disabled: False
PostgreSQL version: 11.4
Cluster state: in production
pgespresso extension: Not available
Current data size: 1.1 GiB
PostgreSQL Data directory: /PgData
Current WAL segment: 000000010000000000000073
PostgreSQL 'archive_command' setting: rsync -a %p barman@barman:/home/barman/pgsql_streaming/incoming/%f
Last archived WAL: 000000010000000000000072, at Fri Dec 4 16:50:13 2020
Failures of WAL archiver: 0
Server WAL archiving rate: 0.26/hour
Passive node: False
Retention policies: enforced (mode: auto, retention: RECOVERY WINDOW OF 4 WEEKS, WAL retention: MAIN)
No. of available backups: 1
First available backup: 20201204T164310
Last available backup: 20201204T164310
Minimum redundancy requirements: satisfied (1/0)
說明：
配置文件中的barman_home，即爲備份存放路徑
打開barman_home下對應的備份服務下，能夠看到如下目錄：
drwxrwxr-x. 2 barman barman 6 Dec 4 15:36 errors
-rw-r–r--. 1 barman barman 63 Dec 4 16:43 identity.json
drwxrwxr-x. 3 barman barman 45 Dec 4 16:43 wals
drwxrwxr-x. 5 barman barman 75 Dec 4 17:14 base
drwxrwxr-x. 2 barman barman 78 Dec 4 17:14 streaming
drwxrwxr-x. 2 barman barman 6 Dec 4 17:14 incoming
其中，比較重要的有wals、base、streaming、incoming
base中，存放的是數據庫的基本備份文件(最近的一份全備)
streaming中，存放的是實時傳過來的wal文件
incoming中，存放的是主庫傳輸過來的歸檔文件
wals中，是全部的歸檔文件，能夠根據這個和基礎備份恢復到任意時間點
barman恢復：
查看擁有的備份集：
barman list-backup pgsql_streaming pgsql_streaming 20201207T231544 - Sun Dec 6 23:15:56 2020 - Size: 356.9 MiB - WAL Size: 0 B pgsql_streaming 20201207T225457 - Sun Dec 6 22:55:44 2020 - Size: 1.1 GiB - WAL Size: 4.4 MiB pgsql_streaming 20201204T171433 - Fri Dec 4 17:14:42 2020 - Size: 1.1 GiB - WAL Size: 6.3 MiB pgsql_streaming 20201204T170853 - Fri Dec 4 17:09:11 2020 - Size: 1.1 GiB - WAL Size: 503.1 KiB pgsql_streaming 20201204T164310 - Fri Dec 4 16:43:39 2020 - Size: 1.1 GiB - WAL Size: 2.6 MiB
檢查選定的備份的詳細信息:
[barman@barman 0000000100000000]$ barman show-backup pgsql_streaming 20201207T225457
Backup 20201207T225457:
Server Name : pgsql_streaming
System Id : 6901481653023568481
Status : DONE
PostgreSQL Version : 110004
PGDATA directory : /PgData
Base backup information:
Disk usage : 1.1 GiB (1.1 GiB with WALs)
Incremental size : 1.1 GiB (-0.00%)
Timeline : 1
Begin WAL : 00000001000000000000007C
End WAL : 00000001000000000000007E
WAL number : 3
WAL compression ratio: 99.90%
Begin time : 2020-12-06 22:55:02+08:00
End time : 2020-12-06 22:55:44.674452+08:00
Copy time : 42 seconds
Estimated throughput : 26.4 MiB/s
Begin Offset : 40
End Offset : 96
Begin LSN : 0/7C000028
End LSN : 0/7E000060
WAL information:
No of files : 4
Disk usage : 4.4 MiB
Compression ratio : 93.14%
Last available : 000000010000000000000082
Catalog information:
Retention Policy : VALID
Previous Backup : 20201204T171433
Next Backup : 20201207T231544
barman恢復（恢復到遠端數據庫）：
[barman@barman PgData_recover]$ barman recover pgsql_streaming 20201207T225457 /PgData --remote-ssh-command "ssh postgres@pgsql11-recover" --target-time "2020-12-06 22:57:00.0000"
Processing xlog segments from streaming for pgsql_streaming
000000010000000000000083
Processing xlog segments from file archival for pgsql_streaming
000000010000000000000083
The authenticity of host 'pgsql11-recover (192.168.138.143)' can't be established.
ECDSA key fingerprint is SHA256:vu/gUNno3I8feQxKWcPkcm2kSS+g72zAE2JafM9CBF4.
ECDSA key fingerprint is MD5:59:a7:b9:12:29:31:b3:78:7f:ed:71:44:7a:7e:f2:fc.
Are you sure you want to continue connecting (yes/no)? yes
postgres@pgsql11-recover's password:
Starting remote restore for server pgsql_streaming using backup 20201207T225457
Destination directory: /PgData
Remote command: ssh postgres@pgsql11-recover
Doing PITR. Recovery target time: '2020-12-06 22:57:00+08:00'
postgres@pgsql11-recover's password:
postgres@pgsql11-recover's password:
postgres@pgsql11-recover's password:
postgres@pgsql11-recover's password:
Copying the base backup.
postgres@pgsql11-recover's password:
postgres@pgsql11-recover's password:
postgres@pgsql11-recover's password:
postgres@pgsql11-recover's password:
postgres@pgsql11-recover's password:
Copying required WAL segments.
postgres@pgsql11-recover's password:
Generating recovery configuration
postgres@pgsql11-recover's password:
postgres@pgsql11-recover's password:
postgres@pgsql11-recover's password:
postgres@pgsql11-recover's password:
Identify dangerous settings in destination directory.
postgres@pgsql11-recover's password:
IMPORTANT
These settings have been modified to prevent data losses
postgresql.conf line 706: archive_command = false
Recovery completed (start time: 2020-12-07 23:57:04.056352, elapsed time: 1 minute, 54 seconds)
Your PostgreSQL server has been successfully prepared for recovery!
出現：
Your PostgreSQL server has been successfully prepared for recovery!
則表示恢復成功。
經過pg_dump/dblink等方式，將恢復的表或者數據回寫到源端。