Postgresql-rman

• 聯機程序. 而且目標數據庫必須處於歸檔模式。
• 支持在線全備, 增量備份, 歸檔備份
  • 增量備份基於已經存在的一個全庫備份
• rman 自己使用pg_start_backup(), copy, pg_stop_backup() 備份模式

自己採用的是文本拷貝… cp/fwrite;

• pg_start_backup()
  • text 用戶定義的標籤, 是備份轉儲文件將被存儲的名字
  • boolean 指儘快執行pg_start_backup. 這將會強制一個當即執行的檢查點, 會致使I/O操做的峯值, 拖慢任何併發執行的查詢.
  • boolean 若是爲false, 則在完成備份後, pg_stop_backup將當即返回，而無需等待WAL歸檔
• pg_stop_backup()

rman總體架構

默認配置參數:

1. PGDATA
2. BACKUP_PATH
3. ARCLOG_PATH

pg_rman init

pg_rman show

pg_rman config --list

pg_rman backup -b full

​ -b inc [incremental]

​

​

​ -b arch [archive]

pg_rman restore

[新增功能] pg_rman blockrecover --datafile tablespaceOid/databaseOid/relfilenode --block 0

備份策略

1. 恢復窗口: 指定天數. 默認值爲 7.
2. 備份數量: 冗餘度保留。 默認值爲 1.

代碼組織架構:

.
├── backup.c
├── blockrecover.c
├── catalog.c
├── COPYRIGHT
├── data.c
├── delete.c
├── dir.c
├── docs
├── expected
├── idxpagehdr.h
├── init.c
├── Makefile
├── parray.c
├── parray.h
├── pg_rman.c
├── pg_rman.h
├── pgsql_src
├── pgut
├── README.md
├── restore.c
├── script
├── show.c
├── sql
├── util.c
├── validate.c
└── xlog.c

pg_rman-源碼淺析

代碼閱讀

* +----------------+---------------------------------+  
 * | PageHeaderData | linp1 linp2 linp3 ...           |  
 * +-----------+----+---------------------------------+  
 * | ... linpN |                                      |  
 * +-----------+--------------------------------------+  
 * |               ^ pd_lower                         |  
 * |                                                  |  
 * |                     v pd_upper                   |  
 * +-------------+------------------------------------+  
 * |                     | tupleN ...                 |  
 * +-------------+------------------+-----------------+  
 * |       ... tuple3 tuple2 tuple1 | "special space" |  
 * +--------------------------------+-----------------+

若是有數據刷入, 那麼將會作持久化，數據庫頁頭部的pd_lsn表示該數據庫頁最後一次變化時, 變化產生的REDO在wal file中的結束爲止.

若是wal flush的lsn插入位置 大於或者等於這個pd_lsn將表示這個頁的更改是可靠的. 即每次修改都將發生塊的變化: 包含LSN的修改.

便可以經過第一次備份開始時的全局LSN, 以及當前須要備份的數據的Page LSN來判斷此頁是否發生過修改.

修改了即備份，沒修改不須要備份, 從而實現數據庫的塊級別增量備份

增量備份關聯代碼:

pgBackupGetPath(prev_backup, prev_file_txt, lengthof(prev_file_txt),
                DATABASE_FILE_LIST);
            prev_files = dir_read_file_list(pgdata, prev_file_txt);

            /*
             * Do backup only pages having larger LSN than previous backup.
             */
            lsn = &prev_backup->start_lsn;
            xlogid = (uint32) (*lsn >> 32);
            xrecoff = (uint32) *lsn;
            elog(DEBUG, _("backup only the page updated after LSN(%X/%08X)"),
                            xlogid, xrecoff);

        /* Construct the directory for this backup within BACKUP_PATH. */
        pgBackupGetPath(&current, path, lengthof(path), DATABASE_DIR);

        /* Save the files listed above. */
        backup_files(pgdata, path, files, prev_files, lsn, current.compress_data, NULL);

[新增]塊恢復代碼:

for (loop = 0; loop <= brc.base_index; loop++)
    {
        backup = (pgBackup *) parray_get(backups, loop);

        /* don't use incomplete nor different timeline backup */
        if (backup->status != BACKUP_STATUS_OK || backup->tli != base_backup->tli)
            continue;
        if(-1 == brc.lastBackupIndex && HAVE_ARCLOG(backup) && brc.last_needed_index >= loop)
        {
            restore_archive_logs(backup,true);
        }
        /* use database backup only */
        if (BACKUP_MODE_INCREMENTAL > backup->backup_mode || brc.last_needed_index < loop)
            continue;

        elog(DEBUG, "found backup BK_KEY: \"%d\" can be used ",backup->backup_id);

        recoverBackup(backup,loop);
        =>  [[
                for(loop = 0; loop < brc.rbNum; loop++)
                {
                    /*If this block has find a page,skip it*/
                    if(brc.pageArray[loop])
                    {
                        elog(DEBUG,"block \'%u\' has find it's page,skip.",brc.recoverBlock[loop]);
                        continue;
                    }
                    page = findPageInBackup(backup, brc.recoverBlock[loop]);
                    if(page)
                    {
                        brc.pageArray[loop] = page;
                        if(-1 == brc.lastBackupIndex)
                        {
                            brc.lastBackupIndex = backupindex;
                            elog(DEBUG,"Find last backup can be used:BK_KEY \'%d\'",backup->backup_id);
                        }
                    }
                }
            ]]
    }

問題:

1. 隨意增大filenode大小, 即沒法整除8192時, 會默認增大一個Page。 此時的Page是不完整的. pg默認不開啓checksum校驗. 所以Pg會提示blk Num無效, 進行blockrecover操做時, 將會發生沒法恢復. 由於整個filenode自己就沒有正確的此Page;
2. 當隨意修改Page數據時, 有時會發生顯示數據不全，即數據條目與插入條目不符的狀況. 此時Pg自己沒法正常的數據異常告警. 請開啓checksum. 進行驗證.

checkSum異常告警;

WARNING:  01000: page verification failed, calculated checksum 11654 but expected 8293

1. 肯定table的tuple Num
2. 肯定table的page Num

確保開啓checksum功能, 保證Page的數據正常. 但對上述問題不產生有效影響;;