關於Oracle GoldenGate中Extract的checkpoint的理解轉載

時間 2019-11-06

標籤關於 oracle goldengate extract checkpoint 理解轉載欄目 Oracle 简体版

原文原文鏈接

什麼是checkpoint？

在Oracle 數據庫中checkpoint的意思是將內存中的髒數據強制寫入到磁盤的事件,其做用是保持內存中的數據與磁盤上的數據一致。SCN是用來描述該事件發生的準確的時間點。

而GoldenGate中出現的checkpoint有着不一樣的含義。簡單的說就是position，位置的意思。它記錄了Extract進程在抽取事務時的進度。
使用INFO showch 命令能夠查看當前正在運行的Extract進程中所記錄的checkpoint。

當GoldenGate被計劃內或者是計劃外中斷了，爲了保證事務的連續性咱們從哪裏續傳呢？答案是：哪裏中斷，哪裏繼續。在一個繁忙的數據庫中用時間戳來衡量OGG抽取進度是不許確的，由於任何一個時間點都有可能截斷了正在處理中的事務。準確的記錄的方法是參考該事物在發起時所在trail文件的字節位置（RBA）。

startup checkpoint:

Extract進程啓動時的信息，用多個參數描述了一個確切的時間點。包括thread,sequence,timestamp,RBA,SCN,redo log
sequence表示redo log、或者archive log的序列號。
RBA表示改時間點處於該sequence號日誌文件中的字節地址。
SCN表示Oracle內部時鐘

recovery checkpoint:

到目前爲止還沒結束的而且耗時最長的事務信息。包括thread,sequence,timestamp,RBA,SCN,redo log。
若是源端的抽取進程中斷了，那麼斷點續傳的位置就是這裏了。

current checkpoint:

當前正在讀取的事務信息。包括thread,sequence,timestamp,RBA,SCN,redo log

write checkpoint：
當前正在寫入的trail文件信息。包括隊列號（sequence）,RBA,時間戳，隊列文件
sequence表示trail文件的序號，有別於數據庫中的同名概念。例如：trail文件ex000001的sequence爲1

RBA表示當前信息在trail文件中的位置，有別於數據庫中事務日誌的RBA。一般這個RBA的數值等於當前正在寫的trail文件的大小。
能夠經過ls -al ./dirdat/ex000001來查看該文件的字節數來對比write checkpoint的RBA

下面經過一個屏幕輸出來對應一下剛纔提到的概念。在實驗開始時，還須要再次執行該命令來獲取最新的檢查點信息。

查看抽取進程ex1的檢查點：
GGSCI (node3) 9> info ex1 showch

EXTRACT    EX1       Last Started 2014-11-03 13:23   Status RUNNING << 進程開始抽取事務的時間，ADD EXTRACT以後就生效了
Checkpoint Lag       00:00:03 (updated 00:00:07 ago)
Log Read Checkpoint Oracle Redo Logs
                     2014-11-03 15:13:04 Thread 1, Seqno 21, RBA 26012672 << 2個節點的RAC就有2個Checkpoint
                     SCN 0.1614343 (1614343)
Log Read Checkpoint Oracle Redo Logs
                     2014-11-03 15:13:02 Thread 2, Seqno 29, RBA 6584832
                     SCN 0.1614345 (1614345)

Current Checkpoint Detail:

Read Checkpoint #1 << 抽取進程的讀檢查點節點1

Oracle Threaded Redo Log

Startup Checkpoint (starting position in the data source):
    Thread #: 1
    Sequence #: 11
    RBA: 6084096
    Timestamp: 2014-11-02 17:41:26.000000
    SCN: Not available
    Redo File:

Recovery Checkpoint是咱們續傳時關注的位置
Recovery Checkpoint (position of oldest unprocessed transaction in the data source):
    Thread #: 1
    Sequence #: 21 << 所在隊列序號
    RBA: 26012176   << 所在隊列中的具體位置
    Timestamp: 2014-11-03 15:13:04.000000 << 事務發起的時間戳
    SCN: 0.1614343 (1614343)
    Redo File: +DATA/prod/onlinelog/group_1.299.859509355

Current Checkpoint (position of last record read in the data source):
    Thread #: 1
    Sequence #: 21
    RBA: 26012672
    Timestamp: 2014-11-03 15:13:04.000000
    SCN: 0.1614343 (1614343)
    Redo File: +DATA/prod/onlinelog/group_1.299.859509355

Read Checkpoint #2 << 抽取進程的讀檢查點節點2

Oracle Threaded Redo Log

Startup Checkpoint (starting position in the data source):
    Thread #: 2
    Sequence #: 11
    RBA: 4263936
    Timestamp: 2014-11-02 17:41:26.000000
    SCN: Not available
    Redo File:

Recovery Checkpoint (position of oldest unprocessed transaction in the data source):
    Thread #: 2
    Sequence #: 29
    RBA: 6584336
    Timestamp: 2014-11-03 15:13:02.000000
    SCN: 0.1614345 (1614345)
    Redo File: +DATA/prod/onlinelog/group_3.301.859509357

Current Checkpoint (position of last record read in the data source):
    Thread #: 2
    Sequence #: 29
    RBA: 6584832
    Timestamp: 2014-11-03 15:13:02.000000
    SCN: 0.1614345 (1614345)
    Redo File: +DATA/prod/onlinelog/group_3.301.859509357

Write Checkpoint #1

GGS Log Trail

Current Checkpoint (current write position):
    Sequence #: 47
    RBA: 1124760
    Timestamp: 2014-11-03 15:13:07.093886
    Extract Trail: /goldengate/dirdat/ex

CSN state information:
CRC: E4-D6-3B-B4
CSN: Not available

Header:
Version = 2
Record Source = A
Type = 11
# Input Checkpoints = 2
# Output Checkpoints = 1

File Information:
Block Size = 2048
Max Blocks = 100
Record Length = 4096
Current Offset = 0

Configuration:
Data Source = 3
Transaction Integrity = 1
Task Type = 0

Status:
Start Time = 2014-11-03 13:23:05
Last Update Time = 2014-11-03 15:13:07
Stop Status = A
Last Result = 0

當前源端有100萬條記錄
SYS@PROD2 > select count(*) from snow.t1;

COUNT(*)
----------
   1000000

系統時間爲2014-11-03 15:37:08
SYS@PROD2 > alter session set nls_date_format='yyyy-mm-dd hh24:mi:ss';

Session altered.

SYS@PROD2 > select sysdate from dual;

SYSDATE
-------------------
2014-11-03 15:37:08

源端插入一條新數據，不提交
SYS@PROD2 > insert into snow.t1 values(1000001,'Snow');

1 row created.

SYS@PROD2 >

關閉源端全部Extract進程
GGSCI (node3) 4> stop *

Sending STOP request to EXTRACT DP1 ...
Request processed.

Sending STOP request to EXTRACT EX1 ...
Request processed.

觀察最新的checkpoint信息
GGSCI (node3) 14> info ex1 showch

EXTRACT    EX1       Last Started 2014-11-03 15:41   Status RUNNING
Checkpoint Lag       00:00:02 (updated 00:00:01 ago)
Log Read Checkpoint Oracle Redo Logs
                     2014-11-03 15:41:48 Thread 1, Seqno 21, RBA 27887616
                     SCN 0.1620285 (1620285)
Log Read Checkpoint Oracle Redo Logs
                     2014-11-03 15:41:46 Thread 2, Seqno 29, RBA 7889920
                     SCN 0.1620284 (1620284)

Current Checkpoint Detail:

Read Checkpoint #1

Oracle Threaded Redo Log

Startup Checkpoint (starting position in the data source):
    Thread #: 1
    Sequence #: 21
    RBA: 27644432
    Timestamp: 2014-11-03 15:39:18.000000
    SCN: Not available
    Redo File: +DATA/prod/onlinelog/group_3.301.859509357

Recovery Checkpoint (position of oldest unprocessed transaction in the data source):
    Thread #: 1
    Sequence #: 21
    RBA: 27887120
    Timestamp: 2014-11-03 15:41:48.000000
    SCN: 0.1620285 (1620285)
    Redo File: +DATA/prod/onlinelog/group_1.299.859509355

Current Checkpoint (position of last record read in the data source):
    Thread #: 1
    Sequence #: 21
    RBA: 27887616
    Timestamp: 2014-11-03 15:41:48.000000
    SCN: 0.1620285 (1620285)
    Redo File: +DATA/prod/onlinelog/group_1.299.859509355

Read Checkpoint #2

Oracle Threaded Redo Log

Startup Checkpoint (starting position in the data source):
    Thread #: 2
    Sequence #: 29
    RBA: 7750160
    Timestamp: 2014-11-03 15:39:16.000000
    SCN: Not available
    Redo File: +DATA/prod/onlinelog/group_3.301.859509357

Recovery Checkpoint (position of oldest unprocessed transaction in the data source):
    Thread #: 2
    Sequence #: 29
    RBA: 7750160
    Timestamp: 2014-11-03 15:37:40.000000
    SCN: 0.1619666 (1619666)
    Redo File: +DATA/prod/onlinelog/group_3.301.859509357

Current Checkpoint (position of last record read in the data source):
    Thread #: 2
    Sequence #: 29
    RBA: 7889920
    Timestamp: 2014-11-03 15:41:46.000000
    SCN: 0.1620284 (1620284)
    Redo File: +DATA/prod/onlinelog/group_3.301.859509357

Write Checkpoint #1

GGS Log Trail

Current Checkpoint (current write position):
    Sequence #: 48
    RBA: 1076
    Timestamp: 2014-11-03 15:41:50.718150
    Extract Trail: /goldengate/dirdat/ex

CSN state information:
CRC: CE-DB-CE-78
CSN: Not available

Header:
Version = 2
Record Source = A
Type = 11
# Input Checkpoints = 2
# Output Checkpoints = 1

File Information:
Block Size = 2048
Max Blocks = 100
Record Length = 4096
Current Offset = 0

Configuration:
Data Source = 3
Transaction Integrity = 1
Task Type = 0

Status:
Start Time = 2014-11-03 15:41:29
Last Update Time = 2014-11-03 15:41:50
Stop Status = A
Last Result = 0

修改抽取進程啓動位置，兩個進程
GGSCI (node3) 7> ALTER EXTRACT ex1, EXTSEQNO 21, EXTRBA 27644432 THREAD 1
EXTRACT altered.

GGSCI (node3) 8> ALTER EXTRACT ex1, EXTSEQNO 29, EXTRBA 7750160 THREAD 2
EXTRACT altered.

重啓進程後源端發起提交命名，結束該事務。
SYS@PROD2 > commit;

Commit complete.

第1000001條數據插入完成
SYS@PROD2 > select count(*) from snow.t1;

COUNT(*)
----------
   1000001

目標端第1000001條數據複製完成！
SQL> select count(*) from snow.t1;

COUNT(*)
----------
   1000001

實驗結束node