本文介紹了使用快照表和觸發器進行增量數據同步。主庫爲Oracle 11g數據庫,針對須要同步的表創建增量數據臨時表以及觸發器並經過kettle定時同步到PostgreSQL數據庫。
一、主庫建立快照表和觸發器
注意:快照表結構和源表結構必須一致!數據庫
--建立插入快照表 CREATE TABLE SPWUSER.WEB_CUSTOMER_INSERT_TMP ( ID NUMBER NOT NULL, NAME VARCHAR2(128 BYTE) NOT NULL, CREATED TIMESTAMP(6) WITH LOCAL TIME ZONE DEFAULT (current_timestamp) NOT NULL, MODIFIED TIMESTAMP(6) WITH LOCAL TIME ZONE DEFAULT (current_timestamp) NOT NULL ); --建立更新快照表 CREATE TABLE SPWUSER.WEB_CUSTOMER_UPDATE_TMP ( ID NUMBER NOT NULL, NAME VARCHAR2(128 BYTE) NOT NULL, CREATED TIMESTAMP(6) WITH LOCAL TIME ZONE DEFAULT (current_timestamp) NOT NULL, MODIFIED TIMESTAMP(6) WITH LOCAL TIME ZONE DEFAULT (current_timestamp) NOT NULL ); --建立刪除快照表 CREATE TABLE SPWUSER.WEB_CUSTOMER_DELETE_TMP ( ID NUMBER NOT NULL, NAME VARCHAR2(128 BYTE) NOT NULL, CREATED TIMESTAMP(6) WITH LOCAL TIME ZONE DEFAULT (current_timestamp) NOT NULL, MODIFIED TIMESTAMP(6) WITH LOCAL TIME ZONE DEFAULT (current_timestamp) NOT NULL ); --建立插入觸發器 CREATE OR REPLACE TRIGGER SPWUSER.WEB_CUSTOMER_INSERT_TRI AFTER INSERT ON SPWUSER.WEB_CUSTOMER FOR EACH ROW BEGIN INSERT INTO WEB_CUSTOMER_INSERT_TMP (ID, NAME, CREATED, MODIFIED) VALUES (:new.id, :new.name, :new.created, :new.modified); END; / --建立刪除觸發器 CREATE OR REPLACE TRIGGER SPWUSER.WEB_CUSTOMER_DELETE_TRI AFTER DELETE ON SPWUSER.WEB_CUSTOMER FOR EACH ROW BEGIN INSERT INTO WEB_CUSTOMER_DELETE_TMP (ID, NAME, CREATED, MODIFIED) VALUES (:old.ID, :old.NAME, :old.created, :old.modified); END; / --建立更新觸發器 CREATE OR REPLACE TRIGGER SPWUSER.WEB_CUSTOMER_UPDATE_TRI AFTER UPDATE ON SPWUSER.WEB_CUSTOMER FOR EACH ROW BEGIN INSERT INTO WEB_CUSTOMER_UPDATE_TMP (ID, NAME, CREATED, MODIFIED) VALUES (:old.ID, :old.NAME, :old.created, :old.modified); END; /
二、建立一個針對表增刪改的轉換
從kettle菜單中依次選擇「文件」-->「新建」-->「轉換」或者按ctrl+N快捷鍵建立。而後依次選擇「輸入」-->「表輸入」,「輸出」-->「插入/更新」和「輸出」-->「刪除」,執行三次,以下圖所示:
三、「表輸入」控件,查詢主庫(Oracle)的快照表ide
四、「插入/更新」控件,插入數據到從庫(PostgreSQL)的表,查詢的關鍵字要求惟一3d
五、「刪除」控件,將主庫的快照表中的數據刪除code
六、新建「做業」控件
新建做業控件,將三個轉換引用起來,總覽以下圖所示:
Insert轉換:
Update轉換:
Delete轉換:
發送郵件:blog