oracle ogg golden gate 安裝與配置

環境

 oracle12c 12.1.0.2.0
 ogg 19.1
 linux Ubuntu 14.04.3 LTS
 172.18.0.2 源數據庫地址
 172.18.0.3 目標數據庫地址

ogg根據scn同步數據，源庫零停機時間

以下是主要步驟：

1. 配置好ogg源端的mgr、抓取和傳送進程，並啓動。
2. 配置好ogg目標端的mgr、複製進程，僅啓動mgr。
3. 源端可自由進行交易，此時觀察源與目標的trail文件是否都正常。
4. 查詢源端此時的scn，並按照參數flashbask_scn進行expdp。
5. 目標端impdp導入。
6. 目標端start replicat xxx,aftercsn xxxxx

配置和使用GoldenGate的步驟

1 在源端和目標端配置數據庫支持GoldenGate 2 在源端和目標端建立和配置GoldenGate實例 3 在源端建立和配置主抽取進程(Primary Extract) 4 在源端建立和配置Data Pump進程(Secondly Extract) 5 在目標端建立和配置Replicat進程

安裝步驟

源數據庫配置步驟

開啓歸檔，--如未開啓，從新開啓須要重啓實例，在mount狀態下

Alter database archivelog;
修改oracle日誌模式：
sqlplus /nolog
conn / as sysdba
1，關閉數據庫：shutdown immediate;
2，啓動數據庫實例到mount狀態：startup mount;
3，切換數據庫日誌模式：
      alter database archivelog;（設置數據庫爲歸檔日誌模式）
      alter database noarchivelog;（設置數據庫爲非歸檔日誌模式）
4，打開數據庫：alter database open;
5，確認數據庫處於歸檔模式：archive log list;

開啓Force logging

Alter database force logging;

開啓supplemental logging

Alter database add supplemental log data;
alter system switch logfile;
alter database force logging;

設置數據庫GoldenGate參數

show parameter enable_goldengate_replication;
alter system set enable_goldengate_replication=true scope=both ; --RAC的全部實例也須要設置
alter system set enable_goldengate_replication=true scope=both sid='*';

建立OGG表空間

create tablespace ogg_tbs datafile '/u01/app/oracle/oradata/xe/ogg_tbs.dbf' size 5G AUTOEXTEND on extent management local segment space management auto ;

--如要啓用DDL功能，OGG用戶須要獨立的表空間。
建立OGG帳號

create user ggadm identified by "oracle" default tablespace ogg_tbs;
ALTER USER   GGADM QUOTA UNLIMITED ON  OGG_TBS;

給OGG帳號賦權
--源端：無DDL，不使用DBA權限的狀況： ---For 集成模式，db 12.0.1.0
--The following privileges apply to Oracle versions 11.2.0.4 or later.
--源端：Extract Classic Mode ：

grant connect, resource,CREATE SESSION to ggadm;
exec dbms_goldengate_auth.grant_admin_privilege('ggadm');
exec dbms_goldengate_auth.grant_admin_privilege(grantee=>'ggadm');
grant select any dictionary to ggadm;
commit;

配置tnsnames.ora

vi $ORACLE_HOME/network/admin/tnsnames.ora 
OGGSource =
(DESCRIPTION =
(ADDRESS = (PROTOCOL = TCP)(HOST = 192.168.56.101)(PORT = 1521))
(CONNECT_DATA =
(SERVER = DEDICATED)
(SERVICE_NAME = mydb)
)
)

檢測tnsping 和相關網絡是否聯通

OGG上傳目錄：telnet 192.168.56.101 7809
tnsping oggsource
$ sqlplus ggadm/oracle@oggsource

目錄端數據庫配置

設置ogg參數

alter system set enable_goldengate_replication=true scope=both sid='*';
create tablespace ogg_tbs datafile '/u01/app/oracle/oradata/xe/ogg_tbs.dbf' size 5G AUTOEXTEND on extent management local segment space management auto ;
create user ggadm identified by "oracle" default tablespace ogg_tbs;
ALTER USER  ggadm QUOTA UNLIMITED ON ogg_tbs;
grant 
CREATE SESSION,
ALTER SESSION,
ALTER SYSTEM,
RESOURCE,
SELECT ANY DICTIONARY,
FLASHBACK ANY TABLE,
SELECT ANY TABLE,
SELECT ANY TRANSACTION,
insert any table,
update any table,
delete any table,
drop any table,
CREATE TABLE
to ggadm;
grant SELECT on dba_clusters to ggadm;
grant SELECT on V_$DATABASE to ggadm;
grant select on sys.logmnr_buildlog to ggadm;
grant EXECUTE on DBMS_FLASHBACK to ggadm;
grant execute on DBMS_CAPTURE_ADM to ggadm;
grant execute on DBMS_STREAMS to ggadm;  
grant EXECUTE_CATALOG_ROLE to ggadm;
grant connect, resource,CREATE SESSION to ggadm;
exec dbms_goldengate_auth.grant_admin_privilege('ggadm');
exec dbms_goldengate_auth.grant_admin_privilege(grantee=>'ggadm');
grant select any dictionary to ggadm;

配置tnsnames.ora

oggtarget =
  (DESCRIPTION =
    (ADDRESS_LIST =
      (ADDRESS = (PROTOCOL = TCP)(HOST = oracle12c-target)(PORT = 1521))
    )
    (CONNECT_DATA =
      (SERVICE_NAME = xe)
    )
  )

數據庫附加日誌

附加日誌級別分爲:

• 數據庫級別的附加日誌(必須附加日誌)
• 用戶級別的附加日誌(當使用GoldenGate DDL抽取功能時要啓用該級別)
• 表級別的附加日誌(必須附加日誌，除非你啓用了用戶級別的附加日誌)

1. 數據庫級別的附加日誌
   由於GoldenGate須要抽取捕獲變動數據信息和元數據信息，這些信息須要記錄日誌，而且由於日誌老是持續增加的，這些日誌增加的信息對於GoldenGate是必須記錄的，因此必須在GoldenGate進程啓動前開啓數據庫級別的附加日誌； 檢查數據庫是否開啓附加日誌:

SELECT supplemental_log_data_min, force_logging FROM v$database;
SUPPLEME FOR
-------- ---
NO       NO

開啓數據庫級別的附加日誌:

SYS@sydb>alter database add supplemental log data;
SYS@sydb>ALTER DATABASE FORCE LOGGING;

SYS@sydb>SELECT supplemental_log_data_min, force_logging FROM v$database;

SUPPLEME FOR
-------- ---
YES      YES
Elapsed: 00:00:00.00
SYS@sydb>alter system switch logfile;

System altered.
Elapsed: 00:00:00.09

用戶級別的附加日誌

開啓用戶級別的附加日誌的管理用戶必須具備執行DBMS_CAPTURE_ADM包的權限；能夠在GoldenGate GGSCI命令行下登陸到數據庫管理用戶爲其它的用戶開啓附加日誌；
在GGSCI命令行下使用DBLOGIN命令登陸到數據庫管理用戶

GGSCI (sywu) 1> dblogin userid ogg_owner,password ogg_owner
Successfully logged into database.

爲用戶啓用附加日誌

GGSCI (sywu as ogg_owner@sydb) 2> add schematrandata scott
2015-08-24 14:56:26  INFO    OGG-01788  SCHEMATRANDATA has been added on schema scott.
2015-08-24 14:56:27  INFO    OGG-01976  SCHEMATRANDATA for scheduling columns has been added on schema scott.

取消用戶附加日誌

GGSCI (sywu as ogg_owner@sydb) 6> delete schematrandata scott
2015-08-25 20:14:49  INFO    OGG-01792  SCHEMATRANDATA has been deleted on schema scott.
2015-08-25 20:14:49  INFO    OGG-01979  SCHEMATRANDATA for scheduling columns has been deleted on schema scott.

表級別的附加日誌
在沒有啓用用戶級別的附加日誌的狀況下表級別的主鍵或惟一索引附加日誌對於GoldenGate是必須的；在某些狀況下即便你啓用了用戶級別的附加日誌，你也能夠啓用表級別的附加日誌使主鍵附加日誌替代每個在用戶級別爲GoldenGate指定的鍵；必須在表沒有主鍵或惟一索引的狀況下啓用表級別的附加日誌。
在GGSCI命令行下使用dblogin登陸到數據庫管理用戶

GGSCI (sywu) 1> dblogin userid ogg_owner,password ogg_owner
Successfully logged into database.

爲表啓用附加日誌

GGSCI (sywu as ogg_owner@sydb) 3> add trandata ogg_owner.togg
Logging of supplemental redo data enabled for table OGG_OWNER.TOGG.
TRANDATA for scheduling columns has been added on table 'OGG_OWNER.TOGG'.

注意:僅當該表有主鍵或惟一索引時使用上面的命令啓用表級別的附加日誌，若是表中沒有主鍵或惟一索引時則必須指定一個或多個或者所有列作爲主鍵，在GoldenGate這些鍵的做用是過濾重複的數據。

SYS@sydb>create table ogg_owner.togg_nokey(id,name,type,CREATED,update_date) 
      as 
 select object_id,object_name,object_type,CREATED,sysdate from dba_objects  where rownum<1001
Table created.
Elapsed: 00:00:00.11

在沒有主鍵而且沒有指定主鍵時啓用表級別附加日誌，默認全部列作爲主鍵:

GGSCI (sywu as ogg_owner@sydb) 4> add trandata ogg_owner.togg_nokey

2015-08-24 16:05:01  WARNING OGG-06439  No unique key is defined for table TOGG_NOKEY. All viable columns will be used to represent the key, but may not guarantee uniqueness. KEYCOLS may be used to define the key.

Logging of supplemental redo data enabled for table OGG_OWNER.TOGG_NOKEY.
TRANDATA for scheduling columns has been added on table 'OGG_OWNER.TOGG_NOKEY'.

在沒有主鍵而且指定主鍵列的狀況下啓用表級別附加日誌:

GGSCI (sywu as ogg_owner@sydb) 26>  add trandata ogg_owner.togg_nokey,cols (ID),nokey

2015-08-24 16:33:19  WARNING OGG-06439  No unique key is defined for table TOGG_NOKEY. All viable columns will be used to represent the key, but may not guarantee uniqueness. KEYCOLS may be used to define the key.

Logging of supplemental redo data enabled for table OGG_OWNER.TOGG_NOKEY.
TRANDATA for scheduling columns has been added on table 'OGG_OWNER.TOGG_NOKEY'.

 安裝ogg,源端與目錄端安裝方法相同

  將ogg壓縮文件放到源數據庫與目錄數據庫的/ogg目錄，這個目錄根據實際狀況設定

   

由於操做系統沒有圖形化，因此採用靜默安裝　　 先複製一份配置文件

cd /ogg/fbo_ggs_Linux_x64_shiphome/Disk1/response/
cp oggcore.rsp oggcore-setup.rsp
vim oggcore-12c.rsp
修改這幾項
INSTALL_OPTION=ORA12c
SOFTWARE_LOCATION=/u01/app/ogg
START_MANAGER=false

開始安裝　 
mkdir /u01/app/ogg
chown -R oracle:dba /u01/app/ogg
chown -R oracle:dba /ogg
su - oracle
cd /ogg/fbo_ggs_Linux_x64_shiphome/Disk1
./runInstaller -silent -responseFile /ogg/fbo_ggs_Linux_x64_shiphome/Disk1/response/oggcore-12c.rsp

安裝成功

配置.bash_profile

PATH=$PATH:$HOME/.local/bin:$HOME/bin
export PATH
export ORACLE_BASE=/data/oracle
export ORACLE_HOME=$ORACLE_BASE/product/11.2.0/db_1
export ORACLE_SID=orcl
export ORACLE_TERM=xterm
export PATH=$ORACLE_HOME/bin:/usr/sbin:$PATH
export LD_LIBRARY_PATH=$ORACLE_HOME/lib:/lib:/usr/lib
export LANG=C
export NLS_LANG=AMERICAN_AMERICA.AL32UTF8
export OGG_HOME=/data/oracle/product/ogg_src
export PATH=$OGG_HOME:$PATH
export LD_LIBRARY_PATH=$OGG_HOME:$LD_LIBRARY_PATH

更新.bash_profile

source .bash_profile
ggsci

ogg安裝成功

檢查依賴包

ldd ggsci #檢查依賴包

建立參數文件、數據文件、日誌文件等的子目錄

ggsci
create subdirs
create subdirs

Creating subdirectories under current directory /home/oracle

Parameter file                 /u01/app/ogg/dirprm: created.
Report file                    /u01/app/ogg/dirrpt: created.
Checkpoint file                /u01/app/ogg/dirchk: created.
Process status files           /u01/app/ogg/dirpcs: created.
SQL script files               /u01/app/ogg/dirsql: created.
Database definitions files     /u01/app/ogg/dirdef: created.
Extract data files             /u01/app/ogg/dirdat: created.
Temporary files                /u01/app/ogg/dirtmp: created.
Credential store files         /u01/app/ogg/dircrd: created.
Masterkey wallet files         /u01/app/ogg/dirwlt: created.
Dump files                     /u01/app/ogg/dirdmp: created.

安裝注意事項
安裝過程當中注意如下幾點：
1） 安裝過程當中若是可使用圖形界面建議使用圖形界面安裝；
2） 安裝過程當中注意配置好環境變量，使用oracle用戶安裝；
3） 若是用戶沒法提供oracle用戶安裝，建議安裝用戶必需要有oracle的全部組權限；
4） 靜默安裝時注意僅須要編輯兩個參數，MGR能夠在安裝完畢後配置，若是須要配置DATABASE_LOCATION，該參數應爲$ORACLE_HOME；
5） 建議安裝過程當中若有報錯須要及時處理，若是靜默模式安裝失敗，建議徹底卸載或者更換安裝目錄。

OGG 配置過程(源端)

建立credential store,建立OGG認證證書文件用於隱藏口令登陸

GGSCI > add credentialstore --把用戶憑證加入Credentialstore裏

把用戶憑證加入Credentialstore

alter credentialstore add user ggadm@oggsource password oracle alias oggsourceadm

測試是否能夠免密登陸

dblogin useridalias oggsourceadm
Successfully logged into database.

配置抽取表
登陸成功後，將要抽取的表加入trandata

--add trandata schema.tablename
--可經過schema.*表明添加該用戶下全部表
add trandata scott.T1,allcols

查看配置狀況

info trandata schema.tablename

建立checkpoint表

add checkpointtable ggadm.checkpoint

管理進程配置(源端mgr)

配置全局參數

edit params ./GLOBALS--注意：對於UNIX/Linux 平臺，此步驟作完後，須要到目錄下檢查,文件名必定要大寫，不能是globals
--For Performance Monitor:
--ENABLEMONITORING
CHECKPOINTTABLE ggadm.checkpoint
-- For DDL :
GGSCHEMA ggadm

配置MGR進程

edit param mgr
PORT 7809
DYNAMICPORTLIST 7810-7820
--AUTOSTART ER *
--AUTORESTART ER *,RETRIES 5,WAITMINUTES 7, RESETMINUTES 60
PURGEOLDEXTRACTS ./dirdat/*,usecheckpoints, minkeepdays 7
--------------------
--For ddl MGR
--userid ggadm, password goldengate123
--useridalias oggsourceadm
--PURGEDDLHISTORY MINKEEPDAYS 7, MAXKEEPDAYS 10
--PURGEMARKERHISTORY MINKEEPDAYS 7, MAXKEEPDAYS 10
--------------------
LAGREPORTHOURS 1
LAGINFOMINUTES 30
LAGCRITICALMINUTES 45

管理MGR進程

START MGR
info MGR
stop MGR

Extract腳本配置(源端)

管理Extract進程和Extract目錄
添加

ADD EXTRACT e1, TRANLOG,BEGIN now
ADD EXTRACT e1, TRANLOG,BEGIN now,threads 2 —RAC爲節點數
ADD EXTTRAIL ./dirdat/e1, EXTRACT e1

查看　　

info EXTTRAIL　　 
info EXTRACT e1　　

刪除

delete EXTTRAIL ./dirdat/e1, EXTRACT e1　　 
delete EXTRACT e1　　

編輯Extract參數

edit params e1
EXTRACT e1
setenv (NLS_LANG=SIMPLIFIED CHINESE_CHINA.AL32UTF8)
USERIDALIAS oggsourceadm
--USERIDALIAS pmsprod
TRANLOGOPTIONS DBLOGREADER  --rac時使用
FETCHOPTIONS NOUSESNAPSHOT
GETTRUNCATES
EXTTRAIL ./dirdat/e1
DISCARDFILE ./dirrpt/e1.dsc, APPEND, MEGABYTES 4000
WARNLONGTRANS 1H, CHECKINTERVAL 5M
CACHEMGR CACHESIZE 1024MB, CACHEDIRECTORY ./dirtmp
 
LOGALLSUPCOLS
NOCOMPRESSUPDATES
UPDATERECORDFORMAT FULL
 
REPORTCOUNT EVERY 2 SECONDS, RATE
-- table scott.*;能夠*代替全部表
table scott.T1;

管理Extract進程

GGSCI > start extract e1 –啓動e1進程
GGSCI > stop extract e1 –關閉e1進程
GGSCI > kill extract e1 --強制中止e1進程
GGSCI > info e1 --查看進程參數信息
GGSCI > stats e1 --查看進程運行信息
GGSCI > view report e1 ---查看運行報告！

問題處理

ERROR   OGG-00717  Oracle GoldenGate Capture for Oracle, EXTRACT3.prm:  Found unsupported in-memory undo record in sequence 11566, at RBA 1473040, with SCN 3330.2812655651

Possible Cause:
Supplemental logging is not enabled on the database level or the extract is positioned to a point where the supplemental logging has not been enabled i.e an attempt was made to start the extract when the supplemental logging was not enabled on the database.

Solution:

Check if Supplemental logging is enabled

SQL> SELECT SUPPLEMENTAL_LOG_DATA_MIN FROM V$DATABASE;

SUPPLEME
--------
NO

Enable Supplemental Logging

SQL> ALTER DATABASE ADD SUPPLEMENTAL LOG DATA;

Database altered.

Switch Logfile

SQL> ALTER SYSTEM SWITCH LOGFILE;

System altered.

And attempt to start the extract with begin now option

GGSCI (sev274) 2> alter extract EXTRACT3, begin now
EXTRACT altered.


GGSCI (sev274) 3> start EXTRACT3

Sending START request to MANAGER ...
EXTRACT EXTRACT3 starting


GGSCI (sev274) 4> info EXTRACT3

EXTRACT    EXTRACT3  Last Started 2015-11-09 16:29   Status RUNNING
Checkpoint Lag       00:00:00 (updated 00:00:05 ago)
Log Read Checkpoint  Oracle Redo Logs
                     2015-11-09 16:29:20  Seqno 0, RBA 0
                     SCN 0.0 (0)

PUMP腳本配置(源端)

管理pump進程和pump目錄 添加

ADD EXTRACT p1 EXTTRAILSOURCE ./dirdat/e1, BEGIN NOW
ADD RMTTRAIL /u01/app/ogg/dirdat/tx, EXTRACT p1

查看

GGSCI>info RMTTRAIL
GGSCI>info EXTRACT p1

刪除

GGSCI>delete RMTTRAIL /u01/app/ogg/dirdat/tx, EXTRACT p1
GGSCI> delete EXTRACT p1

編輯PUMP參數 edit params p1;

EXTRACT p1
--setenv (NLS_LANG="SIMPLIFIED CHINESE_CHINA.AL32UTF8")
--For ogg 12.2 DYNAMICRESOLUTION is deprecated
--DYNAMICRESOLUTION
--USERIDALIAS ggadm
--USERIDALIAS oggsourceadm
discardfile ./dirrpt/p1.dsc,append,megabytes 4000
rmthost oracle12c-target  mgrport 7809
rmttrail /u01/app/ogg/dirdat/tx
 
PASSTHRU 
TABLE SCOTT.*;

管理PUMP進程(源端)

GGSCI > start extract p1 –啓動e1進程
GGSCI > stop extract p1 –關閉e1進程
GGSCI > kill extract p1 --強制中止e1進程
GGSCI > info p1 --查看進程參數信息
GGSCI > stats p1 --查看進程運行信息
GGSCI > view report p1 ---查看運行報告

目標端配置證書（目標） 建立OGG認證證書文件用於隱藏口令登陸

GGSCI > add credentialstore
GGSCI > alter credentialstore add user ggadm@oggtarget,password "oracle" alias ggadmtarget

測試登陸

GGSCI (b28a89583b2e) 3> dblogin useridalias ggadmtarget
Successfully logged into database.

配置globals參數（目標）

--For Performance Monitor:
ENABLEMONITORING
CHECKPOINTTABLE ggadm.checkpoint
-- For DDL :
GGSCHEMA ggadm

添加檢查點表（目標）

add checkpointtable ggadm.checkpoint

添加replicat應用進程及配置參數

add replicat rep,exttrail ./dirdat/tx checkpointtable ggadm.checkpoint
edit params rep
replicat rep
setenv (NLS_LANG="SIMPLIFIED CHINESE_CHINA.AL32UTF8")
assumetargetdefs
discardfile ./dirout/RAMER.DSC, purge
USERIDALIAS ggadmtarget
map SCOTT.*, target SCOTT.*;

replicat

alter REPLICAT rep1, begin now

配置mgr進程(目標)

GGSCI>edit param mgr  
PORT 7809  
DYNAMICPORTLIST 7810-7820 
管理mgr進程命令
GGSCI> START MGR  
GGSCI> info MGR  
GGSCI> stop MGR

啓動,查看replicat

start replicat rep,aftercsn 402721**

ddl支持

oracle 12c如下須要執行如下語句,重啓數據庫才能生效,oracle 12c不須要執行

ALTER SYSTEM SET recyclebin=off SCOPE=SPFILE;

在goldengate目錄下，執行'maker_setup.sql and ddl_setup.sql.'腳本
建立GGS_GGSUSER_ROLE角色，並受權給goldengate schema

@role_setup.sql

啓用ddl支持

@ddl_enable.sql
@ddl_pin ggadm

相關連接

https://www.cnblogs.com/lanston/p/ogg_configAndUse.html
https://www.cnblogs.com/VoiceOfDreams/p/8576973.html
https://blog.csdn.net/anzhen0429/article/details/78637122
https://blog.csdn.net/orabing/article/details/79932093
https://blog.csdn.net/moscot_wu/article/details/78957572
https://blog.csdn.net/tangkailyy/article/details/48026205?utm_source=blogxgwz5
https://blog.csdn.net/orabing/article/details/79932093
http://www.cnblogs.com/junnor/p/4307228.html
https://www.cnblogs.com/kkterry/p/3816793.html
http://www.cnxdug.org/?p=1961
http://blog.itpub.net/7204674/viewspace-713290/
https://docs.oracle.com/goldengate/1212/gg-winux/GIORA.pdf
https://docs.oracle.com/goldengate/1212/gg-winux/GIORA/install.htm#GIORA976
https://blog.yannickjaquier.com/oracle/goldengate-12c-tutorial.html
https://www.cnblogs.com/chinas/p/7339465.html