OGG 18.1 for mysql遠程捕獲測試

Ogg18.1 remote capture要求mysql爲5.7版本，只能從linux遠程捕獲mysql on windows or linux，且不支持DDL捕獲。支持遠程mysql爲community/commerical版本。

 

源端數據庫配置

OGG登陸DB的用戶及密碼： ogg/Welcome!23

 

確保源庫有打開日誌

修改my.cnf，打開日誌存放路徑，並設置格式

[mysqld]

log-bin=/var/log/mysql/mysql-bin.log

確保運行mysql的用戶有權訪問這個目錄.

重啓mysqld以後，能夠看到生成的日誌

 

mysql中查看

mysql>show variables like 'log_%';

mysql>  show master logs;

+------------------------+-----------+

| Log_name               | File_size |

+------------------------+-----------+

| mysqlcs-mysql-1.000001 |       177 |

| mysqlcs-mysql-1.000002 |   7713925 |

+------------------------+-----------+

2 rows in set (0.00 sec)

 

查看日誌內容

mysqlbinlog   mysqlcs-mysql-1.000001

 

 

源DB準備

建立testdb並受權

create database testdb;

CREATE USER 'ogg'@'192.168.1.53' IDENTIFIED BY 'Welcome!23';

GRANT ALL PRIVILEGES ON *.* TO 'ogg'@'192.168.1.53' WITH GRANT OPTION;

FLUSH PRIVILEGES;

 

建立測試表

create table t1 (id int primary key, name varchar(50));

create table t2_tgt (id int primary key, name varchar(50));

create table t3 (id int, name varchar(50));

 

OGG抽取配置

遠程捕獲不支持DDL同步。

 

確認OGG能遠程鏈接到mysql

GGSCI > dblogin sourcedb testdb@192.168.1.52, userid ogg, password Welcome!23

Successfully logged into database.

 

GGSCI (DBLOGIN as root) 10>

 

Mgr.prm

Port 7809

 

 

ex1.prm

extract ex1

sourcedb testdb@192.168.1.52, userid ogg, password Welcome!23

 

TRANLOGOPTIONS ALTLOGDEST REMOTE

exttrail ./dirdat/ea

 

--ddl include mapped

table testdb.*;

 

GGSCI (DBLOGIN as root) 10> add ext ex1, tranlog, begin now

EXTRACT added.

 

GGSCI (DBLOGIN as root) 11> add exttrail ./dirdat/ea, ext ex1

EXTTRAIL added.

 

確認進程正常啓動

Program     Status      Group       Lag at Chkpt  Time Since Chkpt

 

MANAGER     RUNNING                                          

EXTRACT     RUNNING     EX1         10:43:02      00:00:07   

 

 

GGSCI (dbcs12c) 2> info ex1

 

EXTRACT    EX1       Last Started 2018-11-05 21:25  Status RUNNING

Checkpoint Lag       10:43:02 (updated 00:00:09 ago)

Process ID           19857

VAM Read Checkpoint  2018-11-05 23:17:39.000000

  Log Number: 2

    Record Offset: 7713925

已經能夠讀取日誌

測試

Mysql插入數據

mysql> insert into testdb.t1 values(1,'bcs');

Query OK, 1 row affected (0.01 sec)

 

 

 

源DB上插入、更新、刪除記錄，抽取進程可正常捕獲：

GGSCI (dbcs12c) 3> stats ex1, total

 

Sending STATS request to EXTRACT EX1 ...

 

Start of Statistics at 2018-11-06 10:01:27.

 

Output to ./dirdat/ea:

 

Extracting from testdb.t1 to testdb.t1:

 

*** Total statistics since 2018-11-05 21:35:34 ***

        Total inserts                                      2.00

        Total updates                                      4.00

        Total deletes                                      1.00

        Total discards                                     0.00

        Total operations                                   7.00

 

Extracting from testdb.t3 to testdb.t3:

 

*** Total statistics since 2018-11-05 21:35:34 ***

        Total inserts                                      1.00

        Total updates                                      2.00

        Total deletes                                      0.00

        Total discards                                     0.00

        Total operations                                   3.00

 

End of Statistics.

 

查看隊列文件，已經有捕獲到新增數據

能夠看到，經過遠程捕獲配置，goldengate仍然能夠從mysql db中獲取增量數據。

 

經過以上的配置，能夠在一臺linux 節點上，安裝多個goldengate的介質，好比 OGG for oracle, ogg for mysql, ogg for bigdata等，從而實現相似HUB的機制，將數據的抽取和投遞集中在一個節點上進行管理，下降數據集中和數據複製的運維複雜性。