mysql數據庫同步系統otter部署實踐（中國與歐洲同步）

時間 2019-11-15

標籤 mysql 數據庫同步系統 otter 部署實踐歐洲欄目 MySQL 简体版

原文原文鏈接

[本博客已經遷移到http://blog.csdn.net/qq_28106715/article/details/71405507]node

[如下內容再也不維護]mysql

otter的介紹就不說了，本身去看官網https://github.com/alibaba/otter/wikigit

本系統中，中國的服務器部署在阿里雲上，歐洲服務器部署在亞馬遜上，因爲阿里雲的網絡不支持外網迴環訪問（即在本機上向本機的外網地址發送數據），因此須要將node與manager部署在不一樣的主機上。若是隻是在虛擬機中調試，不少服務均可以放在一臺主機中。github

1. 阿里雲服務器

1. mysql服務，外網ip 120.x.x.1, 內網ip 10.x.x.1web

2. manager服務，本系統與mysql安裝在同一臺服務器上sql

3. node服務，外網ip 120.x.x.2, 內網ip 10.x.x.2數據庫

4. zookeeper服務，外網ip 120.x.x.3, 內網ip 10.x.x.3apache

2. 亞馬遜服務器

1. mysql服務，外網ip 59.x.x.1, 內網ip 172.x.x.1vim

2. node服務，外網ip 59.x.x.2, 內網ip 172.x.x.2瀏覽器

3. zookeeper服務，外網ip 59.x.x.2, 內網ip 172.x.x.3

3. 下載otter和zookeeper

已經編譯好的壓縮包

https://github.com/alibaba/otter/releases

下載manager系統表

https://raw.github.com/alibaba/otter/master/manager/deployer/src/main/resources/sql/otter-manager-schema.sql

下載ddl同步策略表

https://raw.githubusercontent.com/alibaba/otter/master/node/deployer/src/main/resources/sql/otter-system-ddl-mysql.sql

下載zookeeper

http://www.apache.org/dyn/closer.cgi/zookeeper/

4. 配置阿里雲mysql

修改兩臺服務器的mysql配置, /etc/mysql/my.cnf (5.7版本的mysql的配置文件應該是/etc/mysql/mysql.conf.d/mysqld.cnf)

注意： server-id須要和manager上配置的node的id一致

[mysqld]

character-set-server=utf8
collation-server=utf8_bin #設置mysql的字符集

#~~bind-address = 127.0.0.1~~ 這一行要註釋掉

server-id                  = 1
log_bin                    = /data/lib/mysql/mysql-bin.log
expire_logs_days      = 5
max_binlog_size       = 200M
binlog_do_db           = xxx
binlog_format          = ROW

重啓mysql

5. 配置亞馬遜上的mysql

只是server-id不同

[mysqld]

character-set-server=utf8
collation-server=utf8_general_ci #設置mysql的字符集

#~~bind-address = 127.0.0.1~~ 這一行要註釋掉

server-id                  = 2
log_bin                    = /data/lib/mysql/mysql-bin.log
expire_logs_days      = 5
max_binlog_size       = 200M
binlog_do_db           = xxx
binlog_format          = ROW

重啓mysql

6. 導入系統表和otter用戶

注意修改密碼

mysql>source otter-manager-schema.sql
mysql>source otter-system-ddl-mysql.sql
mysql>CREATE USER canal IDENTIFIED BY 'canal'; 
mysql>GRANT SELECT, REPLICATION SLAVE, REPLICATION CLIENT ON *.* TO 'canal'@'%';
mysql>GRANT ALL PRIVILEGES ON *.* TO 'canal'@'%';
mysql>FLUSH PRIVILEGES;

若是出現提示

ERROR 1067 (42000): Invalid default value for 'GMT_CREATE'

則須要將sql_mode中的NO_ZERO_DATE去掉

select @@sql_mode

+-------------------------------------------------------------------------------------------------------------------------------------------+
| @@sql_mode                                                                                                                                |
+-------------------------------------------------------------------------------------------------------------------------------------------+
| ONLY_FULL_GROUP_BY,STRICT_TRANS_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,NO_AUTO_CREATE_USER,NO_ENGINE_SUBSTITUTION |
+-------------------------------------------------------------------------------------------------------------------------------------------+

set sql_mode='STRICT_TRANS_TABLES,NO_ZERO_IN_DATE,ERROR_FOR_DIVISION_BY_ZERO,NO_AUTO_CREATE_USER,NO_ENGINE_SUBSTITUTION';

7. 安裝aria2和zookeeper

在須要運行node服務器上安裝aria2，它是一個下載工具， otter使用它下載了什麼文件，我也不知道

apt-get install aria2

在分別在阿里雲和亞馬遜的兩臺服務器上安裝zookeeper和運行zookeeper, 這個過程很簡單

cd zookeeper-3.4.9/conf/
cp zoo_sample.cfg zoo.cfg
cd ../bin/
./zkServer.sh start

8. 配置manager

cd manager/conf/
vim otter.properties

## otter manager domain name
otter.domainName = 120.x.x.1 #修改成當前服務器的公網地址
## otter manager http port
otter.port = 8080
## jetty web config xml
otter.jetty = jetty.xml

## otter manager database config
otter.database.driver.class.name = com.mysql.jdbc.Driver
otter.database.driver.url = jdbc:mysql://127.0.0.1:3306/otter #otter所屬的mysql服務器的地址
otter.database.driver.username = canal
otter.database.driver.password = canal #須要修改成實際的密碼

## otter communication port
otter.communication.manager.port = 1099

## otter communication pool size
otter.communication.pool.size = 10

## default zookeeper address
otter.zookeeper.cluster.default = 120.x.x.3:2181 #修改成阿里雲上安裝的zookeeper
## default zookeeper sesstion timeout = 60s

...