達夢主備MPP

時間 2021-08-12

原文原文鏈接

MPP分佈式，相似分庫分表，數據分散到幾個節點數據庫，登陸本地節點只能看到本節點數據庫的數據，登陸全局數據庫則能夠看到全部節點數據庫的數據，表數據安裝必定規則分佈。sql

架構圖數據庫

DM MPP 中的每個 DM 數據庫服務器實例做爲一個執行節點，簡稱 EP。客戶端可鏈接任意一個 EP 節點進行操做，全部 EP 對客戶來講都是對等的。
DM MPP 系統內每一個 EP 只負責自身部分數據的讀寫，執行計劃在全部 EP 並行執行，能充分利用各 EP 的計算能力及發揮各 EP 獨立存儲的優點。數據只在必要時經過 DM 的高速郵件 MAL 系統在 EP 間傳遞。當通訊代價佔總體執行代價的比例較小時，更能體現大規模並行處理的優點，隨着系統規模的擴大，並行支路越多，優點越明顯。服務器

搭建部署
主備MPP首先是兩套主備集羣，在集羣的基礎上作的MPP架構

一、準備工做分佈式

首先要有兩套主備集羣。ide

建立數據庫：新建數據庫作一次重啓操做
主機102
dminit extent_size=32 page_size=32 CASE_SENSITIVE=1 CHARSET=1 INSTANCE_NAME=MPP_102 DB_NAME=MPP_102 PORT_NUM=5237 path=/data/dm
dminit extent_size=32 page_size=32 CASE_SENSITIVE=1 CHARSET=1 INSTANCE_NAME=MPP_103_STA DB_NAME=MPP_103_STA PORT_NUM=5217 path=/data/dm工具

主機103
dminit extent_size=32 page_size=32 CASE_SENSITIVE=1 CHARSET=1 INSTANCE_NAME=MPP_103 DB_NAME=MPP_103 PORT_NUM=5217 path=/data/dm
dminit extent_size=32 page_size=32 CASE_SENSITIVE=1 CHARSET=1 INSTANCE_NAME=MPP_102_STA DB_NAME=MPP_102_STA PORT_NUM=5237 path=/data/dmui

相應主備庫數據一致：
[dmdba@v3 ~]$ dmrman CTLSTMT="BACKUP DATABASE '/data/dm/MPP_103/dm.ini' FULL TO BACKUP_FILE1 BACKUPSET '/data/dm/bak/103'"
[dmdba@v3 ~]$scp /data/dm/bak/103/* v2:/data/dm/bak/103代理

[dmdba@v2 ~]$ dmrman CTLSTMT="RESTORE DATABASE '/data/dm/MPP_103_STA/dm.ini' FROM BACKUPSET '/data/dm/bak/103'"日誌

[dmdba@v2 ~]$ dmrman CTLSTMT="RECOVER DATABASE '/data/dm/MPP_103_STA/dm.ini' FROM BACKUPSET '/data/dm/bak/103'"

[dmdba@v2 ~]$ dmrman CTLSTMT="RECOVER DATABASE '/data/dm/MPP_103_STA/dm.ini' UPDATE DB_MAGIC"

二、修改參數文件
主庫102
[dmdba@v2 ~]$ cat /data/dm/MPP_102/dm.ini |grep 'INSTANCE_NAME\|PORT_NUM\|MAL_INI\|MPP_INI\|ARCH_INI'
INSTANCE_NAME = MPP_102 #Instance name
PORT_NUM = 5237 #Port number on which the database server will listen
MAL_INI = 1 #dmmal.ini
ARCH_INI = 1 #dmarch.ini
MPP_INI = 1 #dmmpp.ini
備庫102
[dmdba@v3 bak]$ cat /data/dm/MPP_102_STA/dm.ini |grep 'INSTANCE_NAME\|PORT_NUM\|MAL_INI\|MPP_INI\|ARCH_INI'
INSTANCE_NAME = MPP_102_STA #Instance name
PORT_NUM = 5237 #Port number on which the database server will listen
MAL_INI = 1 #dmmal.ini
ARCH_INI = 1 #dmarch.ini
MPP_INI = 1 #dmmpp.ini

主庫103
[dmdba@v3 bak]$ cat /data/dm/MPP_103/dm.ini |grep 'INSTANCE_NAME\|PORT_NUM\|MAL_INI\|MPP_INI\|ARCH_INI'
INSTANCE_NAME = MPP_103 #Instance name
PORT_NUM = 5217 #Port number on which the database server will listen
MAL_INI = 1 #dmmal.ini
ARCH_INI = 1 #dmarch.ini
MPP_INI = 1 #dmmpp.ini

備庫103
[dmdba@v2 ~]$ cat /data/dm/MPP_103_STA/dm.ini |grep 'INSTANCE_NAME\|PORT_NUM\|MAL_INI\|MPP_INI\|ARCH_INI'
INSTANCE_NAME = MPP_103_STA #Instance name
PORT_NUM = 5217 #Port number on which the database server will listen
MAL_INI = 1 #dmmal.ini
ARCH_INI = 1 #dmarch.ini
MPP_INI = 1 #dmmpp.ini

從庫也要修改mpp_ini否則會報錯

三、修改歸檔日誌
主機102
[dmdba@v2 ~]$ cat /data/dm/MPP_102/dmarch.ini
[ARCHIVE_REALTIME]
ARCH_TYPE = REALTIME #實時歸檔類型
ARCH_DEST =MPP_102_STA #實時歸檔目標實例名

[ARCHIVE_LOCAL1]
ARCH_TYPE = LOCAL
ARCH_DEST = /data/dm/MPP_102/arch
ARCH_FILE_SIZE = 128
ARCH_SPACE_LIMIT = 1024

[dmdba@v2 ~]$ cat /data/dm/MPP_103_STA/dmarch.ini
[ARCHIVE_REALTIME]
ARCH_TYPE = REALTIME #實時歸檔類型
ARCH_DEST =MPP_103 #實時歸檔目標實例名

[ARCHIVE_LOCAL1]
ARCH_TYPE = LOCAL
ARCH_DEST = /data/dm/MPP_103_STA/arch
ARCH_FILE_SIZE = 128
ARCH_SPACE_LIMIT = 1024

主機103
[dmdba@v3 bak]$ cat /data/dm/MPP_103/dmarch.ini
[ARCHIVE_REALTIME]
ARCH_TYPE = REALTIME #實時歸檔類型
ARCH_DEST =MPP_103_STA #實時歸檔目標實例名

[ARCHIVE_LOCAL1]
ARCH_TYPE = LOCAL
ARCH_DEST = /data/dm/MPP_103/arch
ARCH_FILE_SIZE = 128
ARCH_SPACE_LIMIT = 1024

[dmdba@v3 bak]$ cat /data/dm/MPP_102_STA/dmarch.ini
[ARCHIVE_REALTIME]
ARCH_TYPE = REALTIME #實時歸檔類型
ARCH_DEST =MPP_102 #實時歸檔目標實例名

[ARCHIVE_LOCAL1]
ARCH_TYPE = LOCAL
ARCH_DEST = /data/dm/MPP_102_STA/arch
ARCH_FILE_SIZE = 128
ARCH_SPACE_LIMIT = 1024

四、修改dmmal.ini
dmmal.ini主備庫一致

[dmdba@v2 ~]$ cat /data/dm/MPP_103_STA/dmmal.ini
MAL_CHECK_INTERVAL = 5
MAL_CONN_FAIL_INTERVAL = 5
[MAL_INST1]
MAL_INST_NAME = MPP_102
MAL_HOST = 10.12.11.102
MAL_PORT = 5235
MAL_INST_HOST = 192.168.56.102
MAL_INST_PORT = 5237
MAL_DW_PORT = 5233
MAL_INST_DW_PORT = 5234
[MAL_INST2]
MAL_INST_NAME = MPP_102_STA
MAL_HOST = 10.12.11.103
MAL_PORT = 5235
MAL_INST_HOST = 192.168.56.103
MAL_INST_PORT = 5237
MAL_DW_PORT = 5233
MAL_INST_DW_PORT = 5234

[MAL_INST3]
MAL_INST_NAME = MPP_103
MAL_HOST = 10.12.11.103
MAL_PORT = 5215
MAL_INST_HOST = 192.168.56.103
MAL_INST_PORT = 5217
MAL_DW_PORT = 5213
MAL_INST_DW_PORT = 5214
[MAL_INST4]
MAL_INST_NAME = MPP_103_STA
MAL_HOST = 10.12.11.102
MAL_PORT = 5215
MAL_INST_HOST = 192.168.56.102
MAL_INST_PORT = 5217
MAL_DW_PORT = 5213
MAL_INST_DW_PORT = 5214

五、守護進程watcher
每臺主機上只能配置一個守護進程，所以將主機上的主備守護進程合併。
主機102
[dmdba@v2 ~]$ cat /data/dm/MPP_102/dmwatcher.ini
[MPP_102]
DW_TYPE = GLOBAL
DW_MODE = AUTO
DW_ERROR_TIME = 10
INST_RECOVER_TIME = 60
INST_ERROR_TIME = 10
INST_OGUID = 45330
INST_INI = /data/dm/MPP_102/dm.ini
INST_AUTO_RESTART = 1
INST_STARTUP_CMD = /data/dmdbms/bin/dmserver
RLOG_SEND_THRESHOLD = 0
RLOG_APPLY_THRESHOLD = 0

[MPP_103]
DW_TYPE = GLOBAL
DW_MODE = AUTO
DW_ERROR_TIME = 10
INST_RECOVER_TIME = 60
INST_ERROR_TIME = 10
INST_OGUID = 45331
INST_INI = /data/dm/MPP_103_STA/dm.ini
INST_AUTO_RESTART = 1
INST_STARTUP_CMD = /data/dmdbms/bin/dmserver
RLOG_SEND_THRESHOLD = 0
RLOG_APPLY_THRESHOLD = 0

主機103
[dmdba@v3 bak]$ cat /data/dm/MPP_103/dmwatcher.ini
[MPP_103]
DW_TYPE = GLOBAL
DW_MODE = AUTO
DW_ERROR_TIME = 10
INST_RECOVER_TIME = 60
INST_ERROR_TIME = 10
INST_OGUID = 45331
INST_INI = /data/dm/MPP_103/dm.ini
INST_AUTO_RESTART = 1
INST_STARTUP_CMD = /data/dmdbms/bin/dmserver
RLOG_SEND_THRESHOLD = 0
RLOG_APPLY_THRESHOLD = 0

[MPP_102]
DW_TYPE = GLOBAL
DW_MODE = AUTO
DW_ERROR_TIME = 10
INST_RECOVER_TIME = 60
INST_ERROR_TIME = 10
INST_OGUID = 45330
INST_INI = /data/dm/MPP_102_STA/dm.ini
INST_AUTO_RESTART = 1
INST_STARTUP_CMD = /data/dmdbms/bin/dmserver

六、監視器
監視器配置在第三臺主機上，此處配置在主機103上

[dmdba@v3 bak]$ cat /data/dm/MPP_103/dmmonitor.ini
MON_DW_CONFIRM = 0 =====MON_DW_CONFIRM：0是普通監視器，1是確認監視器
MON_LOG_PATH = /data/dmdbms/log
MON_LOG_INTERVAL = 60
MON_LOG_FILE_SIZE = 32
MON_LOG_SPACE_LIMIT = 0
[MPP_102] ====與守護進程dmwatcher.ini一致
MON_INST_OGUID = 45330
MON_DW_IP = 10.12.11.102:5233
MON_DW_IP = 10.12.11.103:5233
[MPP_103] ====與守護進程dmwatcher.ini一致
MON_INST_OGUID = 45331
MON_DW_IP = 10.12.11.102:5213
MON_DW_IP = 10.12.11.103:5213

七、mpp.ini配置
[dmdba@v3 bak]$ cat /data/dm/MPP_103/dmmpp.ini
[service_name1]
mpp_seq_no = 0
mpp_inst_name = MPP_102
[service_name2]
mpp_seq_no = 1
mpp_inst_name = MPP_103
編輯dmmpp.ini文件生成dmmpp.ctl
[dmdba@v3 bak]$ dmctlcvt TYPE=2 SRC=/data/dm/MPP_103/dmmpp.ini DEST=/data/dm/MPP_103/dmmpp.ctl
複製到各主備數據庫目錄下

八、啓動數據庫
啓動數據庫到mount狀態，啓動順序與主備集羣一致
主備102
[dmdba@v2 ~]$ dmserver /data/dm/MPP_102/dm.ini mount

[dmdba@v3 ~]$ dmserver /data/dm/MPP_102_STA/dm.ini mount
主備103
[dmdba@v3 ~]$ dmserver /data/dm/MPP_103/dm.ini mount

[dmdba@v2 ~]$ dmserver /data/dm/MPP_103_STA/dm.ini mount
九、修改數據庫角色和oguid

此時數據庫已是MPP架構在登錄時，要注意：

修改時，主備庫先disql登陸後在修改

v102主庫
SQL> SP_SET_PARA_VALUE(1, 'ALTER_MODE_STATUS', 1);
SQL> sp_set_oguid(45330);
SQL> alter database primary;
SQL> SP_SET_PARA_VALUE(1, 'ALTER_MODE_STATUS', 0);
v102備庫
SQL> SP_SET_PARA_VALUE(1, 'ALTER_MODE_STATUS', 1);
SQL> sp_set_oguid(45330);
SQL> alter database standby;
SQL> SP_SET_PARA_VALUE(1, 'ALTER_MODE_STATUS', 0);

v103主庫
SQL> SP_SET_PARA_VALUE(1, 'ALTER_MODE_STATUS', 1);
SQL> sp_set_oguid(45331);
SQL> alter database primary;
SQL> SP_SET_PARA_VALUE(1, 'ALTER_MODE_STATUS', 0);
v103備庫
SQL> SP_SET_PARA_VALUE(1, 'ALTER_MODE_STATUS', 1);
SQL> sp_set_oguid(45331);
SQL> alter database standby;
SQL> SP_SET_PARA_VALUE(1, 'ALTER_MODE_STATUS', 0);

[dmdba@v2 dm]$ disql SYSDBA/SYSDBA@192.168.56.102:5217

[dmdba@v2 dm]$ disql SYSDBA/SYSDBA*local@192.168.56.102:5237

九、啓動守護進程和監視器
啓動守護
[dmdba@v2 ~]$ dmwatcher /data/dm/MPP_102/dmwatcher.ini
DMWATCHER[4.0] V8
DMWATCHER[4.0] IS READY

[dmdba@v3 ~]$ dmwatcher /data/dm/MPP_103/dmwatcher.ini
DMWATCHER[4.0] V8
DMWATCHER[4.0] IS READY

守護會將數據庫由mount---》open

啓動監視器：
[dmdba@v3 ~]$ dmmonitor /data/dm/MPP_103/dmmonitor.ini
[monitor] 2021-02-20 14:27:06: DMMONITOR[4.0] V8
[monitor] 2021-02-20 14:27:06: DMMONITOR[4.0] IS READY.
報錯：

十、開機自啓動
主機102
[root@v2 ~]# /data/dmdbms/script/root/dm_service_installer.sh -t dmserver -dm_ini /data/dm/MPP_102/dm.ini -p MPP_102
Created symlink from /etc/systemd/system/multi-user.target.wants/DmServiceMPP_102.service to /usr/lib/systemd/system/DmServiceMPP_102.service.
建立服務(DmServiceMPP_102)完成
[root@v2 ~]# /data/dmdbms/script/root/dm_service_installer.sh -t dmserver -dm_ini /data/dm/MPP_103_STA/dm.ini -p MPP_103_STA
Created symlink from /etc/systemd/system/multi-user.target.wants/DmServiceMPP_103_STA.service to /usr/lib/systemd/system/DmServiceMPP_103_STA.service.
建立服務(DmServiceMPP_103_STA)完成
[root@v2 ~]# /data/dmdbms/script/root/dm_service_installer.sh -t dmwatcher -watcher_ini /data/dm/MPP_102/dmwatcher.ini -p MPP_102
Created symlink from /etc/systemd/system/multi-user.target.wants/DmWatcherServiceMPP_102.service to /usr/lib/systemd/system/DmWatcherServiceMPP_102.service.
建立服務(DmWatcherServiceMPP_102)完成

主機103
[root@v3 ~]# /data/dmdbms/script/root/dm_service_installer.sh -t dmserver -dm_ini /data/dm/MPP_103/dm.ini -p MPP_103
Created symlink from /etc/systemd/system/multi-user.target.wants/DmServiceMPP_103.service to /usr/lib/systemd/system/DmServiceMPP_103.service.
建立服務(DmServiceMPP_103)完成
[root@v3 ~]# /data/dmdbms/script/root/dm_service_installer.sh -t dmserver -dm_ini /data/dm/MPP_102_STA/dm.ini -p MPP_102_STA
Created symlink from /etc/systemd/system/multi-user.target.wants/DmServiceMPP_102_STA.service to /usr/lib/systemd/system/DmServiceMPP_102_STA.service.
建立服務(DmServiceMPP_102_STA)完成
[root@v3 ~]# /data/dmdbms/script/root/dm_service_installer.sh -t dmwatcher -watcher_ini /data/dm/MPP_103/dmwatcher.ini -p MPP_103
Created symlink from /etc/systemd/system/multi-user.target.wants/DmWatcherServiceMPP_103.service to /usr/lib/systemd/system/DmWatcherServiceMPP_103.service.
建立服務(DmWatcherServiceMPP_103)完成
[root@v3 ~]# /data/dmdbms/script/root/dm_service_installer.sh -t dmmonitor -monitor_ini /data/dm/MPP_103/dmmonitor.ini -p MPP_103
Created symlink from /etc/systemd/system/multi-user.target.wants/DmMonitorServiceMPP_103.service to /usr/lib/systemd/system/DmMonitorServiceMPP_103.service.
建立服務(DmMonitorServiceMPP_103)完成

客戶端訪問
本地登陸：

[dmdba@v3 ~]$ disql SYSDBA/SYSDBA*LOCAL@192.168.56.103:5217

全局登陸

disql SYSDBA/SYSDBA@192.168.56.103:5217

manger工具：

也能夠使用DCP代理訪問，此處不做詳細介紹。

MPP的使用
經過管理工具全局登陸

CREATE TABLE T_HASH1(C1 INT, C2 CHAR(10))DISTRIBUTED BY HASH (C1);

insert into SYSDBA.T_HASH VALUES (1,'q');
insert into SYSDBA.T_HASH VALUES (2,'w');
commit;

數據根據表的hash分佈在不一樣的數據庫中，可是全局查看倒是正常的。

本地登陸：

全局登陸