MariaDB集羣Galera Cluster的研究與測試

MariaDB集羣Galera Cluster的研究與測試

Galera Cluster是MariaDB的一個雙活多主集羣，其可使得MariDB的全部節點保持同步，Galera爲MariaDB提供了同步複製（相對於原生的異步複製），所以其能夠保證HA，且其當前僅支持XtraDB/InnoDB存儲引擎（擴展支持MyISAM），而且只可在Linux下使用。
Galera Cluster擁有如下特性：

• 真正的多主架構，任何節點均可以進行讀寫
• 同步複製，各節點間無延遲且節點宕機不會致使數據丟失
• 緊密耦合，全部節點均保持相同狀態，節點間無不一樣數據
• 無需主從切換操做或使用VIP
• 熱Standby，在Failover過程當中無停機時間（因爲不須要Failover）
• 自動節點配置，無需手工備份當前數據庫並拷貝至新節點
• 支持InnoDB存儲引擎
• 對應於透明，無需更改應用或是進行極小的更改
• 無需進行讀寫分離

Galera使用基於認證的複製，其流程以下：

其主要思想是在不出現衝突的背景下事務正常執行並持續到commit爲止；當客戶端發起commit命令時（此時仍然沒有發生真正的commit），全部本事務內對數據庫的改動與改動數據行的主鍵都會被蒐集到一個寫入集（writeset）中，該寫入集隨後會被複制到其餘節點，該寫入集會在每一個節點上使用搜集到的主鍵進行確認性認證測試（包括被「提交」事務的當前節點）來判斷該寫入集是否能夠被應用。若是認證測試失敗，寫入集會被丟棄而且原始事務會被回滾，若是認證成功，事務會被提交而且寫入集會被在剩餘節點進行應用。
以上的認證測試在Galera集羣中的實現取決於全局事務順序，每一個事務在複製期間都會被指派一個全局順序序列；當一個事務到達提交點時，該事務會知道當前與該事務不衝突的最新已提交事務的順序序號，在這兩個事務的全局順序序列之間的間隔是不肯定區域，在該區域間的事務相互是「看不到」對方的影響的，但全部在這間隔之間的惟物都會被進行主鍵衝突檢測（若是發現衝突認證測試就會失敗）。
如下演示如何配置Galera Cluster：
一、首先在各節點上安裝MariaDB Galera：

 

 

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21

[root@nd1 ~]# ls -lhtr total 123M -rw-r--r-- 1 root root 340K Mar 27 21:30 socat-1.7.2.4-1.el6.rf.x86_64.rpm -rwxrwxrwx 1 root root 5.8M Jun  7 18:59 MariaDB-5.5.38-centos6-x86_64-devel.rpm -rwxrwxrwx 1 root root 2.7M Jun  7 18:59 MariaDB-5.5.38-centos6-x86_64-compat.rpm -rwxrwxrwx 1 root root  24K Jun  7 18:59 MariaDB-5.5.38-centos6-x86_64-common.rpm -rwxrwxrwx 1 root root  11M Jun  7 18:59 MariaDB-5.5.38-centos6-x86_64-client.rpm -rwxrwxrwx 1 root root 1.2M Jun  7 18:59 MariaDB-5.5.38-centos6-x86_64-shared.rpm -rwxrwxrwx 1 root root  43M Jun  7 18:59 MariaDB-5.5.38-centos6-x86_64-server.rpm -rwxrwxrwx 1 root root  29M Jun  7 18:59 MariaDB-5.5.38-centos6-x86_64-test.rpm -rwxrwxrwx 1 root root  26M Jun 25 06:25 MariaDB-Galera-5.5.38-centos6-x86_64-server.rpm -rwxrwxrwx 1 root root 6.0M Jun 25 06:26 galera-25.3.5-1.rhel6.x86_64.rpm [root@nd1 ~]# yum localinstall socat-1.7.2.4-1.el6.rf.x86_64.rpm \ > MariaDB-5.5.38-centos6-x86_64-common.rpm \ > MariaDB-5.5.38-centos6-x86_64-client.rpm \ > MariaDB-Galera-5.5.38-centos6-x86_64-server.rpm \ > MariaDB-5.5.38-centos6-x86_64-compat.rpm \ > galera-25.3.5-1.rhel6.x86_64.rpm Loaded plugins: fastestmirror Setting up Local Package Process ...

 

二、初始化MariaDB數據庫並啓動MariaDB服務：

 

 

1 2 3 4 5 6 7 8 9 10 11 12

[root@nd1 ~]# mysql_install_db Installing MariaDB/MySQL system tables in '/var/lib/mysql' ... 140809  8:39:48 [Note] WSREP: Read nil XID from storage engines, skipping position init 140809  8:39:48 [Note] WSREP: wsrep_load(): loading provider library 'none' 140809  8:39:48 [Note] [Debug] WSREP: dummy_init 140809  8:40:00 [Note] [Debug] WSREP: dummy_disconnect 140809  8:40:00 [Note] WSREP: Service disconnected. 140809  8:40:01 [Note] WSREP: Some threads may fail to exit. 140809  8:40:01 [Note] [Debug] WSREP: dummy_free OK Filling help tables... ...

 

初始化完成後啓動MariaDB：

 

 

1 2 3

[root@nd1 ~]# /etc/init.d/mysql start Starting MySQL..... SUCCESS! [root@nd1 ~]#

 

此時能夠查看到Galera相關的配置選項，能夠看出Galera相關的配置參數有36個之多：

 

 

1 2 3 4 5

MariaDB [(none)]> show global variables like 'wsrep%'; 4d63dbf5f8aa375c0a7cb0459cb319ca  - 36 rows in set (0.01 sec)   MariaDB [(none)]>

 

三、在MariaDB實例上建立用於Galera集羣的用戶，第一個節點創建便可：

 

 

1 2 3 4 5 6 7 8 9 10 11 12 13

MariaDB [(none)]> grant all privileges on *.* to 'wsrep_sst-user'@'nd3' identified by 'password'; Query OK, 0 rows affected (0.01 sec)   MariaDB [(none)]> grant all privileges on *.* to 'wsrep_sst-user'@'nd2' identified by 'password'; Query OK, 0 rows affected (0.00 sec)   MariaDB [(none)]> grant all privileges on *.* to 'wsrep_sst-user'@'nd1' identified by 'password'; Query OK, 0 rows affected (0.00 sec)   MariaDB [(none)]> flush privileges; Query OK, 0 rows affected (0.01 sec)   MariaDB [(none)]>

 

四、先在一個節點上配置Galera Cluster：

 

 

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22

[root@nd1 ~]# ip addr show eth0 | grep 'inet.*scope global' | awk '{print $2}' 192.168.192.129/24 [root@nd1 ~]# cat /etc/my.cnf.d/galera.cnf [server] query_cache_size=0 binlog_format=ROW default_storage_engine=innodb innodb_autoinc_lock_mode=2 wsrep_provider=/usr/lib64/galera/libgalera_smm.so #wsrep_cluster_address=gcomm://192.168.192.133,192.168.192.134 wsrep_cluster_address=gcomm:// wsrep_cluster_name='example_cluster' wsrep_node_address='192.168.192.129' wsrep_node_name='nd1' wsrep_sst_method=rsync #wsrep_sst_method=xtrabackup wsrep_sst_auth=wsrep_sst-user:password [root@nd1 ~]# /etc/init.d/mysql bootstrap Bootstrapping the clusterStarting MySQL...... SUCCESS! [root@nd1 ~]# ps -ef | grep mysqld | grep -v grep | wc -l 2 [root@nd1 ~]#

補充：後續過程發現bootstrap模式初始化後並啓動的MariaDB實例不能正常的處理其餘節點請求，所以須要重啓MariaDB實例；
注意：wsrep_node_address和wsrep_node_name需是相對應的用於指定當前Galera節點主機及IP信息；當第一個節點啓動完成後即可以調換以上配置文件中的wsrep_cluster_address參數了；參數wsrep_sst_method可使用rsync方式或Percona提供的Xtrabackup；
經測發現wsrep_sst_method=xtrabackup時不能正常工做，有待進一步測試；
五、向以前配置的Galera集羣（即IP爲192.168.192.129的節點）中加入其餘新節點：

 

 

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16

[root@nd2 ~]# ip addr show eth0 | grep 'inet.*scope global' | awk '{print $2}' 192.168.192.133/24 [root@nd2 ~]# cat /etc/my.cnf.d/galera.cnf query_cache_size=0 binlog_format=ROW default_storage_engine=innodb innodb_autoinc_lock_mode=2 wsrep_provider=/usr/lib64/galera/libgalera_smm.so wsrep_cluster_address=gcomm://192.168.192.129,192.168.192.134 wsrep_cluster_name='example_cluster' wsrep_node_address='192.168.192.133' wsrep_node_name='nd2' #wsrep_sst_method=xtrabackup wsrep_sst_method=rsync wsrep_sst_auth=wsrep_sst-user:password [root@nd2 ~]#

第三節點：

 

 

1 2 3 4 5 6 7 8 9 10 11 12 13 14

[root@nd3 ~]# cat /etc/my.cnf.d/galera.cnf [server] query_cache_size=0 binlog_format=ROW default_storage_engine=innodb innodb_autoinc_lock_mode=2 wsrep_provider=/usr/lib64/galera/libgalera_smm.so wsrep_cluster_address=gcomm://192.168.192.129,192.168.192.133 wsrep_cluster_name='example_cluster' wsrep_node_address='192.168.192.134' wsrep_node_name='nd3' wsrep_sst_method=rsync wsrep_sst_auth=wsrep_sst-user:password [root@nd3 ~]#

六、依次啓動其餘節點，其餘節點會根據配置自動加入集羣並同步數據：

 

 

1 2 3

[root@nd2 ~]# /etc/init.d/mysql start Starting MySQL......SST in progress, setting sleep higher. SUCCESS! [root@nd2 ~]#

 

七、在Galera上查看各集羣狀態：

 

 

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34

MariaDB [(none)]> show global status like 'wsrep_cluster%'; +--------------------------+--------------------------------------+ | Variable_name            | Value                                | +--------------------------+--------------------------------------+ | wsrep_cluster_conf_id    | 1                                    | | wsrep_cluster_size       | 1                                    | | wsrep_cluster_state_uuid | 98f4ddfe-205a-11e4-b7a2-7ede18e6364f | | wsrep_cluster_status     | Primary                              | +--------------------------+--------------------------------------+ 4 rows in set (0.00 sec)   MariaDB [(none)]> show global status like 'wsrep_cluster%'; +--------------------------+--------------------------------------+ | Variable_name            | Value                                | +--------------------------+--------------------------------------+ | wsrep_cluster_conf_id    | 4                                    | | wsrep_cluster_size       | 2                                    | | wsrep_cluster_state_uuid | 98f4ddfe-205a-11e4-b7a2-7ede18e6364f | | wsrep_cluster_status     | Primary                              | +--------------------------+--------------------------------------+ 4 rows in set (0.01 sec)   MariaDB [(none)]> show global status like 'wsrep_cluster%'; +--------------------------+--------------------------------------+ | Variable_name            | Value                                | +--------------------------+--------------------------------------+ | wsrep_cluster_conf_id    | 5                                    | | wsrep_cluster_size       | 3                                    | | wsrep_cluster_state_uuid | 98f4ddfe-205a-11e4-b7a2-7ede18e6364f | | wsrep_cluster_status     | Primary                              | +--------------------------+--------------------------------------+ 4 rows in set (0.00 sec)   MariaDB [(none)]>

今後處能夠看出當有其餘節點加入到集羣中時，wsrep_cluster_size能夠反映出當前集羣節點數量；
八、在一節點上新建表並插入數據以在其餘實例上觀測數據是否同步：

 

 

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28

MariaDB [(none)]> create database galera; Query OK, 1 row affected (0.14 sec)   MariaDB [(none)]> create table t_galera(     ->   id int auto_increment primary key,     ->   name varchar(10)); Query OK, 0 rows affected (0.31 sec)   MariaDB [galera]> insert into t_galera values(null,rand()); Query OK, 1 row affected (0.01 sec)   MariaDB [galera]> insert into t_galera values(null,rand()),(null,rand()),(null,rand()); Query OK, 3 rows affected (0.07 sec) Records: 3  Duplicates: 0  Warnings: 0   MariaDB [galera]> select *,@@global.hostname from t_galera; +----+------------+-------------------+ | id | name       | @@global.hostname | +----+------------+-------------------+ |   3 | 0.47992669 | nd1.galera        | |   6 | 0.88471807 | nd1.galera        | |   9 | 0.98381029 | nd1.galera        | | 12 | 0.26489729 | nd1.galera        | | 15 | 0.37305561 | nd1.galera        | +----+------------+-------------------+ 5 rows in set (0.00 sec)   MariaDB [galera]>

在其餘節點上觀測數據：

 

 

1 2 3 4 5 6 7 8 9 10 11 12 13

MariaDB [galera]> select *,@@global.hostname from t_galera; +----+------------+-------------------+ | id | name       | @@global.hostname | +----+------------+-------------------+ |   3 | 0.47992669 | nd3.galera        | |   6 | 0.88471807 | nd3.galera        | |   9 | 0.98381029 | nd3.galera        | | 12 | 0.26489729 | nd3.galera        | | 15 | 0.37305561 | nd3.galera        | +----+------------+-------------------+ 5 rows in set (0.05 sec)   MariaDB [galera]>

 

 

 

1 2 3 4 5 6 7 8 9 10 11 12 13

MariaDB [galera]> select *,@@global.hostname from t_galera; +----+------------+-------------------+ | id | name       | @@global.hostname | +----+------------+-------------------+ |   3 | 0.47992669 | nd2.galera        | |   6 | 0.88471807 | nd2.galera        | |   9 | 0.98381029 | nd2.galera        | | 12 | 0.26489729 | nd2.galera        | | 15 | 0.37305561 | nd2.galera        | +----+------------+-------------------+ 5 rows in set (0.07 sec)   MariaDB [galera]>

八、測試節點故障機恢復：
首先中止nd1節點（已調整過該節點的wsrep_cluster_address配置）：

 

 

1 2 3

[root@nd1 ~]# /etc/init.d/mysql stop Shutting down MySQL....                                    [  OK  ] [root@nd1 ~]#

 

在nd2節點上刪除並更新數據：

 

 

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28

MariaDB [galera]> select *,@@global.hostname from t_galera; +----+------------+-------------------+ | id | name       | @@global.hostname | +----+------------+-------------------+ |   3 | 0.47992669 | nd2.galera        | |   6 | 0.88471807 | nd2.galera        | |   9 | 0.98381029 | nd2.galera        | | 12 | 0.26489729 | nd2.galera        | | 15 | 0.37305561 | nd2.galera        | +----+------------+-------------------+ 5 rows in set (0.00 sec)   MariaDB [galera]> delete from t_galera where id != 12; Query OK, 4 rows affected (0.07 sec)   MariaDB [galera]> update t_galera set name=rand() where id in (10,12); Query OK, 1 row affected (0.03 sec) Rows matched: 1  Changed: 1  Warnings: 0   MariaDB [galera]> select *,@@global.hostname from t_galera; +----+------------+-------------------+ | id | name       | @@global.hostname | +----+------------+-------------------+ | 12 | 0.00109373 | nd2.galera        | +----+------------+-------------------+ 1 row in set (0.00 sec)   MariaDB [galera]>

 

在nd3節點上查看數據是否一致：

 

 

1 2 3 4 5 6 7 8 9

MariaDB [galera]> select *,@@global.hostname from t_galera; +----+------------+-------------------+ | id | name       | @@global.hostname | +----+------------+-------------------+ | 12 | 0.00109373 | nd3.galera        | +----+------------+-------------------+ 1 row in set (0.00 sec)   MariaDB [galera]>

 

啓動中止的nd1節點：

 

 

1 2 3

[root@nd1 ~]# /etc/init.d/mysql start Starting MySQL........SST in progress, setting sleep higher[  OK  ] [root@nd1 ~]#

 

在nd1節點上查看數據是否自動同步：

 

 

1 2 3 4 5 6 7 8 9 10 11 12 13 14

MariaDB [galera]> select *,@@global.hostname from t_galera; ERROR 2006 (HY000): MySQL server has gone away No connection. Trying to reconnect... Connection id:    8 Current database: galera   +----+------------+-------------------+ | id | name       | @@global.hostname | +----+------------+-------------------+ | 12 | 0.00109373 | nd1.galera        | +----+------------+-------------------+ 1 row in set (0.12 sec)   MariaDB [galera]>

 

關於更多的INSERT、UPDATE、DELETE測試道理是相同的，各位可自行完成；固然，若是要在生產環境中使用的話還須要進行更復雜功能性、邏輯性測試。也歡迎各位一塊兒探討Galera Cluster的使用及測試經驗。