Docker MariaDB 10.3 Galera Cluster 集羣同步複製 多主 Docker Haproxy 負載均衡

mariadb 現有動態列,支持json格式存儲,相似mongodb的bson,可是操做能力較爲尷尬,中間件有spider,我很是感興趣的一個東西

關於spider 這裏有一篇很好的博文,有時間必定得嘗試

Spider引擎分佈式數據庫解決方案

mariadb 10.3 自帶了 Galera ,可是坑爹的是官方鏡像裏面徹底沒有提起.不要緊,咱們程序員靠猜能完成不少事情.

在docker下部署,wsrep_sst_method=xtrabackup-v2 同步方式有坑,花了些時間,沒有解決,目前就用wsrep_sst_method=rsync.

在 I7 4790k , nvme 固態硬盤下,測試了50條線程併發負載均衡寫入,同步未見異常,可是個人表只設置了一個字段.

在併發寫入的時候,把其中一個節點關閉,而後再啓動,數據自動同步.

如下是在我電腦上建的一個虛擬機,16G內存,4 * 2 CPU 設置,單機部署的集羣環境

docker pull mariadb:10.3

使用微容器的鏡像版本,我用的是1.7.9

docker pull haproxy:alpine

haproxy.cfg

global  
chroot /usr/local  
daemon  
nbproc 1  
group nobody  
user nobody  
pidfile /opt/haproxy/logs/haproxy.pid  
ulimit-n 65536  
#spread-checks 5m   
#stats timeout 5m  
#stats maxconn 100  
  
########默認配置############  
defaults  
mode tcp              
retries 3              #兩次鏈接失敗就認爲是服務器不可用，也能夠經過後面設置  
option redispatch      #當serverId對應的服務器掛掉後，強制定向到其餘健康的服務器  
option abortonclose    #當服務器負載很高的時候，自動結束掉當前隊列處理比較久的連接  
maxconn 32000          #默認的最大鏈接數  
timeout connect 5000ms #鏈接超時  
timeout client 30000ms #客戶端超時  
timeout server 30000ms #服務器超時  
#timeout check 2000    #心跳檢測超時  
log 127.0.0.1 local0 err #[err warning info debug]  
   
########MariaDB配置#################  
listen mariadb  
bind 0.0.0.0:3306  
mode tcp  
balance roundrobin  
server s1 192.168.1.9:3306 weight 1 maxconn 10000 check inter 10s  
server s2 192.168.1.9:3307 weight 1 maxconn 10000 check inter 10s  
server s3 192.168.1.9:3308 weight 1 maxconn 10000 check inter 10s  

 

 

 

關掉selinux和防火牆,避免夜長夢多

 

 /et/hosts

 

 

mkdir /home/docker/mariadb/cluster0/
mkdir /home/docker/mariadb/cluster0/conf
mkdir /home/docker/mariadb/cluster0/data
mkdir /home/docker/mariadb/cluster1/
mkdir /home/docker/mariadb/cluster1/conf
mkdir /home/docker/mariadb/cluster1/data
mkdir /home/docker/mariadb/cluster2/
mkdir /home/docker/mariadb/cluster2/data
mkdir /home/docker/mariadb/cluster2/conf

Cluster0/server.cnf配置文件 先註釋掉  wsrep_cluster_address,由於要初始化用戶以後再開始集羣

注意: wsrep_provider_options="gcache.size=128M"#同步複製緩衝池 ,這裏我是由於個人電腦固態硬盤容量有限才設置成128M實際使用請根據狀況修改

vi /home/docker/mariadb/cluster0/conf/server.cnf

[server]  
[mysqld]  
server_id=130
pid-file=/var/run/mysqld/mysqld.pid
socket=/var/run/mysqld/mysqld.sock
basedir=/usr
datadir=/var/lib/mysql
tmpdir=/tmp
user=mysql
skip-external-locking
skip-name-resolve
character-set-server=utf8
port=3306


#
# Instead of skip-networking the default is now to listen only on
# localhost which is more compatible and is not less secure.
#bind-address        = 127.0.0.1
#
# * Fine Tuning
#
max_connections=1000
connect_timeout=5
wait_timeout=600
max_allowed_packet=16M
thread_cache_size=128
sort_buffer_size=4M
bulk_insert_buffer_size    =16M
tmp_table_size=32M
max_heap_table_size    =32M
[galera]  
wsrep_causal_reads=ON  #節點應用完事務才返回查詢請求  
wsrep_provider_options="gcache.size=128M"#同步複製緩衝池  
wsrep_certify_nonPK=ON   #爲沒有顯式申明主鍵的表生成一個用於certificationtest的主鍵，默認爲ON  
#log-bin=/app/galera/mysql-bin  #若是不接從庫，註釋掉  
#log_slave_updates=1         #若是不接從庫，註釋掉  
query_cache_size=0           #關閉查詢緩存  
wsrep_on=ON   #開啓全同步複製模式  
wsrep_provider=/usr/lib/galera/libgalera_smm.so #galera library  
wsrep_cluster_name=MariaDB-Galera-Cluster  
#wsrep_cluster_address="gcomm://192.168.1.9:4567,192.168.1.9:4568,192.168.1.9:4569"  #galera cluster URL  
wsrep_node_name=mariadb-0 
#wsrep_node_address=172.18.0.4
wsrep_sst_auth=syncuser:syncuser
#wsrep_sst_method=xtrabackup-v2
wsrep_sst_method=rsync
binlog_format=row  
default_storage_engine=InnoDB  
innodb_autoinc_lock_mode=2   #主鍵自增模式修改成交叉模式  
wsrep_slave_threads=8  #開啓並行複製線程，根據CPU核數設置  
innodb_flush_log_at_trx_commit=0   #事務提交每隔1秒刷盤  
innodb_buffer_pool_size=2G
[embedded]  
[mariadb]  
[mariadb-10.3]

Cluster1差別部分

wsrep_node_name=mariadb-1

Cluster2差別部分

wsrep_node_name=mariadb-2

 

啓動第一個節點

docker run -d --name mariadb-cluster0 --expose 4567 -p 4567:4567 -p 3306:3306 -e TIMEZONE=Asia/Shanghai -v /home/docker/mariadb/cluster0/conf:/etc/mysql/conf.d  -v /home/docker/mariadb/cluster0/data:/var/lib/mysql -e MYSQL_ALLOW_EMPTY_PASSWORD=yse mariadb:10.3 

進去容器添加用戶

[root@laizhenwei ~]# docker exec -it 1195455508b1 /bin/bash
root@1195455508b1:/# mysql -uroot -p
Enter password: 
Welcome to the MariaDB monitor.  Commands end with ; or \g.
Your MariaDB connection id is 8
Server version: 10.3.2-MariaDB-10.3.2+maria~jessie mariadb.org binary distribution

Copyright (c) 2000, 2017, Oracle, MariaDB Corporation Ab and others.

Type 'help;' or '\h' for help. Type '\c' to clear the current input statement.

MariaDB [(none)]> GRANT ALL PRIVILEGES ON *.* TO 'root'@'%'IDENTIFIED BY '' WITH GRANT OPTION;
Query OK, 0 rows affected (0.001 sec)

MariaDB [(none)]> GRANT ALL PRIVILEGES ON *.* TO 'syncuser'@'%'IDENTIFIED BY 'syncuser' WITH GRANT OPTION;
Query OK, 0 rows affected (0.001 sec)

MariaDB [(none)]> flush privileges;
Query OK, 0 rows affected (0.001 sec)

MariaDB [(none)]> shutdown;
Query OK, 0 rows affected (0.001 sec)

MariaDB [(none)]>

 

第二個節點,重複以上添加用戶操做

docker run -d --name mariadb-cluster1 --expose 4567 -p 4568:4567 -p 3307:3306 -e TIMEZONE=Asia/Shanghai -v /home/docker/mariadb/cluster1/conf:/etc/mysql/conf.d  -v /home/docker/mariadb/cluster1/data:/var/lib/mysql -e MYSQL_ALLOW_EMPTY_PASSWORD=yse mariadb:10.3

第三個,重複以上添加用戶操做

docker run -d --name mariadb-cluster2  --expose 4567 -p 4569:4567 -p 3308:3306 -e TIMEZONE=Asia/Shanghai -v /home/docker/mariadb/cluster2/conf:/etc/mysql/conf.d  -v /home/docker/mariadb/cluster2/data:/var/lib/mysql -e MYSQL_ALLOW_EMPTY_PASSWORD=yse mariadb:10.3

中止以及刪除容器(注意操做,我這裏是中止全部以及刪除全部容器)

docker stop $(docker ps -a -q)
docker rm $(docker ps -a -q)

 

解開wsrep_cluster_address註釋,再次啓動3個節點  --wsrep-new-cluster 只有在第一次集羣,第一個節點,才須要加上,下次啓動不須要此參數

cluster0

docker run --name mariadb-cluster0 --expose 4567 -p 4567:4567 -p 3306:3306 -v /home/docker/mariadb/cluster0/conf:/etc/mysql/conf.d  -v /home/docker/mariadb/cluster0/data:/var/lib/mysql mariadb:10.3 --wsrep-new-cluster

cluster1

docker run --name mariadb-cluster1 --expose 4567 -p 4568:4567 -p 3307:3306 -v /home/docker/mariadb/cluster1/conf:/etc/mysql/conf.d  -v /home/docker/mariadb/cluster1/data:/var/lib/mysql mariadb:10.3

cluster2

docker run --name mariadb-cluster2 --expose 4567 -p 4569:4567 -p 3308:3306 -v /home/docker/mariadb/cluster2/conf:/etc/mysql/conf.d  -v /home/docker/mariadb/cluster2/data:/var/lib/mysql mariadb:10.3

 

查看SHOW STATUS LIKE 'wsrep_cluster_size';

 

最後啓動haproxy容器,這裏不搞keepalive,我相信docker 重啓機制

docker run -d --restart=always --name haproxy --expose 3366 -p 3366:3366 -v /home/docker/haproxy/conf:/usr/local/etc/haproxy:ro 87e3fb63d1ff

用navicat 鏈接到haproxy,建立一個test數據庫