Ubuntu16.04環境下搭建基於三臺主機的mysql galera cluster集羣（實測有效）

（注意：

（1）文中紅色字體部分不必定須要操做

（2）因爲word文檔編輯的緣由，實際操做時部分命令須要手動輸入！！直接複製粘貼會提示錯誤！！

）

一  搭建環境：

1 Ubuntu16.04版本（系統登陸：root（199077））

2 mysql5.7（用戶名root 密碼199077）

二 安裝環境：

首先，不管是在雲服務器、物理機仍是虛擬機上，至少須要3臺主機（咱們將這三臺主機稱爲「Nodes」），對於每個」Node」，硬件最低配置要求爲： 

- 1GHz單核CPU 
- 512M的內存 
- 100Mbps的網絡鏈接 
- 三臺主機IP爲： 
- Node1: 192.168.93.135（主節點）
- Node2: 192.168.93.132（從節點1）
- Node3: 192.168.93.134（從節點2）

其次要禁用SElinux、防火牆配置、禁用AppArmor，

這些在Ubuntu14.04下都是不用配置的，因此就不詳細說明配置步驟。但在16.04環境下需嚴格配置：

（1）禁用SELinux的mysqld

         若是你啓用了SELinux,它可能會阻止 mysqld從執行所需的操做。你必須禁用SELinux mysqld或容許配置它 mysqld上運行外部程序和開放的監聽套接字端口外,非特權用戶能夠作的事情。

       禁用SELinux mysql運行如下命令:

semanage permissive -a mysqld_t

（2）防火牆配置：咱們須要在每一個節點上設置其防火牆規則，使得各個節點能與集羣通訊。在每一個節點上均執行以下命令：

  iptables --append INPUT --protocol tcp \

      --source 192.168.93.135 --jump ACCEPT

 iptables --apend INPUT --protocol tcp \

      --source 192.168.93.132 --jump ACCEPT

 iptables --append INPUT --protocol tcp \

      --source 192.168.93.134 --jump ACCEPT

 

而後保存防火牆規則：

iptables-save

 

(3)禁用Ubuntu的apparmor：apparmor主要是Ubuntu系統中對mysql訪問權限的控制，咱們必須禁用AppArmor，以便配置mysql的mysqld可以運行外部程序。

禁用apparmor：

sudo ln -s /etc/apparmor.d/usr /etc/apparmor.d/disable/.sbin.mysqld

重啓apparmor：

sudo service apparmor restart

注：若是不由用，則在登陸mysql時會出現以下錯誤：
apparmor="DENIED" operation="open" ........
對應解決方案：
第一種：依次執行以下命令：

apt install apparmor-utils

 sudo aa-complain/etc/apparmor.d/usr.sbin.mysqld

service apparmor restart

第二種：直接修改apparmor配置文件

nano /etc/apparmor.d/usr.sbin.mysqld

在配置文件中增長」flags=(complain)」

/usr/sbin/mysqld flags=(complain) {

  #include <abstractions/base>

  #include <abstractions/nameservice>

  #include <abstractions/user-tmp>

  #include <abstractions/mysql>

 ........

 

三 開始安裝
（1）安裝對應依賴

apt-get install python-software-properties  

apt-get install software-properties-common

（2）添加GunPG key

apt-key adv --keyserver keyserver.ubuntu.com --recv BC19DDBA

（3）配置Ubuntu下galera源

建立galera.list文件：

cd /etc/apt/sources.list.d

touch galera.list 

nano -w galera.list

在galera.list中添加內容：

#deb http://releases.galeracluster.com/mysql-wsrep-5.7/ubuntu xenial main  

#deb http://releases.galeracluster.com/galera-3/ubuntu xenial main  

deb http://releases.galeracluster.com/ubuntu xenial main

若是不清楚本身的Ubuntu詳細信息，可經過命令lsb_release -a來查看。

（4）繼續配置源

 cd /etc/apt/preferences.d

touch  galera.pref 

nano -w /etc/apt/preferences.d/galera.pref

在galera.pref文件中添加內容：

Package: *

Pin: origin releases.galeracluster.com

Pin-Priority: 1001

（5）接下來更新一下：apt-get update

(6) 一旦在全部三個服務器上更新了存儲庫，咱們就能夠在三臺服務器上運行如下命令安裝MySQL5.7的修補程序與Galera

apt-get install galera-3 galera-arbitrator-3 mysql-wsrep-5.7

（7）安裝完成以後，在每個節點上啓動Mysql：

Service mysql start

（8）驗證節點之間是否能互相ping通：

主節點135上：

Ping 192.168.93.132

Ping 192.168.93.134

從節點132上：

Ping 192.168.93.135

Ping 192.168.93.134

從節點134上：

Ping 192.168.93.135

Ping 192.168.93.132

能ping同則表示三節點之間能互相通訊。

（9）分別在三個節點上修改mysql的配置文件/etc/mysql/my.cnf

主節點135上：

# Copyright (c) 2016, Oracle and/or its affiliates. All rights reserved.

#

# This program is free software; you can redistribute it and/or modify

# it under the terms of the GNU General Public License as published by

# the Free Software Foundation; version 2 of the License.

#

# This program is distributed in the hope that it will be useful,

# but WITHOUT ANY WARRANTY; without even the implied warranty of

# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the

# GNU General Public License for more details.

#

# You should have received a copy of the GNU General Public License

# along with this program; if not, write to the Free Software

# Foundation, Inc., 51 Franklin St, Fifth Floor, Boston, MA  02110-1301 USA

#

# The MySQL database server configuration file.

#

# You can copy this to one of:

# - "/etc/mysql/my.cnf" to set global options,

# - "~/.my.cnf" to set user-specific options.

#

# One can use all long options that the program supports.

# Run program with --help to get a list of available options and with

# --print-defaults to see which it would actually understand and use.

#

# For explanations see

# http://dev.mysql.com/doc/mysql/en/server-system-variables.html

 

#

# * IMPORTANT: Additional settings that can override those from this file!

#   The files must end with '.cnf', otherwise they'll be ignored.

#

!includedir /etc/mysql/conf.d/

!includedir /etc/mysql/mysql.conf.d/

[mysqld]

binlog_format=ROW

default-storage-engine=innodb

innodb_autoinc_lock_mode=2

wsrep_provider=/usr/lib/libgalera_smm.so

wsrep_provider_options="gcache.size=300M; gcache.page_size=1G"

wsrep_cluster_name="galeracluster1"

wsrep_cluster_address="gcomm://"

wsrep_sst_method=rsync

wsrep_sst_auth=wsrep:199077

wsrep_node_address="192.168.93.135"

wsrep_node_name=node135

bind-address=0.0.0.0

 

從節點132上：

# Copyright (c) 2016, Oracle and/or its affiliates. All rights reserved.

#

# This program is free software; you can redistribute it and/or modify

# it under the terms of the GNU General Public License as published by

# the Free Software Foundation; version 2 of the License.

#

# This program is distributed in the hope that it will be useful,

# but WITHOUT ANY WARRANTY; without even the implied warranty of

# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the

# GNU General Public License for more details.

#

# You should have received a copy of the GNU General Public License

# along with this program; if not, write to the Free Software

# Foundation, Inc., 51 Franklin St, Fifth Floor, Boston, MA  02110-1301 USA

 

#

# The MySQL database server configuration file.

#

# You can copy this to one of:

# - "/etc/mysql/my.cnf" to set global options,

# - "~/.my.cnf" to set user-specific options.

#

# One can use all long options that the program supports.

# Run program with --help to get a list of available options and with

# --print-defaults to see which it would actually understand and use.

#

# For explanations see

# http://dev.mysql.com/doc/mysql/en/server-system-variables.html

#

# * IMPORTANT: Additional settings that can override those from this file!

#   The files must end with '.cnf', otherwise they'll be ignored.

#

 

!includedir /etc/mysql/conf.d/

!includedir /etc/mysql/mysql.conf.d/

[mysqld]

binlog_format=ROW

default-storage-engine=innodb

innodb_autoinc_lock_mode=2

wsrep_provider=/usr/lib/libgalera_smm.so

wsrep_provider_options="gcache.size=300M; gcache.page_size=1G"

wsrep_cluster_name="galeracluster1"

wsrep_cluster_address="gcomm://192.168.93.135"

wsrep_sst_method=rsync

wsrep_sst_auth=wsrep:199077

wsrep_node_address="192.168.93.132"

wsrep_node_name=node132

bind-address=0.0.0.0

 

從節點134上：

# Copyright (c) 2016, Oracle and/or its affiliates. All rights reserved.

#

# This program is free software; you can redistribute it and/or modify

# it under the terms of the GNU General Public License as published by

# the Free Software Foundation; version 2 of the License.

#

# This program is distributed in the hope that it will be useful,

# but WITHOUT ANY WARRANTY; without even the implied warranty of

# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the

# GNU General Public License for more details.

#

# You should have received a copy of the GNU General Public License

# along with this program; if not, write to the Free Software

# Foundation, Inc., 51 Franklin St, Fifth Floor, Boston, MA  02110-1301 USA

 

#

# The MySQL database server configuration file.

#

# You can copy this to one of:

# - "/etc/mysql/my.cnf" to set global options,

# - "~/.my.cnf" to set user-specific options.

#

# One can use all long options that the program supports.

# Run program with --help to get a list of available options and with

# --print-defaults to see which it would actually understand and use.

#

# For explanations see

#

# * IMPORTANT: Additional settings that can override those from this file!

#   The files must end with '.cnf', otherwise they'll be ignored.

#

!includedir /etc/mysql/conf.d/

!includedir /etc/mysql/mysql.conf.d/

[mysqld]

binlog_format=ROW

default-storage-engine=innodb

innodb_autoinc_lock_mode=2

wsrep_provider=/usr/lib/libgalera_smm.so

wsrep_provider_options="gcache.size=300M; gcache.page_size=1G"

wsrep_cluster_name="galeracluster1"

wsrep_cluster_address="gcomm://192.168.93.135"

wsrep_sst_method=rsync

wsrep_sst_auth=wsrep:199077

wsrep_node_address="192.168.93.134"

wsrep_node_name=node134

bind-address=0.0.0.0

（10）

在每一個節點上重啓mysql：

Service mysql restart

(11) 試着在任意一個節點上登陸其他兩個節點，若是都能彼此登陸，則代表三臺服務器上的mysql彼此監聽成功。

如在135上登陸132，134

mysql -h 192.168.93.132 -u root -p

mysql -h 192.168.93.134 -u root -p

若是彼此登陸不成功，則須要在每一個節點上建立用戶以實現mysql的遠程登陸。

。建立用戶時，須要對應每一個節點的mysql的配置文件/etc/mysql/my.cnf中的「wsrep_sst_auth=wsrep:199077」項，該項用來設置服務器之間mysql彼此訪問的用戶名和密碼，其中wsrep爲用戶名，199077爲密碼。

具體建立命令以下：

每一個節點數據庫遠程訪問的建立

create user 'wsrep'@'%' identified by '199077';

create user 'root'@'%' identified by '199077';#這一句不是很清楚是否能夠去掉

GRANT ALL PRIVILEGES ON *.* TO 'wsrep'@'%' WITH GRANT OPTION;

FLUSH PRIVILEGES;

建立完後重啓各個節點的mysql,再試着在每一個節點上登陸其餘節點是否成功。

如在從節點132上登陸13五、134的mysql

mysql -h 192.168.93.135 -u root -p  #成功

mysql -h 192.168.93.134 -u root -p  #成功

mysql -h 192.168.93.135 -u wsrep -p #成功

mysql -h 192.168.93.134 -u wsrep -p #成功

主節點135和從節點134也進行一樣操做，若是均能登陸，則設置成功！

 

（12）接下來，Ubuntu有一個特別的用戶叫debian-sys-maint會來維持Mysql的運行任務，因爲數據庫的複製，這個維護的運行只會在第一個節點（即主節點）成功進行，在其餘節點上則會失敗，因此要將啓動節點（即主節點）上的/etc/mysql/debian.cnf文件拷貝到其餘節點上。

即在主節點135上執行以下命令：

scp /etc/mysql/debian.cnf root@192.168.93.132:/etc/mysql/ 

scp /etc/mysql/debian.cnf root@192.168.93.134:/etc/mysql/

 

執行過程當中可能會失敗，這個問題是由於在主節點的/etc/mysql/my.cnf文件中有一行要改成：

PermitRootLogin yes

這樣就會成功了！當以上步驟都完成了以後，能夠準備開始啓動集羣了！

注：步驟（12）我在實際中未操做過，但集羣也成功了，Debian 對mysql集羣的平常維護可能有必定影響，讀者可自行研究。

 

四 啓動集羣

在主節點上將mysql服務先停掉而後重啓，注意必定要在前面提到的wsrep_cluster_address="gcomm://"的節點上執行最開始的啓動程序：

service mysql stop 

service mysql start --wsrep-new-cluster

在其餘節點上執行：service mysql restart

 

五 驗證是否成功

查看數據表

登陸主節點135：

mysql -u root -p 

Enter password:

mysql> SHOW STATUS LIKE 'wsrep%';

重點關注的是下面這幾行：

wsrep_ready     |     ON

若是狀態是 ON，那麼就要看：

wsrep_cluster_size    |   3

若是是3，那麼是成功的，不然是不成功的！

六 數據庫複製測試

在主節點135上建立一個表：

CREATE database test;

馬上到節點132和134上使用 show databases; 查看是否也有了test這個數據庫，若是有則成功！

 

七 集羣的狀態監控相關參數

（1）wsrep_cluster_state_uuid顯示了cluster的state UUID,由此可看出該節點是否仍是集羣的一員；

SHOW GLOBAL STATUS LIKE 'wsrep_cluster_state_uuid'

（2）wsrep_cluster_conf_id顯示了整個集羣的變化次數。全部節點都應相同，不然說明某個節點與集羣斷開了；

（3）wsrep_cluster_status顯示集羣裏節點的主狀態。標準返回primary。如返回non-Primary或其餘值說明是多個節點改變致使的節點丟失或者腦裂。若是全部節點都返回不是Primary，則要重設quorum。具體參見http://galeracluster.com/documentation-webpages/quorumreset.html若是返回都正常，說明覆制機制在每一個節點都能正常工做；

show global status like 'wsrep_cluster_status';
+----------------------+---------+
| Variable_name        | Value   |
+----------------------+---------+
| wsrep_cluster_status | Primary |
+----------------------+---------+

 

八 重啓集羣

偶爾，您可能不得不從新啓動整個Galera集羣。例如，在停電的狀況下，每一個節點都被關閉，而且根本沒有mysqld進程，這可能會發生。 

 

要從新啓動整個Galera集羣，請完成如下步驟：

 

 

 

 

 

九  配置過程當中可能遇到的錯誤：

（1）Failed to restart mysql.service: Unit mysql.service is masked.

systemctl unmask mysql.service

service mysql start

（2）Can't connect to local MySQL server through socket '/var/run/mysqld/mysqld.sock'

解決途徑：http://blog.csdn.net/luoxia0805/article/details/30265479

（3）