高可用OpenStack（Queen版）集羣-2.基礎服務

參考文檔：

1. Install-guide：https://docs.openstack.org/install-guide/
2. OpenStack High Availability Guide：https://docs.openstack.org/ha-guide/index.html
3. 理解Pacemaker：http://www.cnblogs.com/sammyliu/p/5025362.html
4. Ceph: http://docs.ceph.com/docs/master/start/intro/

 三．Mariadb集羣

1. 安裝mariadb

# 在所有controller節點安裝mariadb，以controller01節點爲例
[root@controller01 ~]# yum install mariadb mariadb-server python2-PyMySQL -y

# 安裝galera相關插件，利用galera搭建集羣
[root@controller01 ~]# yum install mariadb-server-galera mariadb-galera-common galera xinetd rsync -y 

2. 初始化mariadb

# 在所有控制節點初始化數據庫密碼，以controller01節點爲例；
# root初始密碼爲空
[root@controller01 ~]# systemctl restart mariadb.service
[root@controller01 ~]# mysql_secure_installation
Enter current password for root (enter for none): 
Set root password? [Y/n] y
New password: 
Re-enter new password: 
Remove anonymous users? [Y/n] y
Disallow root login remotely? [Y/n] n
Remove test database and access to it? [Y/n] y
Reload privilege tables now? [Y/n] y 

3. 修改mariadb配置文件

# 在所有控制節點/etc/my.cnf.d/目錄下新增openstack.cnf配置文件，主要設置集羣同步相關參數，以controller01節點爲例，個別涉及ip地址/host名等參數根據實際狀況修改
[root@controller01 my.cnf.d]# cat /etc/my.cnf.d/openstack.cnf 
[mysqld]
binlog_format = ROW
bind-address = 172.30.200.31
default-storage-engine = innodb
innodb_file_per_table = on
max_connections = 4096
collation-server = utf8_general_ci
character-set-server = utf8
[galera]
bind-address = 172.30.200.31
wsrep_provider = /usr/lib64/galera/libgalera_smm.so
wsrep_cluster_address ="gcomm://controller01,controller02,controller03"
wsrep_cluster_name = openstack-cluster-01
wsrep_node_name = controller01
wsrep_node_address = 172.30.200.31
wsrep_on=ON
wsrep_slave_threads=4
wsrep_sst_method=rsync
default_storage_engine=InnoDB
[embedded]
[mariadb]
[mariadb-10.1] 

4. 構建mariadb集羣

# 中止所有控制節點的mariadb服務，以controller01節點爲例
[root@controller01 ~]# systemctl stop mariadb.service

# 任選1個控制節點以以下方式啓動mariadb服務，這裏選擇controller01節點
[root@controller01 ~]# /usr/libexec/mysqld --wsrep-new-cluster --user=root &

# 其餘控制節點加入mariadb集羣，以controller02節點爲例；
# 啓動後加入集羣，controller02節點從controller01節點同步數據，也可同步查看mariadb日誌/var/log/mariadb/mariadb.log
[root@controller02 ~]# systemctl start mariadb.service
[root@controller02 ~]# systemctl status mariadb.service

# 從新啓動controller01節點；
# 啓動前刪除contrller01節點的數據 
[root@controller01 ~]# pkill -9 mysql
[root@controller01 ~]# rm -rf /var/lib/mysql/*

# 注意以system unit方式啓動mariadb服務時的權限
[root@controller01 ~]# chown mysql:mysql /var/run/mariadb/mariadb.pid

# 啓動後查看節點所在服務狀態，controller01節點從controller02節點同步數據
[root@controller01 ~]# systemctl start mariadb.service
[root@controller01 ~]# systemctl status mariadb.service

# 查看集羣狀態
[root@controller01 ~]# mysql -uroot -pmysql_pass
MariaDB [(none)]> show status like "wsrep_cluster_size";
MariaDB [(none)]> SHOW status LIKE 'wsrep_ready';

5. 設置心跳檢測clustercheck

1）準備腳本

# 下載clustercheck腳本
[root@controller01 ~]# wget https://raw.githubusercontent.com/olafz/percona-clustercheck/master/clustercheck

# 賦權
[root@controller01 ~]# chmod +x clustercheck
[root@controller01 ~]# cp ~/clustercheck /usr/bin/ 

2）建立心跳檢測用戶

# 在任意控制節點建立clustercheck_user用戶並賦權；
# 注意帳號/密碼與腳本中的帳號/密碼對應，這裏採用的是腳本默認的帳號/密碼，不然須要修改clustercheck腳本文件
[root@controller01 ~]# mysql -uroot -pmysql_pass
MariaDB [(none)]> GRANT PROCESS ON *.* TO 'clustercheckuser'@'localhost' IDENTIFIED BY 'clustercheckpassword!';
MariaDB [(none)]> FLUSH PRIVILEGES; 

3）檢測配置文件

# 在所有控制節點新增心跳檢測服務配置文件/etc/xinetd.d/mysqlchk，以controller01節點爲例
[root@controller01 ~]# touch /etc/xinetd.d/mysqlchk
[root@controller01 ~]# vim /etc/xinetd.d/mysqlchk
# default: on
# description: mysqlchk
service mysqlchk
{
   port = 9200
   disable = no
   socket_type = stream
   protocol = tcp
   wait = no
   user = root
   group = root
   groups = yes
   server = /usr/bin/clustercheck
   type = UNLISTED
   per_source = UNLIMITED
   log_on_success =
   log_on_failure = HOST
   flags = REUSE
} 

4）啓動心跳檢測服務

# 修改/etc/services，變動tcp9200端口用途，以controller01節點爲例
[root@controller01 ~]# vim /etc/services
#wap-wsp         9200/tcp                # WAP connectionless session service
mysqlchk        9200/tcp                # mysqlchk

# 啓動xinetd服務，以controller01節點爲例
[root@controller01 ~]# systemctl daemon-reload
[root@controller01 ~]# systemctl enable xinetd
[root@controller01 ~]# systemctl start xinetd 

5）測試心跳檢測腳本

# 在所有控制節點驗證，以controller01節點爲例
[root@controller01 ~]# /usr/bin/clustercheck

四．RabbitMQ集羣

採用openstack官方的安裝方法，在未更新erlang的狀況下，rabbitmq不是最新版本。

若是須要部署最新版本rabbitmq集羣，可參考：http://www.cnblogs.com/netonline/p/7678321.html 

1. 安裝rabbitmq

# 在所有控制節點，使用aliyun的epel鏡像，以controller01節點爲例
[root@controller01 ~]# wget -O /etc/yum.repos.d/epel.repo http://mirrors.aliyun.com/repo/epel-7.repo
[root@controller01 ~]# yum install erlang rabbitmq-server -y

# 設置開機啓動
[root@controller01 ~]# systemctl enable rabbitmq-server.service 

2. 構建rabbitmq集羣

# 任選1個控制節點首先啓動rabbitmq服務，這裏選擇controller01節點
[root@controller01 ~]# systemctl start rabbitmq-server.service
[root@controller01 ~]# rabbitmqctl cluster_status

# 分發.erlang.cookie
[root@controller01 ~]# scp /var/lib/rabbitmq/.erlang.cookie root@172.30.200.32:/var/lib/rabbitmq/
[root@controller01 ~]# scp /var/lib/rabbitmq/.erlang.cookie root@172.30.200.33:/var/lib/rabbitmq/

# 修改controller02/03節點.erlang.cookie文件的用戶/組，以controller02節點爲例
[root@controller02 ~]# chown rabbitmq:rabbitmq /var/lib/rabbitmq/.erlang.cookie

# 注意修改所有控制節點.erlang.cookie文件的權限，默認即400權限，可不修改
[root@controller02 ~]# ll /var/lib/rabbitmq/.erlang.cookie

# 啓動controller02/03節點的rabbitmq服務 
[root@controller02 ~]# systemctl start rabbitmq-server

[root@controller03 ~]# systemctl start rabbitmq-server

# 構建集羣，controller02/03節點以ram節點的形式加入集羣
[root@controller02 ~]# rabbitmqctl stop_app
[root@controller02 ~]# rabbitmqctl join_cluster --ram rabbit@controller01
[root@controller02 ~]# rabbitmqctl start_app

[root@controller03 ~]# rabbitmqctl stop_app
[root@controller03 ~]# rabbitmqctl join_cluster --ram rabbit@controller01
[root@controller03 ~]# rabbitmqctl start_app

# 任意節點可驗證集羣狀態
[root@controller01 ~]# rabbitmqctl cluster_status

3. rabbitmq帳號

# 在任意節點新建帳號並設置密碼，以controller01節點爲例
[root@controller01 ~]# rabbitmqctl add_user openstack rabbitmq_pass

# 設置新建帳號的狀態
[root@controller01 ~]# rabbitmqctl set_user_tags openstack administrator

# 設置新建帳號的權限
[root@controller01 ~]# rabbitmqctl set_permissions -p "/" openstack ".*" ".*" ".*"

# 查看帳號
[root@controller01 ~]# rabbitmqctl list_users 

4. 鏡像隊列ha

# 設置鏡像隊列高可用
[root@controller01 ~]# rabbitmqctl set_policy ha-all "^" '{"ha-mode":"all"}'

# 查看鏡像隊列策略
[root@controller01 ~]# rabbitmqctl list_policies 

5. 安裝web管理插件

# 在所有控制節點安裝web管理插件，以controller01節點爲例
[root@controller01 ~]# rabbitmq-plugins enable rabbitmq_management

訪問任意節點，如：http://172.30.200.31:15672

五．Memcached集羣

Memcached是無狀態的，各控制節點獨立部署，openstack各服務模塊統一調用多個控制節點的memcached服務便可。

採用openstack官方的安裝方法，若是須要部署最新版本memcached，可參考：http://www.cnblogs.com/netonline/p/7805900.html

如下配置以controller01節點爲例。 

1. 安裝memcached

# 在所有控制節點安裝memcached
[root@controller01 ~]# yum install memcached python-memcached -y

2. 設置memcached

# 在所有安裝memcached服務的節點設置服務監聽地址
[root@controller01 ~]# sed -i 's|127.0.0.1,::1|0.0.0.0|g' /etc/sysconfig/memcached 

3. 開機啓動

[root@controller01 ~]# systemctl enable memcached.service
[root@controller01 ~]# systemctl start memcached.service
[root@controller01 ~]# systemctl status memcached.service