【rabbitmq】rabbitmq集羣環境搭建

安裝rabbitmq-server

總共有3臺虛擬機，都安裝有rabbitmq服務，安裝過程可參考：
【rabbitmq】Centos7 下安裝rabbitmq

建立用戶和vhost

說明： 此步驟不是必須的，文章後面的用戶和vhost可能與此步驟建立的不一致，此處僅僅是建立的示例。

rabbitmqctl add_vhost /my_vhost

rabbitmqctl add_user admin admin
rabbitmqctl set_user_tags admin administrator #管理員權限

# rabbitmqctl set_permissions [-p vhost] [user] [permission ⇒ (modify) (write) (read)]
rabbitmqctl set_permissions -p /my_vhost admin ".*"  ".*" ".*"

rabbitmqctl list_permissions -p /my_vhost

機器列表

機器	ip	/etc/hostname	/etc/hostname	說明
node1	192.168.35.135	rmq-node1	rmq-node1	做爲首節點
node2	192.168.35.136	rmq-node2	rmq-node2	加入node1集羣
node3	192.168.35.137	rmq-node3	rmq-node3	加入node1集羣

注意：修改/etc/hostname爲上述表格中的值

# node1執行
[root@rmq-node1 ~]# rabbitmqctl status
Status of node 'rabbit@rmq-node1'
...

# node2執行
[root@rmq-node2 ~]#  rabbitmqctl status
Status of node 'rabbit@rmq-node2'
...

# node3執行
[root@rmq-node3 ~]#  rabbitmqctl status
Status of node 'rabbit@rmq-node3'
...

---

搭建集羣環境

保證magic cookie相同

rabbitmq使用erlang編寫，erlang自然具備分佈式特性，爲了是各erlang節點可以互相通訊，須要保證 magic cookie相同；

將node1中的cookie複製到node2和node3

# node2和 node3上分別執行
scp root@192.168.35.135:/var/lib/rabbitmq/.erlang.cookie /var/lib/rabbitmq/.erlang.cookie

如今3個node上的.erlang.cookie都是相同的；

注意點：

1. 須要保證.erlang.cookie的默認權限爲：400;
   
2. 全部用戶和組爲 rabbitmq

chmod 400 /var/lib/rabbitmq/.erlang.cookie
chown rabbitmq:rabbitmq /var/lib/rabbitmq/.erlang.cookie

修改hosts文件

# 3個節點都要修改 hosts文件
vi /etc/hosts 

# 添加
192.168.35.135 rmq-node1
192.168.35.136 rmq-node2
192.168.35.137 rmq-node3

在每一個節點上，都保證能ping通

ping rmq-node1
ping rmq-node2
ping rmq-node3

將node2節點和node3節點加入集羣

node2和node3加入集羣前，node1的狀態

[root@rmq-node1 ~]# rabbitmqctl  cluster_status
Cluster status of node 'rabbit@rmq-node1'
[{nodes,[{disc,['rabbit@rmq-node1']}]},   //此時只有一個節點
 {running_nodes,['rabbit@rmq-node1']},
 {cluster_name,<<"rabbit@rmq-node1">>},
 {partitions,[]},
 {alarms,[{'rabbit@rmq-node1',[]}]}]

說明：上面的rabbit@rmq-node1就是後面node2和node3要加入的首節點信息

node2做爲disk節點

未加入以前

[root@rmq-node2 ~]# rabbitmqctl cluster_status
Cluster status of node 'rabbit@rmq-node2'
[{nodes,[{disc,['rabbit@rmq-node2']}]},
 {running_nodes,['rabbit@rmq-node2']},
 {cluster_name,<<"rabbit@rmq-node2">>},
 {partitions,[]},
 {alarms,[{'rabbit@rmq-node2',[]}]}]

加入集羣

# stop application and reset
rabbitmqctl stop_app 
rabbitmqctl reset 

#join in cluster (specify only hostname, not with FQDN) 
rabbitmqctl join_cluster rabbit@rmq-node1

# start application
rabbitmqctl start_app

# show status
rabbitmqctl cluster_status

查看狀態：

[root@rmq-node1 ~]# rabbitmqctl  cluster_status
Cluster status of node 'rabbit@rmq-node1'
[{nodes,[{disc,['rabbit@rmq-node1','rabbit@rmq-node2']}]},
 {running_nodes,['rabbit@rmq-node2','rabbit@rmq-node1']},
 {cluster_name,<<"rabbit@rmq-node1">>},
 {partitions,[]},
 {alarms,[{'rabbit@rmq-node2',[]},{'rabbit@rmq-node1',[]}]}]


[root@rmq-node2 ~]#  rabbitmqctl  cluster_status
Cluster status of node 'rabbit@rmq-node2'
[{nodes,[{disc,['rabbit@rmq-node1','rabbit@rmq-node2']}]},
 {running_nodes,['rabbit@rmq-node1','rabbit@rmq-node2']},
 {cluster_name,<<"rabbit@rmq-node1">>},
 {partitions,[]},
 {alarms,[{'rabbit@rmq-node1',[]},{'rabbit@rmq-node2',[]}]}]

node3做爲ram節點

未加入以前

[root@rmq-node3 ~]# rabbitmqctl cluster_status
Cluster status of node 'rabbit@rmq-node3'
[{nodes,[{disc,['rabbit@rmq-node3']}]},
 {running_nodes,['rabbit@rmq-node3']},
 {cluster_name,<<"rabbit@rmq-node3">>},
 {partitions,[]},
 {alarms,[{'rabbit@rmq-node3',[]}]}]

加入集羣

# stop application and reset
rabbitmqctl stop_app 
rabbitmqctl reset 

#join in cluster (specify only hostname, not with FQDN) 
rabbitmqctl join_cluster rabbit@rmq-node1 --ram

# start application
rabbitmqctl start_app

# show status
rabbitmqctl cluster_status

加入集羣后的狀態

[root@rmq-node1 ~]# rabbitmqctl  cluster_status
Cluster status of node 'rabbit@rmq-node1'
[{nodes,[{disc,['rabbit@rmq-node1','rabbit@rmq-node2']},
         {ram,['rabbit@rmq-node3']}]},
 {running_nodes,['rabbit@rmq-node3','rabbit@rmq-node2','rabbit@rmq-node1']},
 {cluster_name,<<"rabbit@rmq-node1">>},
 {partitions,[]},
 {alarms,[{'rabbit@rmq-node3',[]},
          {'rabbit@rmq-node2',[]},
          {'rabbit@rmq-node1',[]}]}]

-------------------
[root@rmq-node2 ~]# rabbitmqctl cluster_status
Cluster status of node 'rabbit@rmq-node2'
[{nodes,[{disc,['rabbit@rmq-node1','rabbit@rmq-node2']},
         {ram,['rabbit@rmq-node3']}]},
 {running_nodes,['rabbit@rmq-node3','rabbit@rmq-node1','rabbit@rmq-node2']},
 {cluster_name,<<"rabbit@rmq-node1">>},
 {partitions,[]},
 {alarms,[{'rabbit@rmq-node3',[]},
          {'rabbit@rmq-node1',[]},
          {'rabbit@rmq-node2',[]}]}]

-----------------------------

[root@rmq-node3 ~]# rabbitmqctl cluster_status
Cluster status of node 'rabbit@rmq-node3'
[{nodes,[{disc,['rabbit@rmq-node2','rabbit@rmq-node1']},
         {ram,['rabbit@rmq-node3']}]},
 {running_nodes,['rabbit@rmq-node1','rabbit@rmq-node2','rabbit@rmq-node3']},
 {cluster_name,<<"rabbit@rmq-node1">>},
 {partitions,[]},
 {alarms,[{'rabbit@rmq-node1',[]},
          {'rabbit@rmq-node2',[]},
          {'rabbit@rmq-node3',[]}]}]

登陸界面查看集羣狀態

http://192.168.35.135:15672/#/

---

相關概念

epmd

erlang port monitor deamon： 負責集羣中節點ip和port的監控；
集羣間node的發現都是經過該後臺程序來完成的；

Mnesia

這是erlang的分佈式數據庫，用於存儲queue、exchange、binding等信息；

---

網絡分區（network partitions）

可參考： https://www.cnblogs.com/ssslinppp/p/9470962.html

---

其餘運維操做

日誌文件

/var/log/rabbitmq/xxxx.log

將某個node從集羣中刪除

rabbitmqctl forget_cluster_node rabbit@node-1  //這個屬於硬刪除

將node1節點刪除後，如果從新加入該集羣，可能報錯，如：

inconsistent_cluster,"Node 'rabbit@node-1' thinks it's clustered with node 'rabbit@node-3', but 'rabbit@node-3' disagrees"}}

緣由：
從node-1的啓動報錯來看，像是集羣信息殘留；
在node-2上操做將node-1移除集羣，node-1的rabbitmq服務已經down掉了，因此數據庫沒法同步更新，記載的還是舊的集羣信息(數據庫記錄裏自身節點仍屬於集羣)
而node-2和node-3的數據庫記錄已經更新(數據庫信息裏面集羣不包含node-1節點了)
解決方式
刪除node-1的數據庫記錄

rm -rf /var/lib/rabbitmq/mnesia/*

---

出現的錯誤

1. 啓動失敗：rabbitmq-server.service: control process exited, code=exited status=75

journalctl -xe

# 錯誤信息以下
Aug 08 19:25:44 localhost.localdomain systemd[1]: rabbitmq-server.service: control process exited, code=exited status=75
Aug 08 19:25:44 localhost.localdomain systemd[1]: Failed to start RabbitMQ broker.
-- Subject: Unit rabbitmq-server.service has failed
-- Defined-By: systemd
-- Support: http://lists.freedesktop.org/mailman/listinfo/systemd-devel
-- 
-- Unit rabbitmq-server.service has failed.
-- 
-- The result is failed.
Aug 08 19:25:44 localhost.localdomain systemd[1]: Unit rabbitmq-server.service entered failed state.
Aug 08 19:25:44 localhost.localdomain systemd[1]: rabbitmq-server.service failed.
Aug 08 19:25:44 localhost.localdomain polkitd[902]: Unregistered Authentication Agent for unix-process:3504:33878 (system bus name :1.17, object path /org/freedesktop/PolicyKit1/Aut
[root@localhost mnesia]# journalctl -xe

解決方式：

chown rabbitmq:rabbitmq /var/lib/rabbitmq/.erlang.cookie
rm -rf /var/lib/rabbitmq/mnesia/*