[置頂] RabbitMQ概念及環境搭建（三）RabbitMQ cluster

http://blog.csdn.net/zyz511919766/article/details/41896747

 

 

測試環境：VMS00781 VMS00782 VMS00386 (centos5.8)
1.先在三臺機器上分別安裝RabbitMQ Server

2.讀取其中一個節點的cookie，並複製到其餘節點（節點間經過cookie肯定相互是否可通訊）
二者之一都可：
sudo vim /var/lib/rabbitmq/.erlang.cookie
sudo vim $HOME/.erlang.cookie

3.逐個啓動節點
sudo service rabbitmq-server start

4.查看各節點中的RabbitMQ brokers
sudo rabbitmqctl cluster_status

5.建集羣
分別在VMS0038六、VMS00782 上執行
sudo rabbitmqctl stop_app
sudo rabbitmqctl join_cluster --ram rabbit@VMS00781
sudo rabbitmqctl start_app
sudo rabbitmqctl stop_app
sudo rabbitmqctl join_cluster rabbit@VMS00781
sudo rabbitmqctl start_app

6.排錯
建集羣過程當中碰到以下錯誤：
sudo rabbitmqctl join_cluster --ram rabbit@VMS00386
Clustering node rabbit@VMS00782 with rabbit@VMS00386 ...
Error: unable to connect to nodes [rabbit@VMS00386]: nodedown
DIAGNOSTICS
===========
attempted to contact: [rabbit@VMS00386]
rabbit@VMS00386:
  * unable to connect to epmd (port 4369) on VMS00386: nxdomain (non-existing domain)
current node details:
- node name: 'rabbitmqctl-8666@VMS00782'
- home dir: /var/lib/rabbitmq
- cookie hash: 50YO3zK+HJHos0tab1vHjg==
解決之道：
集羣節點間需能互相訪問，故每一個集羣節點的hosts文件應包含集羣內全部節點的信息以保證互相解析
vim /etc/hosts
781's IP   VMS00781
782's IP   VMS00782
386's IP   vms00386
以後重啓各節點中的rabbitmq

7.其餘問題

cluster搭建起來後若在web管理工具中rabbitmq_management的Overview的Nodes部分看到「Node statistics not available」的信息，說明在該節點上web管理插件還未啓用。

直接在顯示提示信息的節點上運行sudo rabbitmq-plugins enable rabbitmq_management便可。

 

Error: mnesia_unexpectedly_running
緣由：忘記先中止stop_app
解決：sudo rabbitmqctl stop_app

若rabbitmq-server第一次啓動後hostname不能被解析或者發生了更改則會致使啓動失敗
需執行以下操做
sudo rm -rf /var/lib/rabbitmq/mnesia（由於相關信息會記錄在此數據庫）
重裝RabbitMQ Server

#####################################################
RabbitMQ cluster 管理
#####################################################
1.查看集羣狀態
可分別在集羣中各個節點執行
sudo rabbitmqctl cluster_status

2.更改節點類型（內存型或磁盤型）
sudo rabbitmqctl stop_app
sudo rabbitmqctl change_cluster_node_type disc
或
sudo rabbitmqctl change_cluster_node_type ram
sudo rabbitmqctl start_app

3.重啓cluster中的節點
中止某個節點或者節點down掉剩餘節點不受影響
[op1@vms00386 ~]$ sudo rabbitmqctl stop
Stopping and halting node rabbit@vms00386 ...

[op1@VMS00781 ~]$ sudo rabbitmqctl cluster_status
Cluster status of node rabbit@VMS00781 ...
[{nodes,[{disc,[rabbit@VMS00781,rabbit@VMS00782,rabbit@vms00386]}]},
 {running_nodes,[rabbit@VMS00782,rabbit@VMS00781]},
 {cluster_name,<<"rabbit@VMS00781">>},
 {partitions,[]}]

[op1@VMS00782 ~]$ sudo rabbitmqctl cluster_status
Cluster status of node rabbit@VMS00782 ...
[{nodes,[{disc,[rabbit@VMS00781,rabbit@VMS00782,rabbit@vms00386]}]},
 {running_nodes,[rabbit@VMS00781,rabbit@VMS00782]},
 {cluster_name,<<"rabbit@VMS00781">>},
 {partitions,[]}]

[op1@VMS00782 ~]$ sudo rabbitmqctl stop
Stopping and halting node rabbit@VMS00782 ...

[op1@VMS00781 ~]$ sudo rabbitmqctl cluster_status
Cluster status of node rabbit@VMS00781 ...
[{nodes,[{disc,[rabbit@VMS00781,rabbit@VMS00782,rabbit@vms00386]}]},
 {running_nodes,[rabbit@VMS00781]},
 {cluster_name,<<"rabbit@VMS00781">>},
 {partitions,[]}]

待節點重啓後自動追上其餘節點
[op1@vms00386 ~]$ sudo service rabbitmq-server start
Starting rabbitmq-server: SUCCESS
rabbitmq-server.

[op1@VMS00781 ~]$ sudo rabbitmqctl cluster_status
Cluster status of node rabbit@VMS00781 ...
[{nodes,[{disc,[rabbit@VMS00781,rabbit@VMS00782,rabbit@vms00386]}]},
 {running_nodes,[rabbit@vms00386,rabbit@VMS00781]},
 {cluster_name,<<"rabbit@VMS00781">>},
 {partitions,[]}]

[op1@VMS00782 ~]$ sudo service rabbitmq-server start
Starting rabbitmq-server: SUCCESS
rabbitmq-server.

[op1@VMS00781 ~]$ sudo rabbitmqctl cluster_status
Cluster status of node rabbit@VMS00781 ...
[{nodes,[{disc,[rabbit@VMS00781,rabbit@VMS00782,rabbit@vms00386]}]},
 {running_nodes,[rabbit@VMS00782,rabbit@vms00386,rabbit@VMS00781]},
 {cluster_name,<<"rabbit@VMS00781">>},
 {partitions,[]}]

[op1@VMS00782 ~]$ sudo rabbitmqctl cluster_status
Cluster status of node rabbit@VMS00782 ...
[{nodes,[{disc,[rabbit@VMS00781,rabbit@VMS00782,rabbit@vms00386]}]},
 {running_nodes,[rabbit@VMS00781,rabbit@vms00386,rabbit@VMS00782]},
 {cluster_name,<<"rabbit@VMS00781">>},
 {partitions,[]}]

[op1@vms00386 ~]$ sudo rabbitmqctl cluster_status
Cluster status of node rabbit@vms00386 ...
[{nodes,[{disc,[rabbit@VMS00781,rabbit@VMS00782,rabbit@vms00386]}]},
 {running_nodes,[rabbit@VMS00782,rabbit@VMS00781,rabbit@vms00386]},
 {cluster_name,<<"rabbit@VMS00781">>},
 {partitions,[]}]

幾點注意：
保證集羣中至少有一個磁盤類型的節點以防數據丟失，在更改節點類型時尤爲要注意。
若整個集羣被停掉了，應保證最後一個down掉的節點被最早啓動，若不能則要使用forget_cluster_node命令將其移出集羣
若集羣中節點幾乎同時以不可控的方式down了此時在其中一個節點使用force_boot命令重啓節點

4.從集羣移除節點
[op1@vms00386 ~]$ sudo rabbitmqctl stop_app
Stopping node rabbit@vms00386 ...
[op1@vms00386 ~]$ sudo rabbitmqctl reset
Resetting node rabbit@vms00386 ...
[op1@vms00386 ~]$ sudo rabbitmqctl start_app
Starting node rabbit@vms00386 ...

[op1@vms00386 ~]$ sudo rabbitmqctl cluster_status
Cluster status of node rabbit@vms00386 ...
[{nodes,[{disc,[rabbit@vms00386]}]},
 {running_nodes,[rabbit@vms00386]},
 {cluster_name,<<"rabbit@vms00386">>},
 {partitions,[]}]

[op1@VMS00781 ~]$ sudo rabbitmqctl cluster_status
Cluster status of node rabbit@VMS00781 ...
[{nodes,[{disc,[rabbit@VMS00781,rabbit@VMS00782]}]},
 {running_nodes,[rabbit@VMS00782,rabbit@VMS00781]},
 {cluster_name,<<"rabbit@VMS00781">>},
 {partitions,[]}]

[op1@VMS00782 ~]$ sudo rabbitmqctl cluster_status
Cluster status of node rabbit@VMS00782 ...
[{nodes,[{disc,[rabbit@VMS00781,rabbit@VMS00782]}]},
 {running_nodes,[rabbit@VMS00781,rabbit@VMS00782]},
 {cluster_name,<<"rabbit@VMS00781">>},
 {partitions,[]}]
可見rabbit@vms00386成爲了獨立的節點，原集羣只剩rabbit@VMS00781,rabbit@VMS00782了

也可在某個節點移除集羣中其餘節點
如繼續在rabbit@VMS00781上移除rabbit@VMS00782
[op1@VMS00781 ~]$ sudo rabbitmqctl forget_cluster_node rabbit@VMS00782
Removing node rabbit@VMS00782 from cluster ...

[op1@VMS00781 ~]$ sudo rabbitmqctl cluster_status
Cluster status of node rabbit@VMS00781 ...
[{nodes,[{disc,[rabbit@VMS00781]}]},
 {running_nodes,[rabbit@VMS00781]},
 {cluster_name,<<"rabbit@VMS00781">>},
 {partitions,[]}]

可見集羣只剩rabbit@VMS00781一個節點了

這裏有個問題，在遠程其餘節點中被移除的節點會自認爲仍屬於集羣

[op1@VMS00782 ~]$ sudo rabbitmqctl start_app
Starting node rabbit@VMS00782 ...
BOOT FAILED
===========
Error description:
   {error,{inconsistent_cluster,"Node rabbit@VMS00782 thinks it's clustered with node rabbit@VMS00781, but rabbit@VMS00781 disagrees"}}
Log files (may contain more information):
   /var/log/rabbitmq/rabbit@VMS00782.log
   /var/log/rabbitmq/rabbit@VMS00782-sasl.log
Stack trace:
   [{rabbit_mnesia,check_cluster_consistency,0},
    {rabbit,'-start/0-fun-0-',0},
    {rabbit,start_it,1},
    {rpc,'-handle_call_call/6-fun-0-',5}]
Error: {rabbit,failure_during_boot,
           {error,
               {inconsistent_cluster,
                   "Node rabbit@VMS00782 thinks it's clustered with node rabbit@VMS00781, but rabbit@VMS00781 disagrees"}}}
須要重置一下
[op1@VMS00782 ~]$ sudo rabbitmqctl reset
Resetting node rabbit@VMS00782 ...
[op1@VMS00782 ~]$ sudo rabbitmqctl start_app
Starting node rabbit@VMS00782 ...

此時三個節點均已成爲獨立的節點
其中rabbit@vms0038六、rabbit@VMS00782均被重置爲了新的RabbitMQ broker而rabbit@VMS00781還保留着原cluster的殘留狀態可經過以下步驟重置
[op1@VMS00781 ~]$ sudo rabbitmqctl stop_app
Stopping node rabbit@VMS00781 ...
[op1@VMS00781 ~]$ sudo rabbitmqctl reset
Resetting node rabbit@VMS00781 ...
[op1@VMS00781 ~]$ sudo rabbitmqctl start_app
Starting node rabbit@VMS00781 ...

5.自動配置cluster
顯然，這是經過配置文件而非命令行工具進行的
首先重置各節點
[op1@VMS00781 ~]$ sudo rabbitmqctl stop_app
Stopping node rabbit@VMS00781 ...
[op1@VMS00781 ~]$ sudo rabbitmqctl reset
Resetting node rabbit@VMS00781 ...
...
其次調整配置文件
[{rabbit,
  [{cluster_nodes, {['rabbit@VMS00781', 'rabbit@VMS00782', 'rabbit@vms00386'], disc}}]}].
...
以後啓動各節點
[op1@VMS00781 ~]$ sudo service rabbitmq-server start
Starting rabbitmq-server: SUCCESS
rabbitmq-server.

查看集羣狀態
[op1@VMS00781 ~]$ sudo rabbitmqctl cluster_status

幾點注意：
不管經過命令行仍是經過配置文件配置，請確保各節點上Erlang和RabbitMQ版本一致
配置文件僅對新鮮節點有效，也即被reset或者第一次啓動的節點。所以在重啓節點後自動化集羣過程並不會發生。也覺得這經過rabbitmq進行的改變優先於自動化集羣配置。

在一臺機器上部署集羣，通常用戶測試集羣特性
這裏的關鍵是已不一樣的端口可節點名稱啓動多個rabbitmq-server實例，其他過程同多機器上部署集羣相似

其餘注意事項：
如防火牆策略等

參考：
http://www.rabbitmq.com/clustering.html