rabbitmq+ keepalived+haproxy高可用集羣詳細命令

公司要用rabbitmq研究了兩週，特把

rabbitmq 高可用的研究成果備下

後續會更新封裝的類庫

安裝erlang

wget http://www.gelou.me/yum/erlang-18.3-1.el6.x86_64.rpm

yum install erlang-18.3-1.el6.x86_64.rpm

 

 

安裝rabbitmq rpm包：

wget http://www.gelou.me/yum/rabbitmq-server-3.6.1-1.noarch.rpm

rpm --import http://www.rabbitmq.com/rabbitmq-signing-key-public.asc

yum install rabbitmq-server-3.6.1-1.noarch.rpm

 

啓動rabbitmq，並驗證啓動狀況

rabbitmq-server --detached &ps aux |grep rabbitm

 

以服務的方式啓動

service rabbitmq-server start

 

 

開啓系列防火牆端口

/sbin/iptables -I INPUT -p tcp --dport 5672 -j ACCEPT       

/sbin/iptables -I INPUT -p tcp --dport 4369 -j ACCEPT 

/sbin/iptables -I INPUT -p tcp --dport 25672 -j ACCEPT 

/sbin/iptables -I INPUT -p tcp --dport 15672 -j ACCEPT 

iptables -I INPUT -p tcp --dport 8888 -j ACCEPT

iptables -I INPUT -p tcp --dport 5670 -j ACCEPT

iptables -I INPUT -p tcp --dport 5670 -j ACCEPT

/etc/rc.d/init.d/iptables save     

/etc/init.d/iptables restart      

/etc/init.d/iptables status

設置開機啓動

 rpm -aq |grep chkconfig

 

export PATH=/sbin:$PATH

chkconfig

 

chkconfig rabbitmq-server on

安裝web管理

rabbitmq-plugins enable rabbitmq_management   

 

訪問地址

http://172.20.16.57:15672/#/

 

 

 

 

 

重複安裝第二臺等

 

 

集羣命令重複安裝第二臺等

 

 

vim /etc/hosts

 

 

192.168.43.74 TD-YichehuiLinu

192.168.87.5  YichehuiLinux

 

重啓

reboot

 

 

chmod 700 /var/lib/rabbitmq/.erlang.cookie

echo -n "AZVOCZYZZBVFLBPTBXU" > /var/lib/rabbitmq/.erlang.cookie

chmod 400 /var/lib/rabbitmq/.erlang.cookie

ps -ef | grep ^rabbitmq | cut -c 9-16 | xargs kill -9

 

 

 

 

 

rabbitmq-server --detached &ps aux |grep rabbitm

 

機器全啓動後分別執行

rabbitmqctl stop_app    

 rabbitmqctl join_cluster rabbit@各機器名

rabbitmqctl start_app    

rabbitmqctl cluster_status

 

設置policy，以ha.開頭的隊列將會被鏡像到集羣其餘全部節點,一個節點掛掉而後重啓後會自動同步隊列消息

必須將整個RabbitMQ集羣的狀態設置爲鏡像模式， 節點都執行

rabbitmqctl set_policy ha-all-queue "^ha\." '{"ha-mode":"all","ha-sync-mode":"automatic"}'

 

備用命令

rabbitmqctl stop_app

rabbitmqctl change_cluster_node_type disc

或

rabbitmqctl change_cluster_node_type ram

rabbitmqctl start_app

 

 

 

添加用戶名admin,密碼admin

rabbitmqctl add_user zhang 1234

 

設置admin用戶tags爲管理員

rabbitmqctl set_user_tags zhang administrator

 

設置admin用戶配置、寫、讀的權限

rabbitmqctl set_permissions -p / zhang ".*" ".*" ".*"

 

 

 

haproxy搭建

 

   一、安裝haproxy

yum install haproxy

   二、安裝rsyslog

   a. 檢查rsyslog是否已安裝

rpm -q rsyslog

   b. 添加haproxy的log配置

cd /etc/rsyslog.d

vim haproxy.conf

=== 文件內容

$ModLoad imudp

$UDPServerRun 514

local2.* /var/log/haproxy.log

local3.* /var/log/haproxy.log

local10.* /var/log/haproxy.log

==========

   c. 編輯/etc/sysconfig/rsyslog

vim /etc/sysconfig/rsyslog

    添加內容：

SYSLOGD_OPTIONS="-c 2 -r -m 0"

   d. 建立日誌文件,並受權

cd /var/log

touch haproxy.log

chmod a+w haproxy.log

e. 重啓rsyslog服務用法：

/etc/init.d/rsyslog {start|stop|restart|condrestart|try-restart|reload|force-reload|status}

 /etc/init.d/rsyslog restart

   配置全部機器

四、配置haproxy

 > /etc/haproxy/haproxy.cfg

 

vim /etc/haproxy/haproxy.cfg

#---------------------------------------------------------------------

# Example configuration for a possible web application. See the

# full configuration options online.

#

# http://haproxy.1wt.eu/download/1.4/doc/configuration.txt

#

#---------------------------------------------------------------------

 

 

#---------------------------------------------------------------------

# Global settings

#---------------------------------------------------------------------

global

# to have these messages end up in /var/log/haproxy.log you will

# need to:

#

# 1) configure syslog to accept network log events. This is done

# by adding the '-r' option to the SYSLOGD_OPTIONS in

# /etc/sysconfig/syslog

#

# 2) configure local2 events to go to the /var/log/haproxy.log

# file. A line like the following can be added to

# /etc/sysconfig/syslog

#

# local2.* /var/log/haproxy.log

#

log 127.0.0.1 local2 notice

 

 

chroot /var/lib/haproxy

pidfile /var/run/haproxy.pid

maxconn 4000

user haproxy

group haproxy

daemon

 

 

# turn on stats unix socket

stats socket /var/lib/haproxy/stats

 

 

#---------------------------------------------------------------------

# common defaults that all the 'listen' and 'backend' sections will

# use if not designated in their block

#---------------------------------------------------------------------

defaults

mode tcp

option tcplog

option dontlognull

option http-server-close

option redispatch

retries 3

timeout http-request 10s

timeout queue 1m

timeout connect 10s

timeout client 1m

timeout server 1m

timeout http-keep-alive 10s

timeout check 10s

maxconn 3000

 

 

 

 

###haproxy statistics monitor by laijingli 20160222

listen statics 0.0.0.0:8888

mode http

log 127.0.0.1 local0 debug

transparent

stats refresh 60s

stats uri / haproxy-stats

stats realm Haproxy \ statistic

stats auth zhang:1234

 

 

#---------------------------------------------------------------------

# main frontend which proxys to the backends

#---------------------------------------------------------------------

listen rabbitmq_cluster

bind 0.0.0.0:5670

mode tcp

balance roundrobin

 

server ych51  172.20.16.51:5672 check inter 5000 rise 2 fall 2

server ych50  172.20.16.50:5672 check inter 5000 rise 2 fall 2

 

 

server TD-YichehuiLinu  192.168.43.74:5672 check inter 5000 rise 2 fall 2

server YichehuiLinux  192.168.87.5:5672 check inter 5000 rise 2 fall 2

server YichehuiLinux  192.168.87.5:9200 check inter 5000 rise 2 fall 2

五、 啓動haproxy

/etc/init.d/haproxy restart

六、 備用命令中止haproxy

killall haproxy

 

http://192.168.87.5:8888/

 

http://192.168.43.74:15672/#/

 

 

3、安裝和設置keepalived

一、安裝keepalived

 yum install keepalived

 chkconfig --add keepalived

chkconfig --level 35 keepalived on

二、修改配置文件

配置keepalived

>  /etc/keepalived/keepalived.conf

vim  /etc/keepalived/keepalived.conf

內容：

=====================================================================

! Configuration File for keepalived

# 全局配置，demo中配置了郵件信息，此處由於調試中，暫時沒有添加

global_defs {

}

# 集羣資源監控，組合track_script進行

vrrp_script check_haproxy {

script "killall -0 haproxy"

interval 2

}

vrrp_instance HAPROXY_HA {

# 設置當前主機爲主節點，若是是備用節點，則設置爲BACKUP

# 備用節點時，設置爲：

# state BACKUP

state MASTER

# 指定HA監測網絡接口，能夠用ifconfig查看來決定設置哪個

interface eth1

# 虛擬路由標識，同一個VRRP實例要使用同一個標識，主備機

virtual_router_id 80

# 由於當前環境中VRRP組播有問題，改成使用單播發送VRRP報文

# 這個地方須要關注，以前未作此設置，結果主備節點互相不能發現，所以主備節點都升級成了MASTER，而且綁定了VIP

# 備用節點時，內容爲：

# unicast_src_ip 192.168.200.199

# unicast_peer {

# 192.168.200.200

# }

 

# 設置優先級，確保主節點的優先級高過備用節點

# 備用節點時，設置爲：

priority 80

# priority 100

# 用於設定主備節點間同步檢查時間間隔

advert_int 2

# 設置高可用集羣中不搶佔功能，在主機down後，從機接管，當主機從新恢復後，設置此功能，備機將繼續提供服務，從而避免因切換致使的隱患

nopreempt

# 設置主備節點間的通訊驗證類型及密碼，同一個VRRP實例中需一致

authentication {

auth_type PASS

auth_pass 1234

}

# 當keepalived切換狀態到MASTER時，執行腳本

notify_master "/etc/keepalived/master.sh"

# 當keepalived切換狀態到BACKUP時，執行腳本

notify_backup "/etc/keepalived/backup.sh"

# 當keepalived切換狀態到FAULT時，執行腳本

notify_fault "/etc/keepalived/fault.sh"

# 當keepalived切換狀態到STOP時，執行腳本

notify_fault "/etc/keepalived/stop.sh"

# 集羣資源監控，組合vrrp_script進行

track_script {

check_haproxy

}

# 設置虛擬IP地址，當keepalived狀態切換爲MASTER時，此IP會自動添加到系統中

# 當狀態切換到BACKUP時，此IP會自動從系統中刪除

# 能夠經過命令ip add查看切換後的狀態

virtual_ipaddress {

192.168.43.201

}

}

====================================================================

 

 配置所有

三、啓動服務

由於是爲了實現haproxy的高可用，啓動時須要順序啓動：

(1) 啓動haproxy：

haproxy -f /etc/haproxy/haproxy.cfg

(2) 啓動keeepalived：先啓動master節點，後啓動BACKUP節點

/etc/init.d/keepalived restart

四、中止keeepalived服務

/etc/init.d/keepalived stop

 

 

大哥廣告 閣樓