VRRP沒法切換VIP的問題分析:
Keepalived的配置文件:
/etc/kolla/keepalived/keepalived.conf
當中,nopreempt選項是影響切換的因素之一,另外一個因素則是:
vrrp_instance kolla_internal_vip_51 {
...
track_script {
check_alive
}
}
而check_alive的定義是:
vrrp_script check_alive {
script "/check_alive.sh"
interval 2
fall 2
rise 10
}
即Keepalived根據健康檢查腳本/check_alive.sh的返回值決定是否成爲主路由器。
/check_alive.sh腳本檢查本地HAProxy是否在運行,而本地HAProxy沒有在運行,腳本的健康檢查結果是失敗。
因此VIP沒法切換。
本地HAProxy沒有在運行,本地haproxy.cfg配置要求監聽VIP,而此時本地網卡未配置VIP,致使HAProxy沒法監聽。
所以只能手動先把VIP轉移:
docker stop keepalived
docker stop haproxy
ip addr del 172.24.9.198/32 dev enp2s0f0
ip addr add 172.24.9.198/32 dev enp2s0f0
docker start haproxy
去掉nopreempt選項:
docker start keepalived
此處備忘如何清楚網卡的全部IP地址:
ip addr flush dev eth0
***** ***** ***** ***** ***** ***** ***** ***** ***** ***** ***** ***** ***** ***** ***** *****
查看HAProxy的啓動日誌:
[root@node2 ~]# docker logs haproxy
...
[ALERT] 123/132247 (11) : Starting proxy rabbitmq_management: cannot bind socket [172.24.9.198:15672]
[ALERT] 123/132247 (11) : Starting proxy keystone_internal: cannot bind socket [172.24.9.198:5000]
[ALERT] 123/132247 (11) : Starting proxy keystone_admin: cannot bind socket [172.24.9.198:35357]
[ALERT] 123/132247 (11) : Starting proxy glance_registry: cannot bind socket [172.24.9.198:9191]
[ALERT] 123/132247 (11) : Starting proxy glance_api: cannot bind socket [172.24.9.198:9292]
[ALERT] 123/132247 (11) : Starting proxy nova_api: cannot bind socket [172.24.9.198:8774]
[ALERT] 123/132247 (11) : Starting proxy nova_metadata: cannot bind socket [172.24.9.198:8775]
[ALERT] 123/132247 (11) : Starting proxy placement_api: cannot bind socket [172.24.9.198:8780]
[ALERT] 123/132247 (11) : Starting proxy nova_novncproxy: cannot bind socket [172.24.9.198:6080]
[ALERT] 123/132247 (11) : Starting proxy neutron_server: cannot bind socket [172.24.9.198:9696]
[ALERT] 123/132247 (11) : Starting proxy horizon: cannot bind socket [172.24.9.198:80]
[ALERT] 123/132247 (11) : Starting proxy cinder_api: cannot bind socket [172.24.9.198:8776]
[ALERT] 123/132247 (11) : Starting proxy heat_api: cannot bind socket [172.24.9.198:8004]
[ALERT] 123/132247 (11) : Starting proxy heat_api_cfn: cannot bind socket [172.24.9.198:8000]
[ALERT] 123/132247 (11) : Starting proxy mariadb: cannot bind socket [172.24.9.198:3306]
[ALERT] 123/132247 (11) : Starting proxy rabbitmq: cannot bind socket [172.24.9.198:5672]
從日誌看出,HAProxy沒法在VIP進行監聽。
經確認,內核參數「net.ipv4.ip_nonlocal_bind=1」使得HAProxy可以在非本地的IP地址進行監聽。
檢查發現此內核參數已經設置開機啓動:
[root@node2 ~]# grep ip_nonlocal_bind /etc/sysctl.conf
net.ipv4.ip_nonlocal_bind=1
檢查內核運行過程當中此參數的狀態:
[root@node2 ~]# cat /proc/sys/net/ipv4/ip_nonlocal_bind
0
發現此參數爲0,不知被誰篡改。
設置爲1:
[root@node2 ~]# echo 1 >/proc/sys/net/ipv4/ip_nonlocal_bind
重啓haproxy容器:
[root@node2 ~]# docker stop haproxy;docker start haproxy
驗證HAProxy已經可以在VIP進行監聽:
[root@node2 ~]# ss -lnp|grep '\.198:'
tcp LISTEN 0 128 172.24.9.198:3306 *:* users:(("haproxy",pid=10784,fd=21))
tcp LISTEN 0 128 172.24.9.198:8780 *:* users:(("haproxy",pid=10784,fd=14))
tcp LISTEN 0 128 172.24.9.198:9292 *:* users:(("haproxy",pid=10784,fd=11))
tcp LISTEN 0 128 172.24.9.198:80 *:* users:(("haproxy",pid=10784,fd=17))
node