005.HAProxy+Keepalived高可用負載均衡

一 基礎準備

1.1 部署環境及說明

系統OS：CentOS 6.8 64位

HAProxy軟件：HA-Proxy version 1.5.18

Keepalived軟件：keepalived-1.3.6.tar.gz

官方連接：http://www.haproxy.org/ （國內可能沒法打開）

下載鏈接：http://pkgs.fedoraproject.org/repo/pkgs/haproxy/

部署說明：當用戶訪問對應的域名時，HAProxy能將請求發送到對應的後端主機上，同時當主HAProxy服務器發生故障後，能馬上將負載均衡服務切換到備用HAProxy服務器上。

主機名	主機IP地址	集羣角色	虛機IP/域名
master	172.24.8.10	主HAProxy服務器	172.24.8.100
backup	172.24.8.11	備HAProxy服務器
webapp1	172.24.8.30	後端web服務器	www.lz.com
webapp2	172.24.8.31		static.lz.com
webapp3	172.24.8.32		video.lz.com

1.2 架構規劃

二 後端httpd集羣部署

2.1 部署httpd集羣

  1 [root@webapp1 ~]# yum -y install httpd
  2 [root@webapp1 ~]# vi /var/www/html/index.html
  3 This is my www.lz.com!
  4 [root@webapp1 ~]# vi /var/www/html/index.html
  5 This is my static.lz.com!
  6 [root@webapp1 ~]# vi /var/www/html/index.html
  7 This is my video.lz.com!
  8 [root@webapp1 ~]# systemctl start httpd.service
  9 [root@webapp1 ~]# systemctl enable httpd.service
 10 [root@webapp1 ~]# systemctl stop firewalld.service
 11 [root@webapp1 ~]# systemctl disable firewalld.service
 12 [root@webapp1 ~]# vi /etc/selinux/config
 13 SELINUX=disabled
 14 [root@webapp1 ~]# setenforce 0			#關閉SELinux及防火牆

 

注意：後端real server節點都須要安裝，本環境針對httpd簡單安裝便可，無需過多配置。

三 基礎NTP部署

3.1 NTP部署

  1 [root@webapp1 ~]# yum -y install ntp
  2 [root@webapp1 ~]# systemctl start  ntpd.service

 

建議：替換附件中的ntp配置文件，建議採用阿里雲時鐘進行同步。

注意：爲了保證集羣的穩定性，強烈建議在全部節點均部署NTP同步服務，保證全部時鐘一致。

四 Keepalived部署

4.1 編譯環境

安裝基礎環境及依賴：

  1 # yum -y install gcc gcc-c++ make kernel-devel kernel-tools kernel-tools-libs kernel libnl libnl-devel libnfnetlink-devel openssl-devel wget openssh-clients

4.2 安裝Keepalived

  1 [root@haproxy_master ~]# wget http://www.keepalived.org/software/keepalived-1.3.6.tar.gz
  2 [root@haproxy_master ~]# tar -zxvf keepalived-1.3.6.tar.gz
  3 [root@haproxy_master ~]# cd keepalived-1.3.6/
  4 [root@lvsmaster keepalived-1.3.6]# ./configure --prefix=/usr/local/keepalived
  5 [root@haproxy_master keepalived-1.3.6]# make && make install

 

注意：CentOS6.8安裝高於1.3.6版本會出現未知錯誤。

4.3 添加Keepalived啓動相關服務

  1 [root@haproxy_master ~]# mkdir /etc/keepalived
  2 [root@haproxy_master ~]# cp /usr/local/keepalived/etc/keepalived/keepalived.conf /etc/keepalived/
  3 [root@haproxy_master ~]# cp /usr/local/keepalived/etc/sysconfig/keepalived /etc/sysconfig/
  4 [root@haproxy_master ~]# cp /usr/local/keepalived/sbin/keepalived /usr/sbin/
  5 [root@haproxy_master ~]# vi /etc/init.d/keepalived	#建立Keepalived啓動腳本，見附件
  6 [root@haproxy_master ~]# chmod u+x /etc/rc.d/init.d/keepalived

 

4.4 配置Keepalived

  1 [root@haproxy_master ~]# vi /etc/keepalived/keepalived.conf
  2 global_defs {
  3    notification_email {
  4 #……
  5    }
  6    notification_email_from Alexandre.Cassen@firewall.loc
  7    smtp_server 192.168.200.1
  8    smtp_connect_timeout 30
  9    router_id LVS_DEVEL
 10 }
 11 
 12 vrrp_script check_haproxy {
 13     script "/usr/bin/killall -0 haproxy"
 14     interval    2
 15     weight      21
 16     }
 17 
 18 vrrp_instance HAProxy_HA {
 19     state BACKUP			#在HAProxy主備均設置爲BACKUP
 20     interface eth0
 21     virtual_router_id 80
 22     priority 100
 23     advert_int 2
 24     nopreempt				#不搶佔模式
 25     authentication {
 26         auth_type PASS
 27         auth_pass 1111
 28     }
 29 
 30     notify_master "/etc/keepalived/master.sh"
 31     notify_backup "/etc/keepalived/backup.sh"
 32     notify_fault "/etc/keepalived/fault.sh"
 33 
 34     track_script {
 35     check_haproxy
 36     }
 37 
 38     virtual_ipaddress {
 39         172.24.8.100 dev eth0
 40     }
 41 }

 

4.5 編寫Keepalived腳本

  1 [root@haproxy_master ~]# vi /etc/keepalived/master.sh
  2 #!/bin/bash
  3 LOGFILE=/var/log/keepalived-mysql-state/log
  4 date >>$LOGFILE
  5 echo "[Master]" >>$LOGFILE
  6 [root@haproxy_master ~]# vi /etc/keepalived/backup.sh
  7 #!/bin/bash
  8 LOGFILE=/var/log/keepalived-mysql-state/log
  9 date >>$LOGFILE
 10 echo "[BACKUP]" >>$LOGFILE
 11 [root@haproxy_master ~]# vi /etc/keepalived/fault.sh
 12 #!/bin/bash
 13 LOGFILE=/var/log/keepalived-mysql-state/log
 14 date >>$LOGFILE
 15 echo "[FAULT]" >>$LOGFILE
 16 [root@haproxy_master ~]# chmod u+x /etc/keepalived/backup.sh
 17 [root@haproxy_master ~]# chmod u+x /etc/keepalived/master.sh
 18 [root@haproxy_master ~]# chmod u+x /etc/keepalived/fault.sh

 

提示：本環境採用測試腳本，真實環境建議採用自動發送郵件通知運維員的腳本。

4.5 Bakcup節點配置

  1 [root@haproxy_master ~]# scp /etc/keepalived/keepalived.conf 172.24.8.11:/etc/keepalived/keepalived.conf    #將配置好的Master節點的配置文件複製到Backup節點
  2 [root@haproxy_slave ~]# vi /etc/keepalived/keepalived.conf
  3 state BACKUP
  4 priority 80

 

注意：在HAProxy備節點也設置爲BACKUP，priority修改成低於HAProxy主角色的優先級便可，同時去掉nopreempt。

  1 [root@haproxy_master ~]# scp /etc/keepalived/*.sh 172.24.8.11:/etc/keepalived/
  2 #將對應的腳本也複製至backup節點。

 

五 HAProxy部署

5.1 HAProxy安裝

  1 [root@haproxy_master ~]# yum -y install haproxy

提示：須要在主備HAProxy節點都安裝。

5.2 HAProxy配置

  1 [root@haproxy_master ~]# vi /etc/haproxy/haproxy.cfg
  2 global
  3     log         127.0.0.1 local0 info
  4     chroot      /var/lib/haproxy
  5     pidfile     /var/run/haproxy.pid
  6     maxconn     4096
  7     user        haproxy
  8     group       haproxy
  9     daemon
 10     nbproc      1
 11 defaults
 12     mode                    http
 13     log                     global
 14     retries                 3
 15     timeout connect         5s
 16     timeout client          30s
 17     timeout server          30s
 18     timeout check           2s
 19 frontend        www
 20     bind        172.24.8.100:80
 21     mode        http
 22     option      httplog
 23     option      forwardfor
 24     log         global
 25 
 26     acl host_www        hdr_dom(host)   -i  www.lz.com	#配置不一樣域名分發不一樣後端策略
 27     acl host_static     hdr_dom(host)   -i  static.lz.com
 28     acl host_video      hdr_dom(host)   -i  video.lz.com
 29 
 30     use_backend     server_www      if      host_www	#配置不一樣域名分發不一樣後端策略
 31     use_backend     server_static   if      host_static
 32     use_backend     server_video    if      host_video
 33 backend         server_www		#後端真是服務器
 34     mode        http
 35     option      redispatch
 36     option      abortonclose
 37     balance     roundrobin
 38     option      httpchk GET /index.html
 39     server      webapp1 172.24.8.30:80  weight 6 check inter 2000 rise 2 fall 3
 40 backend         server_static		#後端真是服務器
 41     mode        http
 42     option      redispatch
 43     option      abortonclose
 44     balance     roundrobin
 45     option      httpchk GET /index.html
 46     server      webapp2 172.24.8.31:80  weight 6 check inter 2000 rise 2 fall 3
 47 backend         server_video		#後端真是服務器
 48     mode        http
 49     option      redispatch
 50     option      abortonclose
 51     balance     roundrobin
 52     option      httpchk GET /index.html
 53     server      webapp3 172.24.8.32:80  weight 6 check inter 2000 rise 2 fall 3
 54 [root@haproxy_master ~]# scp /etc/haproxy/haproxy.cfg root@172.24.8.11:/etc/haproxy/haproxy.cfg		#將配置文件複製至HAProxy備節點

 

5.3 打開轉發

  1 [root@haproxy_master ~]# vi /etc/sysctl.conf
  2 net.ipv4.ip_nonlocal_bind = 1
  3 [root@haproxy_master ~]# sysctl -p

 

注意：綁定非本機的IP必須在sysctl.conf文件中配置。

六 啓動服務

  1 [root@webapp1 ~]# systemctl start httpd

提示：三個節點的httpd服務均啓動。

  1 [root@haproxy_master ~]# service haproxy start

提示：必須先啓動HAProxy服務，以後啓動Keepalived，由於Keepalived會先檢測HAProxy服務進程。

  1 [root@haproxy_master ~]# service keepalived start

七 驗證測試

7.1 高可用驗證

  1 [root@haproxy_master ~]# ip add

  1 [root@haproxy_master ~]# service haproxy stop		#中止HAProxy主節點的HAProxy進程
  2 [root@haproxy_master ~]# tail -f /var/log/messages	#觀察HAProxy主節點日誌

 

  1 [root@haproxy_slave ~]# ip addr				#查看備HAProxy節點的IP

結論：經過測試可知當主HAProxy節點服務異常，Keepalived會檢測到，同時HAProxy會將vip從主節點移除，備HAProxy會接管。

  1 [root@haproxy_master ~]# service haproxy start

結論：因爲配置了非搶佔模式，主HAProxy恢復進程以後，依舊由備HAProxy提供服務，vip不會切回至主HAProxy。

7.2 負載均衡測試

主機hosts中添加以下解析：

  1 172.24.8.100	www.lz.com
  2 172.24.8.100	static.lz.com
  3 172.24.8.100	video.lz.com

 

瀏覽器分別訪問不用的三個域名：