HAPROXY+KEEPALIVED實現負載均衡

時間  2019-12-11
標籤 haproxy+keepalived haproxy keepalived 實現 負載 均衡 欄目 負載均衡 简体版
原文   原文鏈接

一 原理簡介

1.HAProxy
HAProxy提供高可用性、負載均衡以及基於TCP和HTTP應用的代理，支持虛擬主機，它是免費、快速而且可靠的一種解決方案。HAProxy特別適用於那些負載特大的web站點，這些站點一般又須要會話保持或七層處理。HAProxy運行在時下的硬件上，徹底能夠支持數以萬計的併發鏈接。而且它的運行模式使得它能夠很簡單安全的整合進當前的架構中， 同時能夠保護web服務器不被暴露到網絡上。
2.Keepalived
Keepalived 是一個基於VRRP協議來實現的LVS服務高可用方案，能夠利用其來避免單點故障。一個LVS服務會有2臺服務器運行Keepalived，一臺爲主服務器（MASTER），一臺爲備份服務器（BACKUP），可是對外表現爲一個虛擬IP，主服務器會發送特定的消息給備份服務器，當備份服務器收不到這個消息的時候，即主服務器宕機的時候，備份服務器就會接管虛擬IP，繼續提供服務，從而保證了高可用性。Keepalived是VRRP的完美實現。
3.vrrp協議
在現實的網絡環境中，兩臺須要通訊的主機大多數狀況下並無直接的物理鏈接。對於這樣的狀況，它們之間路由怎樣選擇？主機如何選定到達目的主機的下一跳路由，這個問題一般的解決方法有兩種：
 在主機上使用動態路由協議(RIP、OSPF等) 
 在主機上配置靜態路由 
很明顯，在主機上配置動態路由是很是不切實際的，由於管理、維護成本以及是否支持等諸多問題。配置靜態路由就變得十分流行，但路由器(或者說默認網關default gateway)卻常常成爲單點故障。VRRP的目的就是爲了解決靜態路由單點故障問題，VRRP經過一競選(election)協議來動態的將路由任務交給LAN中虛擬路由器中的某臺VRRP路由器。javascript


二 運行環境

1.系統
CentOS Linux release 7.2.1511 (Core)
2.應用軟件
haproxy-1.5.14-3.el7.x86_64
keepalived-1.2.13-7.el7.x86_64
httpd-2.4.6-45.el7.centos.x86_64
php-5.4.16-42.el7.x86_64
mariadb-server-5.5.52-1.el7.x86_64
mariadb.x86_64
wordpress-4.3.1-zh_CN.zip
nfs-utils-1.3.0-0.33.el7.x86_64
rpcbind-0.2.0-38.el7.x86_64
3.IP配置
負載均衡器
DIP1:172.18.67.13
DIP2:172.18.67.14
後端Real Server
RIP1:172.18.67.11
RIP2:172.18.67.12
數據庫服務器
MIP:172.18.67.1
客戶端IP
IP:172.18.67.3
VIP:172,.18.67.33php


三 架構拓撲及應用軟件安裝

1.拓撲圖css


2.部署應用軟件
在IP爲172.18.67.13與172.18.67.14的服務器上安裝部署haproxy、keepalivedhtml

1
2
[root@inode2 ~] # yum install haproxy keepalived -y
[root@inode3 ~] # yum install haproxy keepalived -y


在IP爲172.18.67.11與172.18.67.12的服務器上安裝部署httpd、phpjava

1
2
[root@inode4 ~] # yum install httpd php -y
[root@inode5 ~] # yum install httpd php -y


在IP爲172.18.67.1的服務器上部署mariadb、mariadb-server、php-mysqlnode

1
[root@inode6 ~] # yum install mariadb mariadb-server php-mysql -y


另外因爲數據庫採用文件共享的方式，因此還要安裝nfs，分別在Real Server端和數據庫服務器端安裝nfs的應用軟件nfs-utils、rpcbindmysql

1
2
3
[root@inode4 ~] # yum install nfs-utils rpcbind -y
[root@inode5 ~] # yum install nfs-utils rpcbind -y
[root@inode6 ~] # yum install nfs-utils rpcbind -y

 

四 配置

1.Real Server配置
在這裏咱們將IP爲172.18.67.11和172.18.67.12的服務器80端口設置爲動態資源站，將IP爲172.18.67.11和172.18.67.12的8080端口服務器模擬另兩臺服務器設置成靜態資源站：將wordpress應用分別解壓至/var/www/html/下，並修改該目錄的屬主和屬組web

1
2
3
4
[root@inode4 ~] # unzip wordpress-4.3.1-zh_CN.zip -C /var/www/html/
[root@inode4 ~] # chown -R apache:apache /var/www/html/wordpress
[root@inode5 ~] # unzip wordpress-4.3.1-zh_CN.zip -C /var/www/html/
[root@inode5 ~] # chown -R apache:apache /var/www/html/wordpress


修改後端Server的httpd配置文件將網站目錄從默認的/var/www/html修改成/var/www/html/wordpress。
2.nfs配置
數據庫端修改配置文件redis

1
2
3
[root@inode6 ~] # vim /etc/exports
/data/  172.18.67.11(rw,async)
/data/  172.18.67.12(rw,async)


修改保存完畢後啓動Real Server和數據庫服務器的nfs應用sql

1
2
3
4
5
6
[root@inode4 ~] # systemctl start rpcbind
[root@inode4 ~] # systemctl start nfs
[root@inode5 ~] # systemctl start rpcbind
[root@inode5 ~] # systemctl start nfs
[root@inode6 ~] # systemctl start rpcbind
[root@inode6 ~] # systemctl start nfs


建立數據存放目錄及修改權

1
2
[root@inode6 ~] # mkdir /data
[root@inode6 ~] # chown -R mysql:mysql /data


修改數據庫配置文件數據存儲目錄

1
2
[root@inode6 ~] # vim /etc/my.cnf
datadir= /data/


啓動數據庫

1
[root@inode6 ~] # systemctl start mariadb-server


將數據庫目錄掛載至web動態資源服務器

1
2
[root@inode4 ~] # mount -t nfs 172.18.67.1:/data/ /mnt
[root@inode5 ~] # mount -t nfs 172.18.67.1:/data/ /mnt


3.keepalived配置
MASTER

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
! Configuration File  for  keepalived
global_defs {
    notification_email {
     root@localhost
    }
    notification_email_from keepalived@localhost
    smtp_server 127.0.0.1
    smtp_connect_timeout 30
    router_id inode2
    vrrp_macst_group4 224.0.67.67
}
vrrp_instance http {
     state MASTER
     interface eno16777736
     virtual_router_id 67
     priority 100
     advert_int 1
     authentication {
         auth_type PASS
     auth_pass lKZvQVv9
     }
     virtual_ipaddress {
         172.18.67.33 /16  dev eno16777736
     }
}
notify_master  "/etc/keepalived/notify.sh master"
notify_backup  "/etc/keepalived/notify.sh backup"
notify_fault  "/etc/keepalived/notify.sh fault"
}


BACKUP

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
! Configuration File  for  keepalived
global_defs {
    notification_email {
     root@localhost
    }
    notification_email_from keepalived@localhost
    smtp_server 127.0.0.1
    smtp_connect_timeout 30
    router_id inode2
    vrrp_macst_group4 224.0.67.67
}
vrrp_instance http {
     state BACKUP
     interface eno16777736
     virtual_router_id 67
     priority 98
     advert_int 1
     authentication {
         auth_type PASS
     auth_pass lKZvQVv9
     }
     virtual_ipaddress {
         172.18.67.33 /16  dev eno16777736
     }
}
notify_master  "/etc/keepalived/notify.sh master"
notify_backup  "/etc/keepalived/notify.sh backup"
notify_fault  "/etc/keepalived/notify.sh fault"
}


通知腳本

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
#!/bin/bash
#
contact= 'root@localhost'
notify() {
  mailsubject= "$(hostname) to be $1, vip floating"
  mailbody= "$(date +'%F %T'): vrrp transition, $(hostname) changed to be $1"
  echo  "$mailbody"  | mail -s  "$mailsubject"  $contact
}
case  $1  in
master)
  notify master
  ;;
backup)
  notify backup
  ;;
fault)
  notify fault
  ;;
*)
  echo  "Usage: $(basename $0) {master|backup|fault}"
  exit  1
  ;;
esac


4.haproxy配置
兩節點的配置內容是同樣的，以下：

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
[root@inode2 haproxy] # vim haproxy.cfg
global
     log         127.0.0.1 local2
     chroot       /var/lib/haproxy
     pidfile      /var/run/haproxy .pid
     maxconn     4000
     user        haproxy
     group       haproxy
     daemon
     stats socket  /var/lib/haproxy/stats
defaults
     mode                    http
     log                     global
     option                  httplog
     option                  dontlognull
     option http-server-close
     option forwardfor       except 127.0.0.0 /8
     option                  redispatch
     retries                 3
     timeout http-request    10s
     timeout queue           1m
     timeout connect         10s
     timeout client          1m
     timeout server          1m
     timeout http-keep-alive 10s
     timeout check           10s
     maxconn                 3000
frontend  web *:80
     mode      http
     maxconn      2000
     acl url_static     path_beg       -i   /static  /images  /javascript  /stylesheets
     acl url_static     path_end       -i  .jpg .gif .png .css .js .html .txt .htm
     use_backend staticsrvs   if  url_static
     default_backend     appsrvs
backend staticsrvs
     balance      roundrobin
     server      stcsrvs1 172.18.67.11:8080 check
     server        stcsrvs2 172.18.67.12:8080 check
backend appsrvs
     balance      roundrobin
     server      wp1 172.18.67.11:80 check
     server        wp2 172.18.67.12:80 check
listen stats
     bind :10086
     stats    enable
     stats   uri      /admin ?stats
     stats   auth    admin:admin
     stats   admin    if  TRUE

 

五 啓動服務並測試

1.啓動haproxy和keepalived

1
2
3
4
[root@inode2 ~] # systemctl restart haproxy
[root@inode2 ~] # systemctl restart keepalived
[root@inode3 ~] # systemctl restart haproxy
[root@inode3 ~] # systemctl restart keepalived


2.測試
inode2：

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
[root@inode2 ~] # systemctl status -l keepalived
● keepalived.service - LVS and VRRP High Availability Monitor
    Loaded: loaded ( /usr/lib/systemd/system/keepalived .service; disabled; vendor preset: disabled)
    Active: active (running) since Wed 2017-05-17 23:49:45 CST; 6s ago
   Process: 28940 ExecStart= /usr/sbin/keepalived  $KEEPALIVED_OPTIONS (code=exited, status=0 /SUCCESS )
  Main PID: 28941 (keepalived)
    CGroup:  /system .slice /keepalived .service
            ├─28941  /usr/sbin/keepalived  -D
            ├─28942  /usr/sbin/keepalived  -D
            └─28943  /usr/sbin/keepalived  -D
May 17 23:49:45 inode2 Keepalived_vrrp[28943]: Registering gratuitous ARP shared channel
May 17 23:49:45 inode2 Keepalived_vrrp[28943]: Opening  file  '/etc/keepalived/keepalived.conf' .
May 17 23:49:45 inode2 Keepalived_vrrp[28943]: Configuration is using : 63025 Bytes
May 17 23:49:45 inode2 Keepalived_vrrp[28943]: Using LinkWatch kernel netlink reflector...
May 17 23:49:45 inode2 Keepalived_vrrp[28943]: VRRP sockpool: [ifindex(2), proto(112), unicast(0), fd(10,11)]
May 17 23:49:46 inode2 Keepalived_vrrp[28943]: VRRP_Instance(http) Transition to MASTER STATE
May 17 23:49:47 inode2 Keepalived_vrrp[28943]: VRRP_Instance(http) Entering MASTER STATE
May 17 23:49:47 inode2 Keepalived_vrrp[28943]: VRRP_Instance(http) setting protocol VIPs.
May 17 23:49:47 inode2 Keepalived_healthcheckers[28942]: Netlink reflector reports IP 172.18.67.33 added
May 17 23:49:47 inode2 Keepalived_vrrp[28943]: VRRP_Instance(http) Sending gratuitous ARPs on eno16777736  for  172.18.67.33

 

咱們看到inode2節點進入了MASTER模式，再查看inode3的狀態

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
[root@inode3 ~] # systemctl start keepalived
[root@inode3 ~] # systemctl status -l  keepalived
● keepalived.service - LVS and VRRP High Availability Monitor
    Loaded: loaded ( /usr/lib/systemd/system/keepalived .service; disabled; vendor preset: disabled)
    Active: active (running) since Wed 2017-05-17 23:51:08 CST; 5s ago
   Process: 42610 ExecStart= /usr/sbin/keepalived  $KEEPALIVED_OPTIONS (code=exited, status=0 /SUCCESS )
  Main PID: 42611 (keepalived)
    CGroup:  /system .slice /keepalived .service
            ├─42611  /usr/sbin/keepalived  -D
            ├─42612  /usr/sbin/keepalived  -D
            └─42613  /usr/sbin/keepalived  -D
May 17 23:51:08 inode3 Keepalived_vrrp[42613]: Netlink reflector reports IP fe80::20c:29ff:fe78:24c3 added
May 17 23:51:08 inode3 Keepalived_vrrp[42613]: Registering Kernel netlink reflector
May 17 23:51:08 inode3 Keepalived_healthcheckers[42612]: Using LinkWatch kernel netlink reflector...
May 17 23:51:08 inode3 Keepalived_vrrp[42613]: Registering Kernel netlink  command  channel
May 17 23:51:08 inode3 Keepalived_vrrp[42613]: Registering gratuitous ARP shared channel
May 17 23:51:08 inode3 Keepalived_vrrp[42613]: Opening  file  '/etc/keepalived/keepalived.conf' .
May 17 23:51:08 inode3 Keepalived_vrrp[42613]: Configuration is using : 63023 Bytes
May 17 23:51:08 inode3 Keepalived_vrrp[42613]: Using LinkWatch kernel netlink reflector...
May 17 23:51:08 inode3 Keepalived_vrrp[42613]: VRRP_Instance(http) Entering BACKUP STATE
May 17 23:51:08 inode3 Keepalived_vrrp[42613]: VRRP sockpool: [ifindex(2), proto(112), unicast(0), fd(10,11)]

 

不難看出inode3節點處於BACKUP狀態，此時咱們將inode2的keepalived服務停掉

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
[root@inode2 ~] # systemctl stop keepalived
[root@inode3 ~] # systemctl status -l  keepalived
● keepalived.service - LVS and VRRP High Availability Monitor
    Loaded: loaded ( /usr/lib/systemd/system/keepalived .service; disabled; vendor preset: disabled)
    Active: active (running) since Wed 2017-05-17 23:51:08 CST; 1min 2s ago
   Process: 42610 ExecStart= /usr/sbin/keepalived  $KEEPALIVED_OPTIONS (code=exited, status=0 /SUCCESS )
  Main PID: 42611 (keepalived)
    CGroup:  /system .slice /keepalived .service
            ├─42611  /usr/sbin/keepalived  -D
            ├─42612  /usr/sbin/keepalived  -D
            └─42613  /usr/sbin/keepalived  -D
May 17 23:51:08 inode3 Keepalived_vrrp[42613]: Opening  file  '/etc/keepalived/keepalived.conf' .
May 17 23:51:08 inode3 Keepalived_vrrp[42613]: Configuration is using : 63023 Bytes
May 17 23:51:08 inode3 Keepalived_vrrp[42613]: Using LinkWatch kernel netlink reflector...
May 17 23:51:08 inode3 Keepalived_vrrp[42613]: VRRP_Instance(http) Entering BACKUP STATE
May 17 23:51:08 inode3 Keepalived_vrrp[42613]: VRRP sockpool: [ifindex(2), proto(112), unicast(0), fd(10,11)]
May 17 23:52:07 inode3 Keepalived_vrrp[42613]: VRRP_Instance(http) Transition to MASTER STATE
May 17 23:52:08 inode3 Keepalived_vrrp[42613]: VRRP_Instance(http) Entering MASTER STATE
May 17 23:52:08 inode3 Keepalived_vrrp[42613]: VRRP_Instance(http) setting protocol VIPs.
May 17 23:52:08 inode3 Keepalived_healthcheckers[42612]: Netlink reflector reports IP 172.18.67.33 added
May 17 23:52:08 inode3 Keepalived_vrrp[42613]: VRRP_Instance(http) Sending gratuitous ARPs on eno16777736  for  172.18.67.33

 

咱們發現inode3節點進入了MASTER狀態，所以體現出了高可用的特性
接下來咱們測試haproxy的特性，在haproxy的配置文件裏有下面這一段

1
2
3
4
5
6
listen stats
     bind :10086
     stats    enable
     stats   uri      /admin ?stats
     stats   auth    admin:admin
     stats   admin    if  TRUE


這段配置可讓咱們在瀏覽器中查看和修改haproxy統計接口啓用相關的參數，在瀏覽器中輸入http://172.18.67.33:10086/admin?stats，就會出現下面這種狀態，輸入帳號和密碼，就進入了haproxy相關參數配置頁面。


在下圖中能夠看出負載均衡的兩臺web服務器一臺負責動態資源解析，另外一臺負責靜態資源。動態資源的數據存放於後端的nfs服務器上。


接下來咱們在瀏覽器中訪問http://172.18.67.33就能夠安裝wordpress了。至此一個簡單的高可用負載均衡服務搭建完畢。

相關文章
相關標籤/搜索
每日一句
    每一个你不满意的现在，都有一个你没有努力的曾经。
本站公眾號
   歡迎關注本站公眾號,獲取更多信息
聯繫我們 最近搜索 最新文章 沪ICP备13005482号-10 MyBatis教程 SQL 教程 MySQL教程 Java 教程 Thymeleaf 教程 Hibernate教程 Spring教程 Redis教程