VM虛擬機上 實現CentOS 6.X下部署LVS(DR)+keepalived實現高性能高可用負載均衡(轉)

時間  2019-11-17
標籤 虛擬 機上 實現 centos 6.x 部署 lvs keepalived 高性能 可用 負載 均衡 欄目 CentOS 简体版
原文   原文鏈接

1、簡介

LVS是Linux Virtual Server的簡寫，意即Linux虛擬服務器，是一個虛擬的服務器集羣系統。本項目在1998年5月由章文嵩博士成立，是中國國內最先出現的自由軟件項目之一。javascript

目前有三種IP負載均衡技術（VS/NAT、VS/TUN和VS/DR）；十種調度算法（rr|wrr|lc|wlc|lblc|lblcr|dh|sh|sed|nq）。
html

Keepalived在這裏主要用做RealServer的健康狀態檢查以及LoadBalance主機和BackUP主機之間failover的實現。java

LVS+Keepalived能實現的功能：利用LVS控制器主備模式避免單點故障以及自動刪除故障WEB服務器結點並當它恢復後再自動添加到羣集中。linux

拓撲圖：web

1 準備工做算法

a) 虛擬機上安裝CentOS6.x操做系統，參考一下博客內容vim

http://www.cnblogs.com/seesea125/archive/2012/02/25/2368255.htmlbash

b)將上一步安裝上的虛擬機clone 3分，如圖所示 服務器

   

2個LVS服務器，一個做爲Master，一個做爲Backup 2個真實web服務器網絡

c)設置各個系統的網絡

c-1) win7操做系統，做爲外部訪問LVS服務器的設備，設置VMnet1的IP地址，如圖設置

 

c-2)在VM軟件中分別設置各個CentOS6系統的網卡

c-2-1)LVS MASTER 服務器的網卡設置，須要1塊網卡鏈接，如圖所示

   

c-2-2)LVS Backup 服務器的網卡設置，須要1塊網卡鏈接，如圖所示

    

c-2-3)設置2臺真實web服務器的網卡鏈接，須要一塊網卡鏈接，如圖所示

 

c-3)經過ifconfig命令，設置各個CentOS系統的網路設置

c-3-1)LVS MASTER 服務器的網路設置，分別設置eth0爲10.0.0.136 255.255.255.0,eth0:1爲10.0.0.148 255.255.255.255  

c-3-2)LVS BACKUP 服務器的網路設置，分別設置eth0爲10.0.0.157 255.255.255.0,eth0:1爲10.0.0.148 255.255.255.255

c-3-3)2臺真實web 服務器的網路設置，分別設置eth0爲10.0.0.137 255.255.255.0和10.0.0.139 255.255.255.0

經過以上步驟的話，經過ping命令檢驗是否全部的系統之間均可以聯通了。

2  安裝必要的軟件

a) LVS MASTER和Backup 服務器

a-1)ipvsadm 和keepalived的安裝

安裝以前最好查看一下系統中是否已經安裝了這2個軟件

命令錄下 yum -y install ipvsadm 和 yum -y install keepalived

a-2) 開啓路由轉發

[root@CentOS-LVS_MASTER ~]# vim /etc/sysctl.conf 
net.ipv4.ip_forward = 1
[root@CentOS-LVS_MASTER ~]# sysctl -p 

a-3) 配置keepalived，如圖所示

Master端的配置：cat /etc/keepalived/keepalived.conf

! Configuration File for keepalived

global_defs {
   notification_email {
     root@linux.tang.chao               #設置報警郵件地址，能夠設置多個，每行一個。
     mchina_tang@qq.com                 #需開啓本機的sendmail服務
   }
   notification_email_from keepalived@localhost         #設置郵件的發送地址
   smtp_server 127.0.0.1                                #設置smtp server地址
   smtp_connect_timeout 30                              #設置鏈接smtp server的超時時間
   router_id LVS_DEVEL                                  #表示運行keepalived服務器的一個標識。發郵件時顯示在郵件主題的信息
}

vrrp_instance VI_1 {
    state MASTER              #指定keepalived的角色，MASTER表示此主機是主服務器，BACKUP表示此主機是備用服務器
    interface eth0            #指定HA監測網絡的接口
    virtual_router_id 51      #虛擬路由標識，這個標識是一個數字，同一個vrrp實例使用惟一的標識。即同一vrrp_instance下，MASTER和BACKUP必須是一致的
    priority 100              #定義優先級，數字越大，優先級越高，在同一個vrrp_instance下，MASTER的優先級必須大於BACKUP的優先級
    advert_int 1              #設定MASTER與BACKUP負載均衡器之間同步檢查的時間間隔，單位是秒
    authentication {          #設置驗證類型和密碼
        auth_type PASS        #設置驗證類型，主要有PASS和AH兩種
        auth_pass 1111        #設置驗證密碼，在同一個vrrp_instance下，MASTER與BACKUP必須使用相同的密碼才能正常通訊
    }
    virtual_ipaddress {       #設置虛擬IP地址，能夠設置多個虛擬IP地址，每行一個
        10.0.0.148
    }
}

virtual_server 10.0.0.148 80 {      #設置虛擬服務器，須要指定虛擬IP地址和服務端口，IP與端口之間用空格隔開
    delay_loop 6                    #設置運行狀況檢查時間，單位是秒
    lb_algo rr                      #設置負載調度算法，這裏設置爲rr，即輪詢算法
    lb_kind DR                      #設置LVS實現負載均衡的機制，有NAT、TUN、DR三個模式可選
    persistence_timeout 50          #會話保持時間，單位是秒。這個選項對動態網頁是很是有用的，爲集羣系統中的session共享提供了一個很好的解決方案。
                                    #有了這個會話保持功能，用戶的請求會被一直分發到某個服務節點，直到超過這個會話的保持時間。
                                    #須要注意的是，這個會話保持時間是最大無響應超時時間，也就是說，用戶在操做動態頁面時，若是50秒內沒有執行任何操做，
                                    #那麼接下來的操做會被分發到另外的節點，可是若是用戶一直在操做動態頁面，則不受50秒的時間限制
    protocol TCP                    #指定轉發協議類型，有TCP和UDP兩種

    real_server 10.0.0.137 80 {     #配置服務節點1，須要指定real server的真實IP地址和端口，IP與端口之間用空格隔開
        weight 3                    #配置服務節點的權值，權值大小用數字表示，數字越大，權值越高，設置權值大小能夠爲不一樣性能的服務器
                                    #分配不一樣的負載，能夠爲性能高的服務器設置較高的權值，而爲性能較低的服務器設置相對較低的權值，這樣才能合理地利用和分配系統資源
        TCP_CHECK {                 #realserver的狀態檢測設置部分，單位是秒
            connect_timeout 10      #表示3秒無響應超時
            nb_get_retry 3          #表示重試次數
            delay_before_retry 3    #表示重試間隔
            connect_port 80
        }
    }
    real_server 10.0.0.139 80 {
        weight 3
        TCP_CHECK {
            connect_timeout 10
            nb_get_retry 3
            delay_before_retry 3
            connect_port 80
        }
    }
}

　　

  Backup端的配置：cat /etc/keepalived/keepalived.conf

<span style= "line-height: 1.5;" >! Configuration File for  keepalived</span> 
global_defs {
   notification_email {
     root@linux.tang.chao
     mchina_tang@qq.com
   }
   notification_email_from Alexandre.Cassen@firewall.loc
   smtp_server 127.0.0.1
   smtp_connect_timeout 30
   router_id LVS_DEVEL
}

vrrp_instance VI_1 {
    state BACKUP
    interface eth0
    virtual_router_id 51
    priority 99
    advert_int 1
    authentication {
        auth_type PASS
        auth_pass 1111
    }
    virtual_ipaddress {
        10.0.0.148
    }
}

virtual_server 10.0.0.148 80 {
    delay_loop 6
    lb_algo rr
    lb_kind DR
    persistence_timeout 50
    protocol TCP

    real_server 10.0.0.137 80 {
        weight 3
        TCP_CHECK {
            connect_timeout 10
            nb_get_retry 3
            delay_before_retry 3
            connect_port 80
        }
    }
    real_server 10.0.0.139 80 {
        weight 3
        TCP_CHECK {
            connect_timeout 10
            nb_get_retry 3
            delay_before_retry 3
            connect_port 80
        }
    }
}

a-4) 將keepalived設置爲服務

chkconfig keepalived on　

b) 在2臺真實的web服務器分別配置HTTP　

[root@WEB1 ~]# yum -y install httpd
[root@WEB1 ~]# cd /var/www/html/
[root@WEB1 html]# cat index.html 
<h1>WEB1/10.0.0.137</h1>
[root@WEB1 html]# /etc/init.d/httpd start
另外一臺機器配置同樣，過程略。

 

三、在兩臺Web Server上執行realserver.sh腳本，爲lo:0綁定VIP地址10.0.0.14八、抑制ARP廣播。

[root@web1 ~]# cat realserver.sh

複製代碼 
#!/bin/bash
#description: Config realserver

VIP=10.0.0.148
 
/etc/rc.d/init.d/functions
 
case "$1" in
start)
       /sbin/ifconfig lo:0 $VIP netmask 255.255.255.255 broadcast $VIP
       /sbin/route add -host $VIP dev lo:0
       echo "1" >/proc/sys/net/ipv4/conf/lo/arp_ignore
       echo "2" >/proc/sys/net/ipv4/conf/lo/arp_announce
       echo "1" >/proc/sys/net/ipv4/conf/all/arp_ignore
       echo "2" >/proc/sys/net/ipv4/conf/all/arp_announce
       sysctl -p >/dev/null 2>&1
       echo "RealServer Start OK"
       ;;
stop)
       /sbin/ifconfig lo:0 down
       /sbin/route del $VIP >/dev/null 2>&1
       echo "0" >/proc/sys/net/ipv4/conf/lo/arp_ignore
       echo "0" >/proc/sys/net/ipv4/conf/lo/arp_announce
       echo "0" >/proc/sys/net/ipv4/conf/all/arp_ignore
       echo "0" >/proc/sys/net/ipv4/conf/all/arp_announce
       echo "RealServer Stoped"
       ;;
*)
       echo "Usage: $0 {start|stop}"
       exit 1
esac
 
exit 0
複製代碼

[root@web1 ~]# sh realserver.sh start

四、分別在LVS-MASTER、LVS-BACKUP上執行service keepalived start啓動keepalived就可實現負載均衡及高可用集羣了；

[root@LVS-MASTER keepalived]# service keepalived start

[root@LVS-MASTER keepalived]# ipvsadm -L
IP Virtual Server version 1.2.1 (size=4096)
Prot LocalAddress:Port Scheduler Flags
  -> RemoteAddress:Port           Forward Weight ActiveConn InActConn
TCP  10.0.0.148:http rr persistent 50
  -> 10.0.0.139:http              Route   3      0          0         
  -> 10.0.0.137:http              Route   3      0          0         
[root@LVS-MASTER keepalived]#

查看日誌/var/log/messages

[root@LVS-MASTER keepalived]# tail -n 30/var/log/messages
Apr 19 03:59:58 LVS-MASTER avahi-daemon[2845]: Withdrawing address record for10.0.0.148 on eth0.
Apr 19 04:00:19 LVS-MASTER Keepalived: Starting Keepalived v1.2.1 (04/17,2012)
Apr 19 04:00:19 LVS-MASTER Keepalived: Starting Healthcheck child process,pid=5960
Apr 19 04:00:19 LVS-MASTER Keepalived_healthcheckers: Netlink reflector reportsIP 10.0.0.136 added
Apr 19 04:00:19 LVS-MASTER Keepalived_healthcheckers: Netlink reflector reportsIP 10.0.0.159 added
Apr 19 04:00:19 LVS-MASTER Keepalived_healthcheckers: Registering Kernelnetlink reflector
Apr 19 04:00:19 LVS-MASTER Keepalived_healthcheckers: Registering Kernelnetlink command channel
Apr 19 04:00:19 LVS-MASTER Keepalived_healthcheckers: Opening file '/etc/keepalived/keepalived.conf'.
Apr 19 04:00:19 LVS-MASTER Keepalived_healthcheckers: Configuration is using :12257 Bytes
Apr 19 04:00:19 LVS-MASTER Keepalived_healthcheckers: Using LinkWatch kernelnetlink reflector...
Apr 19 04:00:19 LVS-MASTER Keepalived_healthcheckers: Activating healtcheckerfor service [10.0.0.137:80]
Apr 19 04:00:19 LVS-MASTER Keepalived_healthcheckers: Activating healtcheckerfor service [10.0.0.139:80]
Apr 19 04:00:19 LVS-MASTER Keepalived_vrrp: Netlink reflector reports IP10.0.0.136 added
Apr 19 04:00:19 LVS-MASTER Keepalived_vrrp: Netlink reflector reports IP10.0.0.159 added
Apr 19 04:00:19 LVS-MASTER Keepalived_vrrp: Registering Kernel netlinkreflector
Apr 19 04:00:19 LVS-MASTER Keepalived_vrrp: Registering Kernel netlink commandchannel
Apr 19 04:00:19 LVS-MASTER Keepalived_vrrp: Registering gratutious ARP sharedchannel
Apr 19 04:00:19 LVS-MASTER Keepalived_vrrp: Opening file'/etc/keepalived/keepalived.conf'.
Apr 19 04:00:19 LVS-MASTER Keepalived_vrrp: Configuration is using : 36698Bytes
Apr 19 04:00:19 LVS-MASTER Keepalived_vrrp: Using LinkWatch kernel netlinkreflector...
Apr 19 04:00:19 LVS-MASTER Keepalived_vrrp: VRRP sockpool: [ifindex(2),proto(112), fd(11,12)]
Apr 19 04:00:19 LVS-MASTER Keepalived: Starting VRRP child process, pid=5961
Apr 19 04:00:20 LVS-MASTER Keepalived_vrrp: VRRP_Instance(VI_1) Transition toMASTER STATE
Apr 19 04:00:21 LVS-MASTER Keepalived_vrrp: VRRP_Instance(VI_1) Entering MASTERSTATE
Apr 19 04:00:21 LVS-MASTER Keepalived_vrrp: VRRP_Instance(VI_1) settingprotocol VIPs.
Apr 19 04:00:21 LVS-MASTER Keepalived_vrrp: VRRP_Instance(VI_1) Sendinggratuitous ARPs on eth0 for 10.0.0.148
Apr 19 04:00:21 LVS-MASTER Keepalived_healthcheckers: Netlink reflector reportsIP 10.0.0.136 added
Apr 19 04:00:21 LVS-MASTER avahi-daemon[2845]: Registering new address recordfor 10.0.0.148 on eth0.
Apr 19 04:00:21 LVS-MASTER Keepalived_vrrp: Netlink reflector reports IP10.0.0.136 added
Apr 19 04:00:26 LVS-MASTER Keepalived_vrrp: VRRP_Instance(VI_1) Sendinggratuitous ARPs on eth0 for 10.0.0.148
[root@LVS-MASTER keepalived]#

4、測試

接下來作高可用性測試&故障切換測試......

####高可用性測試####

模擬故障，將LVS-MASTER上的keepalived服務停掉，而後觀察LVS-BACKUP上的日誌，信息以下

[root@LVS-BACKUP keepalived]# tail -f /var/log/messages
Apr 19 03:33:04 LVS-BACKUP Keepalived_vrrp: VRRP_Instance(VI_1)Transition to MASTER STATE
Apr 19 03:33:05 LVS-BACKUP Keepalived_vrrp: VRRP_Instance(VI_1) Entering MASTERSTATE
Apr 19 03:33:05 LVS-BACKUP Keepalived_vrrp: VRRP_Instance(VI_1) settingprotocol VIPs.
Apr 19 03:33:05 LVS-BACKUP Keepalived_vrrp: VRRP_Instance(VI_1) Sendinggratuitous ARPs on eth0 for 10.0.0.148
Apr 19 03:33:05 LVS-BACKUP Keepalived_vrrp: Netlink reflector reports IP10.0.0.157 added
Apr 19 03:33:05 LVS-BACKUP Keepalived_healthcheckers: Netlink reflector reportsIP 10.0.0.157 added
Apr 19 03:33:05 LVS-BACKUP avahi-daemon[2825]: Registering new address recordfor 10.0.0.148 on eth0.
Apr 19 03:33:10 LVS-BACKUP Keepalived_vrrp: VRRP_Instance(VI_1) Sendinggratuitous ARPs on eth0 for 10.0.0.148

從日誌中可知，主機出現故障後，備機馬上檢測到，此時備機變爲MASTER角色，而且接管了主機的虛擬IP資源，最後將虛擬IP綁定在etho設備上。

將LVS-MASTER 上的keepalived服務開啓後，LVS-BACKUP的日誌狀態。

Apr 19 03:34:02 LVS-BACKUP Keepalived_vrrp: VRRP_Instance(VI_1) Received higher prio advert
Apr 19 03:34:02 LVS-BACKUP Keepalived_vrrp: VRRP_Instance(VI_1)Entering BACKUP STATE
Apr 19 03:34:02 LVS-BACKUP Keepalived_vrrp: VRRP_Instance(VI_1) removing protocol VIPs.
Apr 19 03:34:02 LVS-BACKUP Keepalived_vrrp: Netlink reflector reports IP 10.0.0.157 removed
Apr 19 03:34:02 LVS-BACKUP Keepalived_healthcheckers: Netlink reflector reports IP 10.0.0.157 removed
Apr 19 03:34:02 LVS-BACKUP avahi-daemon[2825]: Withdrawing address record for 10.0.0.148 on eth0.

從日誌可知，備機在檢測到主機從新恢復正常後，釋放了虛擬IP資源從新成爲BACKUP角色

####故障切換測試####

故障切換是測試當某個節點出現故障後，Keepalived監製模塊是否能及時發現而後屏蔽故障節點，同時將服務器轉移到正常節點來執行。

將web2節點服務停掉，假設這個節點出現故障，而後主、備機日誌信息以下

Apr 19 03:35:04 LVS-MASTER Keepalived_healthcheckers:TCP connection to [10.0.0.139:80] failed!!!
Apr 19 03:35:04 LVS-MASTER Keepalived_healthcheckers:Removing service [10.0.0.139:80] from VS [10.0.0.148:80]
Apr 19 03:35:04 LVS-MASTER Keepalived_healthcheckers: Remote SMTP server [127.0.0.1:25] connected.

[root@LVS-MASTER keepalived]# ipvsadm -L
IP Virtual Server version 1.2.1 (size=4096)
Prot LocalAddress:Port Scheduler Flags
  -> RemoteAddress:Port           Forward Weight ActiveConn InActConn
TCP  10.0.0.148:http rr persistent 50
  -> 10.0.0.137:http              Route   3      0          0         
[root@LVS-MASTER keepalived]#

從以上能夠看出，Keepalived監控模塊檢測到10.0.0.139這臺主機出現故障後，將些web2從集羣系統中剔除掉了。 此時訪問http://10.0.0.148只能看到web1了)

keepalived

從新啓動web2節點的服務，日誌信息以下：

Apr 19 03:38:22 LVS-MASTER Keepalived_healthcheckers:TCP connection to [10.0.0.139:80] success.
Apr 19 03:38:22 LVS-MASTER Keepalived_healthcheckers:Adding service [10.0.0.139:80] to VS [10.0.0.148:80]
Apr 19 03:38:22 LVS-MASTER Keepalived_healthcheckers: Remote SMTP server [127.0.0.1:25] connected.

[root@LVS-MASTERkeepalived]# ipvsadm -L
IP Virtual Server version 1.2.1 (size=4096)
Prot LocalAddress:Port Scheduler Flags
  -> RemoteAddress:Port           Forward Weight ActiveConn InActConn
TCP  10.0.0.148:http rr persistent 50
  -> 10.0.0.139:http              Route   3      0          0         
  -> 10.0.0.137:http              Route   3      0          0         
[root@LVS-MASTER keepalived]#

Keepalived監控模塊檢測到10.0.0.139這臺主機恢復正常後，又將此節點加入集羣系統中，再次訪問就能夠訪問到web2頁面了）

lvs

 參考自：http://www.cnblogs.com/mchina/archive/2012/05/23/2514728.html

相關文章
相關標籤/搜索
每日一句
    每一个你不满意的现在，都有一个你没有努力的曾经。
本站公眾號
   歡迎關注本站公眾號,獲取更多信息
聯繫我們 最近搜索 最新文章 沪ICP备13005482号-10 MyBatis教程 SQL 教程 MySQL教程 Java 教程 Thymeleaf 教程 Hibernate教程 Spring教程 Redis教程