LVS/NAT + keepalived負載均衡實現

時間  2019-12-24
標籤 lvs nat keepalived 負載 均衡 實現 欄目 負載均衡 简体版
原文   原文鏈接

1、keepalived簡介前端

keepalived是分佈式部署解決系統高可用的軟件，結合lvs(LinuxVirtual Server)使用，解決單機宕機的問題。
keepalived是一個基於VRRP協議來實現IPVS的高可用的解決方案。對於LVS負載均衡來講，若是前端的調度器direct發生故障，則後端的realserver是沒法接受請求並響應的。所以，保證前端direct的高可用性是很是關鍵的，不然後端的服務器是沒法進行服務的。而咱們的keepalived就能夠用來解決單點故障（如LVS的前端direct故障）問題。keepalived的主要工做原理是：運行keepalived的兩臺服務器，其中一臺爲MASTER，另外一臺爲BACKUP，正常狀況下，全部的數據轉換功能和ARP請求響應都是由MASTER完成的，一旦MASTER發生故障，則BACKUP會立刻接管MASTER的工做，這種切換時很是迅速的。linux

 

2、測試環境web

下面拿4臺虛擬機進行環境測試,實驗環境爲centos6.6 x86_64,在lvs NAT模式環境下只有前端兩臺keepalived調度機有公網ip，後端真實機只有內網ip，具體用途和ip以下後端

服務器類型centos

公網ip服務器

內網ip負載均衡

LVS VIP分佈式

192.168.214.70ide

192.168.211.254oop

Keepalived Master

192.168.214.76

192.168.211.76

Keepalived Backup

192.168.214.77

192.168.211.77

Realserver A

192.168.211.79

        Realserver B

192.168.211.83

 

3、軟件安裝

1、安裝lvs所需包ipvsadm

yum install -y ipvsadm

ln -s /usr/src/kernels/`uname -r`  /usr/src/linux

lsmod |grep ip_vs

 

#注意Centos 6.X安裝lvs，使用1.26版本。而且須要先安裝yuminstall libnl* popt* -y

 

執行ipvsadm(modprobe ip_vs)把ip_vs模塊加載到內核

[root@test85 ~]# ipvsadm -L -n

IP Virtual Server version 1.2.1 (size=4096)

Prot LocalAddress:Port Scheduler Flags

 -> RemoteAddress:Port          Forward Weight ActiveConn InActConn

 

#IP Virtual Server version 1.2.1 ---- ip_vs內核模塊版本

 

 

2、安裝keepalived

yum install -y keepalived

chkconfig keepalived on

注：在centos7系列系統中開機自動啓動使用systemctl enable keepalived


4、開啓路由轉發 

在lvs/NAT模式下前端兩臺調度機192.168.211.76和192.168.211.77上須要開啓路由轉發功能

vi /etc/sysctl.conf

修改net.ipv4.ip_forward = 0爲net.ipv4.ip_forward= 1

保存退出後，使用systcl –p命令讓其生效

 

5、keepalived配置

先看下211.76 keepalivedmaster主機上的配置

具體詳細的參數說明見上篇LVS/DR + keepalived負載均衡實現

[root@localhost ~]# cat /etc/keepalived/keepalived.conf 
! Configuration File for keepalived

global_defs {
   notification_email {
     charles@test.com 
   }
   notification_email_from ReportLog@test.com 
   #smtp_server 127.0.0.1 
   #smtp_connect_timeout 30
    router_id LVS_1
   #lvs_id LVS_EXAMPLE_01 
}

vrrp_sync_group VG1 {
   group {
      VI_1
      VI_GATEWAY
   }
} 

vrrp_instance VI_1 {
    state MASTER
    interface eth0    #公網ip網口
    virtual_router_id 51
    priority 150
    advert_int 1
    lvs_sync_daemon_inteface eth0
    authentication {
        auth_type PASS
        auth_pass 1111
    }
    virtual_ipaddress {
        192.168.214.70    #公網vip地址
    }
}

vrrp_instance VI_GATEWAY {
    state MASTER
    interface eth1         #內網ip網口
    lvs_sync_daemon_inteface eth1
    virtual_router_id 52
    priority 150
    advert_int 1
    authentication {
        auth_type PASS
        auth_pass 1111
    }
    virtual_ipaddress {
        192.168.211.254  #內網vip地址
    }
}

virtual_server 192.168.214.70 80 {
    delay_loop 6
    lb_algo rr
    lb_kind NAT         #使用lvs的NAT工做模式
    #nat_mask 255.255.255.0
    persistence_timeout 10 
    protocol TCP

    real_server 192.168.211.79 80 {
        TCP_CHECK {
            connect_timeout 3
            connect_port 80
            nb_get_retry 3
            delay_before_retry 3
        }
    }
   real_server 192.168.211.83 80 {
        TCP_CHECK {
            connect_timeout 3
            connect_port 80
            nb_get_retry 3
            delay_before_retry 3
        }
    }
}


再看下211.77 keepalived BACKUP主機上的配置

[root@localhost ~]# cat /etc/keepalived/keepalived.conf 
! Configuration File for keepalived
global_defs {
   notification_email {
     charles@test.com 
   }
   notification_email_from ReportLog@test.com 
   #smtp_server 127.0.0.1 
   #smtp_connect_timeout 30
   router_id LVS_1
   #lvs_id LVS_EXAMPLE_02 
}

vrrp_sync_group VG1 {
   group {
      VI_1
      VI_GATEWAY
   }
} 

vrrp_instance VI_1 {
    state BACKUP
    interface eth0
    lvs_sync_daemon_inteface eth0
    virtual_router_id 51
    priority 100
    advert_int 1
    authentication {
        auth_type PASS
        auth_pass 1111
    }
    virtual_ipaddress {
        192.168.214.70 
    }
}

vrrp_instance VI_GATEWAY {
    state BACKUP
    interface eth1
    lvs_sync_daemon_inteface eth1
    virtual_router_id 52
    priority 100
    advert_int 1
    authentication {
        auth_type PASS
        auth_pass 1111
    }
    virtual_ipaddress {
        192.168.211.254 
    }
}

virtual_server 192.168.214.70 80 {
    delay_loop 6
    lb_algo rr
    lb_kind NAT
    #nat_mask 255.255.255.0
    persistence_timeout 10
    protocol TCP

 real_server 192.168.211.79 80 {
        TCP_CHECK {
            connect_timeout 3
            connect_port 80
            nb_get_retry 3
            delay_before_retry 3
        }
    }
real_server 192.168.211.83 80 {
        TCP_CHECK {
            connect_timeout 3
            connect_port 80
            nb_get_retry 3
            delay_before_retry 3
        }
    }

}

6、後端真實機配置

後端兩臺web服務真實機192.168.211.79和192.168.211.83須要配置默認網關爲vip內網ip地址192.168.211.254。

注意，線上環境若是更改默認網關爲vip內網地址後，可能會形成鏈接不上服務器，須要提早添加相關靜態路由。

 

7、啓動keepalived服務及查看相關信息

在211.76和211.77上分別啓動keepalived服務

 

在211.76keepalived Master主機上查看信息

經過ip addr能夠看到vip 地址已經綁定在eth0和eth1網口

[root@localhost ~]# ip addr
1: lo: <LOOPBACK,UP,LOWER_UP> mtu65536 qdisc noqueue state UNKNOWN 
   link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
   inet 127.0.0.1/8 scope host lo
   inet6 ::1/128 scope host 
      valid_lft forever preferred_lft forever
2: eth0:<BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen1000
   link/ether 52:54:00:55:b2:d4 brd ff:ff:ff:ff:ff:ff
   inet 192.168.214.76/24 brd 192.168.214.255 scope global eth0
   inet 192.168.214.70/32 scope global eth0
   inet6 fe80::5054:ff:fe55:b2d4/64 scope link 
      valid_lft forever preferred_lft forever
3: eth1:<BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen1000
   link/ether 52:54:00:85:11:95 brd ff:ff:ff:ff:ff:ff
   inet 192.168.211.76/24 brd 192.168.211.255 scope global eth1
   inet 192.168.211.254/32 scope global eth1
   inet6 fe80::5054:ff:fe85:1195/64 scope link 
      valid_lft forever preferred_lft forever

在211.76上查看日誌信息，看到已成功進入keepalived主機模式
[root@localhost ~]# tail -f/var/log/messages
May 10 10:25:47 localhostKeepalived_vrrp[6210]: VRRP_Instance(VI_GATEWAY) setting protocol VIPs.
May 10 10:25:47 localhost Keepalived_healthcheckers[6209]:Netlink reflector reports IP 192.168.211.254 added
May 10 10:25:47 localhostKeepalived_vrrp[6210]: VRRP_Instance(VI_GATEWAY) Sending gratuitous ARPs oneth1 for 192.168.211.254
May 10 10:25:47 localhostKeepalived_vrrp[6210]: VRRP_Group(VG1) Syncing instances to MASTER state
May 10 10:25:47 localhostKeepalived_vrrp[6210]: VRRP_Instance(VI_1) EnteringMASTER STATE
May 10 10:25:47 localhostKeepalived_vrrp[6210]: VRRP_Instance(VI_1) setting protocol VIPs.
May 10 10:25:47 localhost Keepalived_healthcheckers[6209]:Netlink reflector reports IP 192.168.214.70 added
May 10 10:25:47 localhostKeepalived_vrrp[6210]: VRRP_Instance(VI_1) Sendinggratuitous ARPs on eth0 for 192.168.214.70
May 10 10:25:52 localhostKeepalived_vrrp[6210]: VRRP_Instance(VI_GATEWAY) Sendinggratuitous ARPs on eth1 for 192.168.211.254
May 10 10:25:52 localhostKeepalived_vrrp[6210]: VRRP_Instance(VI_1) Sending gratuitous ARPs on eth0 for192.168.214.70


在211.77上查看日誌信息，看到已成功進入keepalived備機模式

[root@localhost ~]# tail -f/var/log/messages
May 10 10:27:54 localhostKeepalived_vrrp[6249]: Opening file '/etc/keepalived/keepalived.conf'.
May 10 10:27:54 localhostKeepalived_vrrp[6249]: Configuration is using : 69554 Bytes
May 10 10:27:54 localhostKeepalived_vrrp[6249]: Using LinkWatch kernel netlink reflector...
May 10 10:27:54 localhostKeepalived_vrrp[6249]: VRRP_Instance(VI_1) Entering BACKUP STATE
May 10 10:27:54 localhostKeepalived_vrrp[6249]: VRRP_Instance(VI_GATEWAY) Entering BACKUP STATE
May 10 10:27:54 localhost Keepalived_vrrp[6249]:VRRP sockpool: [ifindex(2), proto(112), unicast(0), fd(10,11)]
May 10 10:27:54 localhostKeepalived_vrrp[6249]: VRRP sockpool: [ifindex(3), proto(112), unicast(0),fd(12,13)]
May 10 10:27:54 localhostKeepalived_healthcheckers[6248]: Using LinkWatch kernel netlink reflector...
May 10 10:27:54 localhostKeepalived_healthcheckers[6248]: Activating healthchecker for service[192.168.211.79]:80
May 10 10:27:54 localhostKeepalived_healthcheckers[6248]: Activating healthchecker for service[192.168.211.83]:80

經過ipvsadm -L –n 查看相應lvs鏈接信息
[root@localhost ~]# ipvsadm -L -n
IP Virtual Server version 1.2.1 (size=4096)
Prot LocalAddress:Port Scheduler Flags
  -> RemoteAddress:Port           Forward Weight ActiveConn InActConn
TCP  192.168.214.70:80 rr persistent 10
  -> 192.168.211.79:80            Masq    1      0          0         
  -> 192.168.211.83:80            Masq    1      3          0


8、keepalived測試

使用vip地址192.168.214.70訪問後端192.168.211.79和192.168.211.83的頁面

01.jpg

02.jpg

正常訪問沒問題後，咱們來模擬lvs集羣故障，讓主備進行自動切換下

首先把keepalived master主機211.76宕機，看備機可否接管過來，vip地址是否會漂移過來

在211.77上查看日誌，發現備機成功切換成了主機狀態，vip地址成功漂移了過來

[root@localhost ~]# ip addr

1: lo: <LOOPBACK,UP,LOWER_UP> mtu65536 qdisc noqueue state UNKNOWN

   link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00

   inet 127.0.0.1/8 scope host lo

   inet6 ::1/128 scope host

      valid_lft forever preferred_lft forever

2: eth0:<BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen1000

   link/ether 52:54:00:1b:a2:11 brd ff:ff:ff:ff:ff:ff

   inet 192.168.214.77/24 brd 192.168.214.255 scope global eth0

   inet 192.168.214.70/32 scope global eth0

   inet6 fe80::5054:ff:fe1b:a211/64 scope link

      valid_lft forever preferred_lft forever

3: eth1: <BROADCAST,MULTICAST,UP,LOWER_UP>mtu 1500 qdisc pfifo_fast state UP qlen 1000

   link/ether 52:54:00:64:47:7d brd ff:ff:ff:ff:ff:ff

   inet 192.168.211.77/24 brd 192.168.211.255 scope global eth1

   inet 192.168.211.254/32 scope global eth1

   inet6 fe80::5054:ff:fe64:477d/64 scope link

      valid_lft forever preferred_lft forever


[root@localhost ~]# tail -f /var/log/messages
May 10 10:27:54 localhost Keepalived_vrrp[6249]: Opening file '/etc/keepalived/keepalived.conf'.
May 10 10:27:54 localhost Keepalived_vrrp[6249]: Configuration is using : 69554 Bytes
May 10 10:27:54 localhost Keepalived_vrrp[6249]: Using LinkWatch kernel netlink reflector...
May 10 10:27:54 localhost Keepalived_vrrp[6249]: VRRP_Instance(VI_1) Entering BACKUP STATE
May 10 10:27:54 localhost Keepalived_vrrp[6249]: VRRP_Instance(VI_GATEWAY) Entering BACKUP STATE
May 10 10:27:54 localhost Keepalived_vrrp[6249]: VRRP sockpool: [ifindex(2), proto(112), unicast(0), fd(10,11)]
May 10 10:27:54 localhost Keepalived_vrrp[6249]: VRRP sockpool: [ifindex(3), proto(112), unicast(0), fd(12,13)]
May 10 10:27:54 localhost Keepalived_healthcheckers[6248]: Using LinkWatch kernel netlink reflector...
May 10 10:27:54 localhost Keepalived_healthcheckers[6248]: Activating healthchecker for service [192.168.211.79]:80
May 10 10:27:54 localhost Keepalived_healthcheckers[6248]: Activating healthchecker for service [192.168.211.83]:80
May 10 11:09:26 localhost Keepalived_vrrp[6249]: VRRP_Instance(VI_1) Transition to MASTER STATE
May 10 11:09:26 localhost Keepalived_vrrp[6249]: VRRP_Group(VG1) Syncing instances to MASTER state
May 10 11:09:26 localhost Keepalived_vrrp[6249]: VRRP_Instance(VI_GATEWAY) Transition to MASTER STATE
May 10 11:09:27 localhost Keepalived_vrrp[6249]: VRRP_Instance(VI_GATEWAY) Entering MASTER STATE
May 10 11:09:27 localhost Keepalived_vrrp[6249]: VRRP_Instance(VI_GATEWAY) setting protocol VIPs.
May 10 11:09:27 localhost Keepalived_vrrp[6249]: VRRP_Instance(VI_GATEWAY) Sending gratuitous ARPs on eth1 for 192.168.211.254
May 10 11:09:27 localhost Keepalived_healthcheckers[6248]: Netlink reflector reports IP 192.168.211.254 added
May 10 11:09:27 localhost Keepalived_vrrp[6249]: VRRP_Instance(VI_1) Entering MASTER STATE
May 10 11:09:27 localhost Keepalived_vrrp[6249]: VRRP_Instance(VI_1) setting protocol VIPs.
May 10 11:09:27 localhost Keepalived_vrrp[6249]: VRRP_Instance(VI_1) Sending gratuitous ARPs on eth0 for 192.168.214.70
May 10 11:09:27 localhost Keepalived_healthcheckers[6248]: Netlink reflector reports IP 192.168.214.70 added
May 10 11:09:32 localhost Keepalived_vrrp[6249]: VRRP_Instance(VI_GATEWAY) Sending gratuitous ARPs on eth1 for 192.168.211.254
May 10 11:09:32 localhost Keepalived_vrrp[6249]: VRRP_Instance(VI_1) Sending


若是想了解更多，請關注咱們的公衆號
公衆號ID:opdevos

掃碼關注

gongzhouhao.jpg

相關文章
相關標籤/搜索
每日一句
    每一个你不满意的现在，都有一个你没有努力的曾经。
本站公眾號
   歡迎關注本站公眾號,獲取更多信息
聯繫我們 最近搜索 最新文章 沪ICP备13005482号-10 MyBatis教程 SQL 教程 MySQL教程 Java 教程 Thymeleaf 教程 Hibernate教程 Spring教程 Redis教程