LVS-dr

1、LVS-dr的基本原理

   direct routing，它經過修改請求報文的目標MAC地址進行轉發。

   請求報文經由director發送至RS，那麼就不能讓除Director外的RS響應，所以，有三種解決方案：1.在上游路由器進行IP和MAC的綁定，2.在RS上作arptables，3.修改RS主機內核參數，也就是說vip別名在環回口，修改的內核參數能使vip對發來的請求不作處理。

   響應報文不經由director，而是有vip直接響應cip，其中vip經過rip的MAC和gateway直接響應cip。

   dr的基本規則：     

（1）保證前端路由器將目標IP爲VIP的請求報文發送給director

　　解決方案：

 　　 靜態綁定

  　　arptables

 　 　修改RS主機內核的參數

（2）RS的RIP可使用私有地址；但也可使用公網地址

（3）RS跟Director必須在同一物理網絡中

（4）請求報文經由Director調度，但響應報文必定不能經由Director；

（5）不支持端口映射

（6）RS能夠大多數OS

（7）RS的網關不能指向DIP

 

2、實驗環境，局域網環境

Client：本機windows7
Director：CentOS 7.1
RealServer：node1，node2均爲CentOS6.7
Director與RealServer的全部IP均爲同一網段

拓撲以下：

 

3、配置

director：
[root@localhost ~]# ifconfig ens33:0 192.168.1.15/32 broadcast 192.168.1.15 up
[root@localhost ~]# route add -host 192.168.1.15 dev ens33:0
 
RS：
 node1：
    [root@jymlinux ~]# echo 1 > /proc/sys/net/ipv4/conf/all/arp_ignore 
    [root@jymlinux ~]# echo 1 > /proc/sys/net/ipv4/conf/eth2/arp_ignore 
    [root@jymlinux ~]# echo 2 > /proc/sys/net/ipv4/conf/all/arp_announce 
    [root@jymlinux ~]# echo 2 > /proc/sys/net/ipv4/conf/eth2/arp_announce 
    [root@jymlinux ~]# ifconfig lo:0 192.168.1.15/32 broadcast 192.168.1.15 up
    [root@jymlinux ~]# route add -host 192.168.1.15 dev lo:0
 node2：
    [root@jymlinux ~]# echo 1 > /proc/sys/net/ipv4/conf/all/arp_ignore
    [root@jymlinux ~]# echo 1 > /proc/sys/net/ipv4/conf/eth0/arp_ignore
    [root@jymlinux ~]# echo 2 > /proc/sys/net/ipv4/conf/all/arp_announce 
    [root@jymlinux ~]# echo 2 > /proc/sys/net/ipv4/conf/eth0/arp_announce 
    [root@jymlinux ~]# ifconfig lo:0 192.168.1.15/32 broadcast 192.168.1.15 up
    [root@jymlinux ~]# route add -host 192.168.1.15 dev lo:0
     
或寫腳本並給予執行權限設置內核參數
[root@jymlinux ~]# vim lvsdrka.sh
#!/bin/bash
#
case $1 in
start)
        echo 1 > /proc/sys/net/ipv4/conf/all/arp_ignore
        echo 1 > /proc/sys/net/ipv4/conf/eth2/arp_ignore
        echo 2 > /proc/sys/net/ipv4/conf/all/arp_announce
        echo 2 > /proc/sys/net/ipv4/conf/eth2/arp_announce
        ;;
stop)
        echo 0 > /proc/sys/net/ipv4/conf/all/arp_ignore
        echo 0 > /proc/sys/net/ipv4/conf/eth2/arp_ignore
        echo 0 > /proc/sys/net/ipv4/conf/all/arp_announce
        echo 0 > /proc/sys/net/ipv4/conf/eth2/arp_announce
        ;;
esac
 
[root@jymlinux ~]# chmod +x lvsdrka.sh 
[root@jymlinux ~]# ./lvsdrka.sh start
[root@jymlinux ~]# cat /proc/sys/net/ipv4/conf/all/arp_ignore  
1
[root@jymlinux ~]# cat /proc/sys/net/ipv4/conf/all/arp_announce 
2

兩個內核參數：

    arp_announce：arp通告

　　0：通告所有IP      默認

　　1：儘可能避免將非本網絡的地址通告給網絡中的其餘地址

　　2：老是用最佳本地地址通告網絡

　arp_ignore：arp響應

　　0：不管從哪一個接口請求的，只要主機有這個地址，就會響應    默認

　　1：從哪一個接口請求的地址，就從哪一個接口的地址響應

　　2,3,4,,5,6,7,8,有9個級別，其他不經常使用

在lvs-dr中，咱們通常使用arp_ignore=1 arp_announce=2

 

測試web服務

[root@localhost ~]# curl http://192.168.1.20
<h1>this is node1 <\h1>
[root@localhost ~]# curl http://192.168.1.21
<h1>this is node2 <\h1>

配置集羣規則

[root@localhost ~]# ipvsadm -A -t 192.168.1.15:80 -s rr
[root@localhost ~]# ipvsadm -a -t 192.168.1.15:80 -r 192.168.1.20 -g
[root@localhost ~]# ipvsadm -a -t 192.168.1.15:80 -r 192.168.1.21 -g

4、配置http與https雙集羣服務

此處網段改成192.168.3.0網段。

1、建立私有CA（以Director主機爲例）
[root@localhost ~]# cd /etc/pki/CA
[root@localhost CA]# (umask 077; openssl genrsa -out private/cakey.pem 2048)
Generating RSA private key, 2048 bit long modulus
..................+++
.......+++
e is 65537 (0x10001)
[root@localhost CA]# touch index.txt
[root@localhost CA]# echo 01 > serial
[root@localhost CA]# openssl req -new -x509 -key private/cakey.pem -out cacert.pem -days 365
You are about to be asked to enter information that will be incorporated
into your certificate request.
What you are about to enter is what is called a Distinguished Name or a DN.
There are quite a few fields but you can leave some blank
For some fields there will be a default value,
If you enter '.', the field will be left blank.
-----
Country Name (2 letter code) [XX]:CN
State or Province Name (full name) []:XJ
Locality Name (eg, city) [Default City]:XJ
Organization Name (eg, company) [Default Company Ltd]:JJ
Organizational Unit Name (eg, section) []:Ops
Common Name (eg, your name or your server's hostname) []:CA
Email Address []:ca.admin.com
 
2、RS主機申請證書
[root@jymlinux ~]# cd /etc/httpd/
[root@jymlinux httpd]# mkdir ssl
[root@jymlinux httpd]# cd ssl
[root@jymlinux ssl]# (umask 077; openssl genrsa -out httpd.key 2048)
Generating RSA private key, 2048 bit long modulus
..............+++
..................................+++
e is 65537 (0x10001)
[root@jymlinux ssl]# openssl req -new -key httpd.key -out httpd.csr
You are about to be asked to enter information that will be incorporated
into your certificate request.
What you are about to enter is what is called a Distinguished Name or a DN.
There are quite a few fields but you can leave some blank
For some fields there will be a default value,
If you enter '.', the field will be left blank.
-----
Country Name (2 letter code) [XX]:CN
State or Province Name (full name) []:XJ
Locality Name (eg, city) [Default City]:XJ
Organization Name (eg, company) [Default Company Ltd]:JJ
Organizational Unit Name (eg, section) []:Ops
Common Name (eg, your name or your server's hostname) []:CA
Email Address []:rs1.admin.com
 
Please enter the following 'extra' attributes
to be sent with your certificate request
A challenge password []:
An optional company name []:
 
3、將申請的證書發往CA
[root@jymlinux ssl]# scp httpd.csr root@192.168.3.10:/root
The authenticity of host '192.168.3.10 (192.168.3.10)' can't be established.
RSA key fingerprint is ef:85:f8:aa:1c:de:41:5a:fd:93:8d:9f:83:f7:a2:ff.
Are you sure you want to continue connecting (yes/no)? y
Please type 'yes' or 'no': yes
Warning: Permanently added '192.168.3.10' (RSA) to the list of known hosts.
root@192.168.3.10's password: 
httpd.csr                                           100% 1013     1.0KB/s   00:00 
 
4、CA簽署併發證
[root@localhost CA]# openssl ca -in /root/httpd.csr -out /root/httpd.crt
Using configuration from /etc/pki/tls/openssl.cnf
Check that the request matches the signature
Signature ok
Certificate Details:
        Serial Number: 1 (0x1)
        Validity
            Not Before: Nov  9 13:48:21 2016 GMT
            Not After : Nov  9 13:48:21 2017 GMT
        Subject:
            countryName               = CN
            stateOrProvinceName       = XJ
            organizationName          = JJ
            organizationalUnitName    = Ops
            commonName                = CA
            emailAddress              = rs1.admin.com
        X509v3 extensions:
            X509v3 Basic Constraints: 
                CA:FALSE
            Netscape Comment: 
                OpenSSL Generated Certificate
            X509v3 Subject Key Identifier: 
                7E:FA:3A:6F:89:28:EF:D1:CF:5C:42:75:50:7B:C6:99:1D:98:91:B6
            X509v3 Authority Key Identifier: 
                keyid:91:9D:0E:8E:86:45:09:DE:C3:3F:63:61:C2:3D:CB:E1:E3:1C:F1:B6
 
Certificate is to be certified until Nov  9 13:48:21 2017 GMT (365 days)
Sign the certificate? [y/n]:y
 
 
1 out of 1 certificate requests certified, commit? [y/n]y
Write out database with 1 new entries
Data Base Updated
[root@localhost CA]# scp /root/httpd.crt root@192.168.3.20:/etc/httpd/ssl/
The authenticity of host '192.168.3.20 (192.168.3.20)' can't be established.
RSA key fingerprint is e5:84:6c:f7:c0:60:3d:0b:39:b6:1e:12:0d:48:8b:07.
Are you sure you want to continue connecting (yes/no)? yes
Warning: Permanently added '192.168.3.20' (RSA) to the list of known hosts.
root@192.168.3.20's password: 
httpd.crt                                           100% 4482     4.4KB/s   00:00  
 
4、安裝mod-ssl
[root@jymlinux ~]# yum install mod_ssl
 
5、修改ssl的配置文件
[root@jymlinux ~]# cd /etc/httpd/conf.d/
[root@jymlinux conf.d]# vim ssl.conf 
DocumentRoot "/var/www/html"                       #啓用
SSLCertificateFile /etc/httpd/ssl/httpd.crt        #修改證書以及密鑰的所在路徑
SSLCertificateKeyFile /etc/httpd/ssl/httpd.key
 
6、重啓httpd服務
[root@jymlinux conf.d]# service httpd restart
 
7、Director配置ipvsadm規則
#使用iptables在PREROUTING鏈上的MARK標記將http與https標記爲一組
[root@localhost ~]# iptables -t mangle -A PREROUTING -d 192.168.3.15 -p tcp --dport 80 -j MARK --set-mark 10
[root@localhost ~]# iptables -t mangle -A PREROUTING -d 192.168.3.15 -p tcp --dport 443 -j MARK --set-mark 10
 
[root@localhost ~]# ipvsadm -A -f 10 -s rr
[root@localhost ~]# ipvsadm -a -f 10 -r 192.168.3.20 -g
[root@localhost ~]# ipvsadm -a -f 10 -r 192.168.3.21 -g