heartrbeat實現web服務器高可用

時間 2019-12-05

原文原文鏈接

今天的內容是用heartbeat實現web服務器高可用html

1、簡介：node

heartbeat的工做原理：heartbeat最核心的包括兩個部分，心跳監測部分和資源接管部分，心跳監測能夠經過網絡鏈路和串口進行，並且支持冗餘鏈路，它們之間相互發送報文來告訴對方本身當前的狀態，若是在指定的時間內未收到對方發送的報文，那麼就認爲對方失效，這時需啓動資源接管模塊來接管運行在對方主機上的資源或者服務。c++

主機名	ip	系統	角色
tiandong63	192.168.199.3	rhel6.5	NFS存儲
tiandong64	192.168.199.4	rhel6.5	heartbeat主節點
tiandong66	192.168.199.6	rhel7.4	heartbeat從節點

heartbeat須要的幾個包web

Heartbeat-3-0-958e11be8686 # 心跳主程序包shell

resource-agents-3.9.6 #集羣實驗資源代理bootstrap

Reusable-Cluster-Components-glue--0a7add1d9996 #可重複使用的羣集組件vim

2、實戰：使用heartbeat實現web服務高可用api

一、拓撲圖：bash

二、配置tiandong63位NFS服務器，提供存儲資源：（https://www.cnblogs.com/winter1519/p/7396135.html 這裏有詳細的講解，有須要的能夠參考。）服務器

[root@tiandong63 ~]# yum install nfs-utils -y
[root@tiandong63 ~]# mkdir /wwwdir
[root@tiandong63 ~]# echo 'heartbeat test' > /wwwdir/index.html
[root@tiandong63 ~]# chmod -R 777 /wwwdir/
[root@tiandong63 ~]# more /etc/exports
/wwwdir 192.168.199.0/24(rw)
[root@tiandong63 ~]# /etc/init.d/nfs start
[root@tiandong63 ~]# chkconfig nfs on
[root@tiandong64 ~]# showmount -e 192.168.199.3
Export list for 192.168.199.3:
/wwwdir 192.168.199.0/24
三、tiandong64,66測試nfs 存儲掛載並安裝httpd web服務器(基本操做是一致)

[root@tiandong64 ~]# showmount -e 192.168.199.3
Export list for 192.168.199.3:
/wwwdir 192.168.199.0/24
[root@tiandong64 ~]# yum install httpd -y
[root@tiandong64 ~]# mount.nfs 192.168.199.3:/wwwdir/ /var/www/html/
[root@tiandong64 ~]# df -h
Filesystem              Size Used Avail Use% Mounted on
/dev/sda3                28G 2.3G   24G   9% /
tmpfs                   491M     0 491M   0% /dev/shm
/dev/sda1               194M   29M 155M 16% /boot
192.168.199.3:/wwwdir/   28G 5.5G   21G 22% /var/www/html
[root@tiandong64 ~]# /etc/init.d/httpd start
[root@tiandong64 ~]# curl 192.168.199.4
heartbeat test

卸載，關閉服務，後期這些資源經過heartbeat管理。
[root@tiandong64 ~]# umount /var/www/html/
[root@tiandong64 ~]# /etc/init.d/httpd stop
Stopping httpd: [ OK ]
[root@tiandong64 ~]# chkconfig httpd off
四、tiandong64安裝heartbeat（tiandong66安裝同樣）
1）安裝依賴包：
[root@tiandong64 ~]# yum install -y bzip2 bzip2-devel gcc gcc-c++ autoconf automake libtool e2fsprogs-devel glib2-devel libxml2 libxml2-devel libtool-ltdl-devel asciidoc libuuid-devel docbook
2）編譯安裝Reusable-Cluster-Components-glue--0a7add1d9996

[root@tiandong64 ~]#groupadd haclient 建立用戶組
[root@tiandong64 ~]#useradd -g haclient hacluster 建立用戶
[root@tiandong64 heartbeat]# cd Reusable-Cluster-Components-glue--0a7add1d9996/
[root@tiandong64 Reusable-Cluster-Components-glue--0a7add1d9996]# ./autogen.sh
[root@tiandong64 Reusable-Cluster-Components-glue--0a7add1d9996]#./configure --prefix=/usr/local/heartbeat/
[root@tiandong64 Reusable-Cluster-Components-glue--0a7add1d9996]#make && make install
3）編譯安裝resource-agents-3.9.6
[root@tiandong64 heartbeat]#cd resource-agents-3.9.6/
[root@tiandong64 resource-agents-3.9.6]#./autogen.sh
[root@tiandong64 resource-agents-3.9.6]#./configure --prefix=/usr/local/heartbeat --with-daemon-user=hacluster --with-daemon-group=haclient --enable-fatal-warnings=no LIBS='/lib64/libuuid.so.1'
[root@tiandong64 resource-agents-3.9.6]#make && make install
4）編譯安裝Heartbeat
[root@tiandong64 heartbeat]# cd Heartbeat-3-0-958e11be8686/
[root@tiandong64 Heartbeat-3-0-958e11be8686]# ./bootstrap
[root@tiandong64 Heartbeat-3-0-958e11be8686]#export CFLAGS="$CFLAGS -I/usr/local/heartbeat/include -L/usr/local/heartbeat/lib"
[root@tiandong64 Heartbeat-3-0-958e11be8686]#./configure --prefix=/usr/local/heartbeat --with-daemon-user=hacluster --with-daemon-group=haclient --enable-fatal-warnings=no LIBS='/lib64/libuuid.so.1'
[root@tiandong64 Heartbeat-3-0-958e11be8686]#make && make install
5）生成配置文件
[root@tiandong64 ~]# cd /usr/local/heartbeat/etc/ha.d/
[root@tiandong64 ha.d]# ls
harc rc.d README.config resource.d shellfuncs
[root@tiandong64 ha.d]# cp /root/heartbeat/Heartbeat-3-0-958e11be8686/doc/{authkeys,ha.cf,haresources} .
[root@tiandong64 ha.d]# ls
authkeys ha.cf harc haresources rc.d README.config resource.d shellfuncs

[root@tiandong64 ha.d]# chmod 600 authkeys
[root@tiandong64 ha.d]# mkdir -pv /usr/local/heartbeat/usr/lib/ocf/lib/heartbeat
mkdir: created directory `/usr/local/heartbeat/usr'
mkdir: created directory `/usr/local/heartbeat/usr/lib'
mkdir: created directory `/usr/local/heartbeat/usr/lib/ocf'
mkdir: created directory `/usr/local/heartbeat/usr/lib/ocf/lib'
mkdir: created directory `/usr/local/heartbeat/usr/lib/ocf/lib/heartbeat'
[root@tiandong64 ha.d]# cd !$
cd /usr/local/heartbeat/usr/lib/ocf/lib/heartbeat
[root@tiandong64 heartbeat]# pwd
/usr/local/heartbeat/usr/lib/ocf/lib/heartbeat
[root@tiandong64 heartbeat]# cp /usr/lib/ocf/lib/heartbeat/ocf-* .
[root@tiandong64 heartbeat]# ls
ocf-binaries ocf-directories ocf-rarun ocf-returncodes ocf-shellfuncs
[root@tiandong64 heartbeat]# ln -sv /usr/local/heartbeat/lib64/heartbeat/plugins/* /usr/local/heartbeat/lib/heartbeat/plugins/
`/usr/local/heartbeat/lib/heartbeat/plugins/HBauth' -> `/usr/local/heartbeat/lib64/heartbeat/plugins/HBauth'
`/usr/local/heartbeat/lib/heartbeat/plugins/HBcomm' -> `/usr/local/heartbeat/lib64/heartbeat/plugins/HBcomm'
`/usr/local/heartbeat/lib/heartbeat/plugins/quorum' -> `/usr/local/heartbeat/lib64/heartbeat/plugins/quorum'
`/usr/local/heartbeat/lib/heartbeat/plugins/tiebreaker' -> `/usr/local/heartbeat/lib64/heartbeat/plugins/tiebreaker'
[root@tiandong64 heartbeat]# chkconfig --add heartbeat
[root@tiandong64 heartbeat]# chkconfig heartbeat on

6）heartbeat配置

heartbeat的配置主要涉及到三個文件：

haresources 用來配置要讓Heartbeat託管的服務

authkeys 是用來指定Heartbeat的認證方式。

ha.cf 主配置文件

（1）主配置文件

[root@tiandong64 ~]# cd /usr/local/heartbeat/etc/ha.d/
[root@tiandong64 ha.d]# grep -v '^#' ha.cf
debugfile /var/log/ha-debug                             #用於記錄heartbeat的調試信息
logfile   /var/log/ha-log                                      #設置heartbeat的日誌,這裏用的是系統日誌
logfacility   local0                                              #設置heartbeat的日誌,這裏用的是系統日誌
keepalive 2                                                        #設定心跳(監測)時間時間爲2秒
deadtime 30                                                      #指定若備用節點在30秒內未收到主節點心跳信號,則接管主服務器資源
warntime 10                                                        #指定心跳延遲的時間爲10秒,10秒內備節點不能接收主節點心跳信號，
initdead 120                                                       #系統啓動或重啓後預留的忽略時間段，取值至少爲deadtime的兩倍
udpport   694                                                    #廣播/單播通信使用的Udp端口
ucast eth0 192.168.199.6                                   #採用網卡eth0的udp單播來組織心跳，後面跟的IP地址爲雙機對方IP地址
auto_failback on                                                 #定義當主節點恢復後，是否將服務自動切回（on是自動切回）
node   tiandong64                                             #主節點名稱
node   tiandong66                                             #備用節點名稱
ping 192.168.199.1                                            #經過ping網關檢測心跳是否正常，僅用來測試網絡
respawn hacluster /usr/local/heartbeat/libexec/heartbeat/ipfail                #指定和heartbeat一塊兒啓動、關閉的進程
apiauth ipfail gid=haclient uid=hacluster                                                   #設置啓動IPfail的用戶和組
資源配置文件

（2）[root@tiandong64 ha.d]# grep -v '^#' haresources 配置資源文件
tiandong64 IPaddr::192.168.199.111/24/eth0 Filesystem::192.168.199.3:/wwwdir::/var/www/html::nfs httpd

tiandong64 主服務器主機名
IPaddr::192.168.199.111/24/eth0   #指定VIP以及綁定的網卡
Filesystem::192.168.199.3:/wwwdir::/var/www/html::nfs   #指定掛載的存儲
httpd                       #指定要啓動的服務。這個服務必須是在/etc/init.d下或者/usr/local/heartbeat/etc/ha.d/resource.d目錄下
注：tiandong64是主服務器的主機名, tiandong66上不須要修改。這樣資源默認會加一這個主機上。當tiandong64壞了，tiandong66會再接管。

（3）認證文件

[root@tiandong64 ha.d]# vim authkeys 認證文件
auth 3
#1 crc
#2 sha1 HI!
3 md5 Hello!
（4）編寫httpd啓動腳本
[root@tiandong64 resource.d]# pwd
/usr/local/heartbeat/etc/ha.d/resource.d

[root@tiandong64 resource.d]# vim httpd
#!/bin/bash
/etc/init.d/httpd $1
[root@tiandong64 resource.d]# chmod 755 httpd

[root@tiandong64 ~]# cd /usr/local/heartbeat/etc/ha.d/
[root@tiandong64 ha.d]# ls
authkeys ha.cf harc haresources rc.d README.config resource.d shellfuncs

（5）配置文件複製到備機上。
[root@tiandong64 ha.d]# scp authkeys ha.cf haresources 192.168.199.6:`pwd`
[root@tiandong64 resource.d]# scp httpd 192.168.199.6:`pwd`

3、測試

一、手動加載VIP：192.168.199.111到網卡eth0上面。
[root@tiandong64 resource.d]# ./IPaddr 192.168.199.111/24/eth0 start
INFO: Using calculated netmask for 192.168.199.111: 255.255.255.0
DEBUG: Using calculated broadcast for 192.168.199.111: 192.168.199.255
INFO: eval ifconfig eth0:0 192.168.199.111 netmask 255.255.255.0 broadcast 192.168.199.255
DEBUG: Sending Gratuitous Arp for 192.168.199.111 on eth0:0 [eth0]
ARPING 192.168.199.111 from 192.168.199.111 eth0
INFO: Success
INFO: Success

[root@tiandong64 resource.d]# ip a
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 16436 qdisc noqueue state UNKNOWN
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
    inet6 ::1/128 scope host
       valid_lft forever preferred_lft forever
2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen 1000
    link/ether 00:0c:29:c7:20:71 brd ff:ff:ff:ff:ff:ff
    inet 192.168.199.4/24 brd 192.168.199.255 scope global eth0
    inet 192.168.199.111/24 brd 192.168.199.255 scope global secondary eth0:0
    inet6 fe80::20c:29ff:fec7:2071/64 scope link
       valid_lft forever preferred_lft forever

二、手動測試NFS掛載

[root@tiandong64 resource.d]# ./Filesystem 192.168.199.3:/wwwdir /var/www/html/ nfs start
INFO: Running start for 192.168.199.3:/wwwdir on /var/www/html
INFO: Success
INFO: Success
[root@tiandong64 resource.d]# df -h
Filesystem             Size Used Avail Use% Mounted on
/dev/sda3               28G 2.3G   24G   9% /
tmpfs                  491M     0 491M   0% /dev/shm
/dev/sda1              194M   29M 155M 16% /boot
192.168.199.3:/wwwdir   28G 5.5G   21G 22% /var/www/html

[root@tiandong64 resource.d]# /etc/init.d/httpd start 測試訪問

[root@tiandong64 resource.d]# curl 192.168.199.4
heartbeat test
[root@tiandong64 resource.d]# ./IPaddr 192.168.199.111/24/eth0 stop
SIOCDELRT: No such process
INFO: ifconfig eth0:0 down
INFO: Success
INFO: Success
[root@tiandong64 resource.d]# ./Filesystem 192.168.199.3:/wwwdir /var/www/html/ nfs stop
INFO: Running stop for 192.168.199.3:/wwwdir on /var/www/html
INFO: Trying to unmount /var/www/html
INFO: unmounted /var/www/html successfully
INFO: Success
INFO: Success
[root@tiandong64 resource.d]# df -h
Filesystem      Size Used Avail Use% Mounted on
/dev/sda3        28G 2.3G   24G   9% /
tmpfs           491M     0 491M   0% /dev/shm
/dev/sda1       194M   29M 155M 16% /boot
[root@tiandong64 resource.d]# /etc/init.d/httpd stop
[root@tiandong64 ~]# /etc/init.d/heartbeat start
Starting High-Availability services: INFO: Resource is stopped
                                                           [ OK ]
三、配置tiandong66（192.168.199.6）
[root@tiandong66 ~]# cd /usr/local/heartbeat/etc/ha.d/
[root@tiandong66 ha.d]# vim ha.cf
121 ucast ens33 192.168.199.4
[root@tiandong66 ha.d]# vim haresources
45 tiandong64 IPaddr::192.168.199.111/24/ens33 Filesystem::192.168.199.3:/wwwdir::/var/www/html::nfs httpd
[root@tiandong66 ha.d]# chmod 755 authkeys

四、兩臺機器啓動heartbeat服務

[root@tiandong64 ~]# /etc/init.d/heartbeat start
[root@tiandong66 ~]# systemctl start heartbeat

五、查看集羣
[root@tiandong64 ~]# ip a      查看VIP加載
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 16436 qdisc noqueue state UNKNOWN
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
    inet6 ::1/128 scope host
       valid_lft forever preferred_lft forever
2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen 1000
    link/ether 00:0c:29:c7:20:71 brd ff:ff:ff:ff:ff:ff
    inet 192.168.199.4/24 brd 192.168.199.255 scope global eth0
    inet 192.168.199.111/24 brd 192.168.199.255 scope global secondary eth0:0
    inet6 fe80::20c:29ff:fec7:2071/64 scope link
       valid_lft forever preferred_lft forever
[root@tiandong64 ~]# df -h|tail -1     查看掛載
192.168.199.3:/wwwdir   28G 5.5G   21G 22% /var/www/html

[root@tiandong64 ~]# netstat -antup|grep httpd 查看web服務
tcp 0 0 :::80 :::* LISTEN 9286/httpd
在備機上沒有這些資源。

六、故障模擬。

中止tiandong64上面的heartbeat服務

[root@tiandong64 ~]# /etc/init.d/heartbeat stop
[root@tiandong66 ~]# ip a |grep ens33
2: ens33: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen 1000
    inet 192.168.199.6/24 brd 192.168.199.255 scope global ens33
    inet 192.168.199.111/24 brd 192.168.199.255 scope global secondary ens33:0
[root@tiandong66 ~]# df -h|tail -1
192.168.199.3:/wwwdir   28G 5.5G   21G   22% /var/www/html
[root@tiandong66 ~]# netstat -antup|grep httpd
tcp6       0      0 :::80                   :::*                    LISTEN      14670/httpd
恢復tiandong64上面的服務，資源有切回來了。
[root@tiandong64 ~]# /etc/init.d/heartbeat start
Starting High-Availability services: INFO: Resource is stopped
                                                           [ OK ]
[root@tiandong64 ~]# ip a
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 16436 qdisc noqueue state UNKNOWN
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
    inet6 ::1/128 scope host
       valid_lft forever preferred_lft forever
2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen 1000
    link/ether 00:0c:29:c7:20:71 brd ff:ff:ff:ff:ff:ff
    inet 192.168.199.4/24 brd 192.168.199.255 scope global eth0
    inet 192.168.199.111/24 brd 192.168.199.255 scope global secondary eth0:1
    inet6 fe80::20c:29ff:fec7:2071/64 scope link
       valid_lft forever preferred_lft forever
[root@tiandong64 ~]# df -h|tail -1
192.168.199.3:/wwwdir   28G 5.5G   21G 22% /var/www/html
[root@tiandong64 ~]# netstat -antup|grep httpd
tcp        0      0 :::80                       :::*                        LISTEN      10646/httpd

OK，heartbeat就到這裏結束了，歡迎各位同窗的指教。
QQ：1127000493

tiandong66配置：

[root@tiandong66 ~]# chkconfig --add heartbeat
[root@tiandong66 ~]# chkconfig heartbeat on
[root@tiandong66 ~]# cd /usr/local/heartbeat/etc/ha.d/
[root@tiandong66 ha.d]# cp /root/heartbeat/Heartbeat-3-0-958e11be8686/doc/{authkeys,ha.cf,haresources} .
[root@tiandong66 ha.d]# chmod 600 authkeys
[root@tiandong66 ha.d]# mkdir -pv /usr/local/heartbeat/usr/lib/ocf/lib/heartbeat
mkdir: 已建立目錄 "/usr/local/heartbeat/usr"
mkdir: 已建立目錄 "/usr/local/heartbeat/usr/lib"
mkdir: 已建立目錄 "/usr/local/heartbeat/usr/lib/ocf"
mkdir: 已建立目錄 "/usr/local/heartbeat/usr/lib/ocf/lib"
mkdir: 已建立目錄 "/usr/local/heartbeat/usr/lib/ocf/lib/heartbeat"
[root@tiandong66 ha.d]# cd !$
cd /usr/local/heartbeat/usr/lib/ocf/lib/heartbeat
[root@tiandong66 heartbeat]# cp /usr/lib/ocf/lib/heartbeat/ocf-* .
[root@tiandong66 heartbeat]# ln -sv /usr/local/heartbeat/lib64/heartbeat/plugins/* /usr/local/heartbeat/lib/heartbeat/plugins/
"/usr/local/heartbeat/lib/heartbeat/plugins/HBauth" -> "/usr/local/heartbeat/lib64/heartbeat/plugins/HBauth"
"/usr/local/heartbeat/lib/heartbeat/plugins/HBcomm" -> "/usr/local/heartbeat/lib64/heartbeat/plugins/HBcomm"
"/usr/local/heartbeat/lib/heartbeat/plugins/quorum" -> "/usr/local/heartbeat/lib64/heartbeat/plugins/quorum"
"/usr/local/heartbeat/lib/heartbeat/plugins/tiebreaker" -> "/usr/local/heartbeat/lib64/heartbeat/plugins/tiebreaker"
[root@tiandong66 heartbeat]#