Heartbeat爲我們提供了廉價的、可伸縮的高可用集羣方案。我們通過heartbeat+drbd+nfs在Linux下創建一個高可用(HA)的集羣 服務器。
一．DRBD是一種塊設備，可以被用於高可用(HA)之中。它類似於一個網絡RAID-1功能。當你將數據寫入本地文件系統時，數據還將會被髮送到網絡中另一臺主機上。以相同的形式記錄在一個文件系統中。本地(主節點)與遠程主機(備節點)的數據可以保證實時同步。當本地系統出現故障時，遠程主機上還會保留有一份相同的數據，可以繼續使用。
二．案例拓撲：
一．Heratbeat 配置
1.修改hosts文件
[[email protected] ~]# vim /etc/hosts
127.0.0.1 localhost.localdomain localhost
::1 localhost6.localdomain6 localhost6
192.168.10.1 node1.a.com
192.168.10.2 node2.a.com
2.修改network文件
[[email protected] ~]# hostname node1.a.com
[[email protected] ~]# vim /etc/sysconfig/network
NETWORKING=yes
NETWORKING_IPV6=no
HOSTNAME=localhost.localdomain
NISDOMAIN=node1.a.com
3.配置yum源
[[email protected] ~]# hwclock -s //同步時間
[[email protected] ~]# mkdir /mnt/cdrom
[[email protected] ~]# mount /dev/cdrom /mnt/cdrom
[[email protected] ~]# vim /etc/yum.repos.d/rhel-debuginfo.repo
[rhel-server]
name=Red Hat Enterprise Linux server
baseurl=file:///mnt/cdrom/Server
enabled=1
gpgcheck=0
gpgkey=file:///mnt/cdrom/RPM-GPG-KEY-redhat-release
[rhel-cluster]
name=Red Hat Enterprise Linux cluster
baseurl=file:///mnt/cdrom/Cluster
enabled=1
gpgcheck=0
gpgkey=file:///mnt/cdrom/RPM-GPG-KEY-redhat-release
node1與node2 的配置同樣。
[[email protected] ~]# scp /etc/yum.repos.d/rhel-debuginfo.repo node2.a.com:/etc/yum.repos.d/
Heartbeat
4.安裝heartbeat所需的包。
[[email protected] ~]# yum localinstall heartbeat-2.1.4-9.el5.i386.rpm heartbeat-lnoded-2.1.4-9.el5.i386.rpm heartbeat-pils-2.1.4-10.el5.i386.rpm heartbeat-stonith-2.1.4-10.el5.i386.rpm libnet-1.1.4-3.el5.i386.rpm perl-MailTools-1.77-1.el5.noarch.rpm --nogpgcheck -y
[[email protected] ~]# cd /usr/share/doc/heartbeat-2.1.4/
[[email protected] heartbeat-2.1.4]# cp authkeys haresources ha.cf /etc/ha.d/
[[email protected] heartbeat-2.1.4]# vim ha.cf
94 bcast eth1 //產生廣播監聽eth1
213 node node1.a.com //定義節點
214 node node2.a.com
5.驗證使用md5 認證密文
[[email protected] ~]# echo "123" |openssl md5
ba1f2511fc30423bdbb183fe33f3dd0f
[[email protected] heartbeat-2.1.4]# vim authkeys
auth 3
3 md5 ba1f2511fc30423bdbb183fe33f3dd0f
[[email protected] ha.d]# chmod 600 authkeys //修改權限
6.修改資源文件/etc/ha.d/haresources，定義HA羣集的資源
[[email protected] ~]# cp /usr/share/doc/heartbeat-lnoded-2.1.4/lnoded.cf /etc/ha.d/
[[email protected] ha.d]# vim haresources
node1.a.com IPaddr::192.168.10.254/24/eth0 drbddisk::web Filesystem::/dev/drbd0::/public::ext3 killnfsd
[[email protected] ~]# vim /etc/exports
/public 192.168.10.0/24(rw,sync)
二．DRBD的配置
1 簡介：
Distributed Replicated Block Device(DRBD)是一個用軟件實現的、無共享的、服務器之間鏡像塊設備內容的存儲複製解決方案。Node1，node2上DRBD 配置
1.添加一塊磁盤分區
[[email protected] ~]# fdisk /dev/sdb
Command (m for help): n
Command action
e extended
p primary partition (1-4)
p
Partition number (1-4): 1
First cylinder (1-1044, default 1):
Using default value 1
Last cylinder or +size or +sizeM or +sizeK (1-1044, default 1044): +1G
Command (m for help): P
Disk /dev/sdb: 8589 MB, 8589934592 bytes
255 heads, 63 sectors/track, 1044 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
Device Boot Start End Blocks Id System
/dev/sdb1 1 123 987966 83 Linux
Command (m for help): W
The partition table has been altered!
2.修改hosts
[[email protected] ~]# vim /etc/hosts
192.168.10.1 node1.a.com
192.168.10.2 node2.a.com
[[email protected] ~]# scp /etc/hosts 192.168.20.2:/etc/
[[email protected] ~]# rpm -ivh drbd83-8.3.8-1.el5.centos.i386.rpm //需要網上下載。
[[email protected] ~]# rpm -ivh kmod-drbd83-8.3.8-1.el5.centos.i686.rpm
3.編輯配置文件
[[email protected] ~]# vim /etc/drbd.conf //在裏面輸入如下
:r /usr/share/doc/drbd83-8.3.8/drbd.conf
[[email protected] ~]# cd /etc/drbd.d/
[[email protected] drbd.d]# vim global_common.conf // 1,$d 幹掉所有添如下內容
global {
usage-count yes;
# minor-count dialog-refresh disable-ip-verification
}
common {
protocol C;
startup {
wfc-timeout 120;
degr-wfc-timeout 120;
}
disk {
on-io-error detach;
fencing resource-only;
}
net {
cram-hmac-alg "sha1";
shared-secret "mydrbdlab";
}
syncer {
rate 100M;
}
}
[[email protected] drbd.d]# vim web.res
resource web {
on node1.a.com {
device /dev/drbd0;
disk /dev/sdb1;
address 192.168.10.1:7789;
meta-disk internal;
}
on node2.a.com {
device /dev/drbd0;
disk /dev/sdb1;
address 192.168.10.2:7789;
meta-disk internal;
}
}
5.開始初始化雙方借點上都要執行
[[email protected] drbd.d]# drbdadm create-md web
6.在雙方的節點上啓動服務 [[email protected] drbd.d]#service drbd start
7. 查看狀態 [[email protected]1~]# cat /proc/drbd
version: 8.3.8 (api:88/proto:86-94)
GIT-hash: d78846e52224fd00562f7c225bcc25b2d422321d build by [email protected], 2010-06-04 08:04:16
0: cs:Connected ro:Secondary/Secondary ds:Inconsistent/Inconsistent C r----
ns:0 nr:0 dw:0 dr:0 al:0 bm:0 lo:0 pe:0 ua:0 ap:0 ep:1 wo:b oos:987896
都爲second 狀態，沒有同步
也可以
[[email protected] ~]# drbd-overview
0:web Connected Secondary/Secondary Inconsistent/Inconsistent C r----
8.在主機點上執行
[[email protected] ~]# drbdadm -- --overwrite-data-of-peer primary web
然後查看同步過程 watch -n 1 'cat /proc/drbd' //一般速度比較快不容易發現
[[email protected] ~]# drbd-overview //節點1是主 節點2是備
0:web Connected Primary/Secondary UpToDate/UpToDate C r----
[[email protected] ~]# drbd-overview
0:web Connected Secondary/Primary UpToDate/UpToDate C r----
9.創建文件系統（在主節點上實現）
[[email protected] ~]# mkfs -t ext3 -L drbdweb /dev/drbd0 //-L 表示卷標
[[email protected] ~]# mkdir /mnt/public
[[email protected] ~]# mount /dev/drbd0 /mnt/public
[[email protected] ~]# cd /mnt/public
10.執行I/O操作
[[email protected] public]# cp /etc/grub.conf ./
[[email protected] public]# ll
-rw------- 1 root root 746 01-19 21:11 grub.conf
drwx------ 2 root root 16384 01-19 21:03 lost+found
[[email protected] public]# cd
[[email protected] ~]# umount /mnt/public
[[email protected] ~]# drbdadm secondary web
[[email protected] ~]# drbd-overview
0:web Connected Secondary/Secondary UpToDate/UpToDate C r----
[[email protected] ~]# drbdadm primary web //手動設置節點2爲主
[[email protected] ~]# drbd-overview
0:web Connected Primary/Secondary UpToDate/UpToDate C r---- //本身設備主/對方設備輔
[[email protected] ~]# mount /dev/drbd0 /mnt/public
[[email protected] ~]# cd /mnt/public
[[email protected] 1]# ll
總計 20
-rw------- 1 root root 746 2013-01-19 grub.conf
drwx------ 2 root root 16384 2013-01-19 lost+found
[[email protected] public]# cp /etc/passwd ./
[[email protected] public# cd
[[email protected] ~]# umount /mnt/1
[[email protected] ~]# drbdadm secondary web
11.切換到主節點1上查看
[[email protected] ~]# drbdadm primary web
[[email protected] ~]# mount /dev/drbd0 /public
[[email protected] ~]# cd /public
[[email protected] public]# ll
總計 24
-rw------- 1 root root 746 01-19 21:11 grub.conf //節點1上的I/O操作
drwx------ 2 root root 16384 01-19 21:03 lost+found
-rw-r--r-- 1 root root 1560 01-09 13:43 passwd //節點2上的I/O操作
只有手動將一個節點設置爲主節點時，纔可訪問該節點下的內容，不符合智能化的要求，可以使用heartbeat工具，將drbd作爲其資源，實現主節點和 備份節點的自動切換。在兩個節點上都安裝nfs服務，將分區共享出來，這樣客戶端就可以看到這些內容
三．NFS搭建
1.配置node1.a.com的NFS server： node2上做同樣的配置
NFS服務是默認安裝的，使用時直接啓動即可；
# service nfs start # service portmap start
2.編寫共享清單：
[[email protected] ~]# vim /etc/exports
/public 192.168.10.0/24(rw,sync)
# exportfs -rv 導出共享清單：
exporting 192.168.10.0/24:/public -r 導出； -v 詳細信息；
3.修改nfs啓動腳本：
# vim /etc/init.d/nfs
122 killproc nfsd -9
4.改變nfs服務的資源控制文件的權限：
# chmod 755 /etc/ha.d/resource.d/killnfsd
5.創建nfs服務的資源控制文件：
# vim /etc/ha.d/resource.d/killnfsd
killall -9 nfsd ; /etc/init.d/nfs restart ; exit 0
6.改變nfs服務的資源控制文件的權限：
# chmod 755 /etc/ha.d/resource.d/killnfsd
啓動node1.a.com節點的heartbeat的服務：
# service heartbeat start
啓動node2.a.com節點的heartbeat的服務：
# service heartbeat start
7.查看heartbeat兩節點的狀態：
[[email protected] ~]# drbd-overview
0:web Connected Primary/Secondary UpToDate/UpToDate C r---- /public ext3 950M 18M 885M 2%
四．測試：
1. 開啓一個客戶端：
配置一個臨時地址： # ifconfig eth0 192.168.10.3
創建一個掛載點：
# mkdir /mnt/nfs # mount 192.168.10.254:/public /mnt/nfs
查看掛載的狀態：
2 當heartbeat正常運行時： 查看NFS網絡文件系統：
[[email protected] ~]# cd /mnt/nfs
[[email protected] nfs]# ll
-rw------- 1 root root 746 01-19 16:11 grub.conf
drwx------ 2 root root 16384 01-19 16:03 lost+found
-rw-r--r-- 1 root root 1560 01-09 08:43 passw
3.在node1.a.com中將heartbeat服務關閉：
[[email protected] ~]# service heartbeat stop
[[email protected] ~]# drbd-overview
[[email protected] ~]# ifconfig
[[email protected] ~]# mount
nfsd on /proc/fs/nfsd type nfsd (rw)
/dev/drbd0 on /public type ext3 (rw)
4.NFS網絡文件系統仍然可以使用
[[email protected] ~]# cd /mnt/nfs
[[email protected] nfs]# ll
總計 24
-rw------- 1 root root 746 01-19 16:11 grub.conf
drwx------ 2 root root 16384 01-19 16:03 lost+found
-rw-r--r-- 1 root root 1560 01-09 08:43 passwd
通過上述案例可以實現高可用性，以及通過heartbeart控制DRDB的自動切換，而且保證文件系統的高可用性。
4.定義資源