RHEL5.4 Heartbeat安裝(第二部份 測試與監控)

9) 啓動服務 監視服務

將 HA1和HA2的heartheat 服務啓動

/etc/init.d/heartbeat start

監視服務：

首先在HA1上查看 messages

#cat /var/log/messages

Sep 19 15:56:37 HA1 heartbeat: [26814]: info: Version 2 support: false

Sep 19 15:56:37 HA1 heartbeat: [26814]: WARN: Logging daemon is disabled --enabling logging daemon is recommended

Sep 19 15:56:37 HA1 heartbeat: [26814]: info: **************************

Sep 19 15:56:37 HA1 heartbeat: [26814]: info: Configuration validated. Starting heartbeat 3.0.2

Sep 19 15:56:37 HA1 heartbeat: [26815]: info: heartbeat: version 3.0.2

Sep 19 15:56:37 HA1 heartbeat: [26815]: info: Heartbeat generation: 1284708296

Sep 19 15:56:37 HA1 heartbeat: [26815]: info: glib: UDP Broadcast heartbeat started on port 694 (694) interface eth1

Sep 19 15:56:37 HA1 heartbeat: [26815]: info: glib: UDP Broadcast heartbeat closed on port 694 interface eth1 - Status: 1

Sep 19 15:56:37 HA1 heartbeat: [26815]: info: G_main_add_TriggerHandler: Added signal manual handler

Sep 19 15:56:37 HA1 heartbeat: [26815]: info: G_main_add_TriggerHandler: Added signal manual handler

Sep 19 15:56:37 HA1 heartbeat: [26815]: info: G_main_add_SignalHandler: Added signal handler for signal 17

Sep 19 15:56:37 HA1 heartbeat: [26815]: info: Local status now set to: 'up'

Sep 19 15:56:41 HA1 heartbeat: [26815]: info: Link ha1:eth1 up.

Sep 19 15:56:47 HA1 heartbeat: [26815]: info: Link ha2:eth1 up.

Sep 19 15:56:47 HA1 heartbeat: [26815]: info: Status update for node ha2: status up

Sep 19 15:56:47 HA1 harc[26822]: info: Running /usr/etc/ha.d//rc.d/status status

Sep 19 15:56:47 HA1 heartbeat: [26815]: info: Comm_now_up(): updating status to active

Sep 19 15:56:47 HA1 heartbeat: [26815]: info: Local status now set to: 'active'

Sep 19 15:56:48 HA1 heartbeat: [26815]: info: Status update for node ha2: status active

Sep 19 15:56:48 HA1 harc[26842]: info: Running /usr/etc/ha.d//rc.d/status status

Sep 19 15:57:04 HA1 heartbeat: [26815]: info: remote resource transition completed.

Sep 19 15:57:04 HA1 heartbeat: [26815]: info: remote resource transition completed.

Sep 19 15:57:04 HA1 heartbeat: [26815]: info: Initial resource acquisition complete (T_RESOURCES(us))

Sep 19 15:57:04 HA1 IPaddr[26898]: INFO:  Resource is stopped

Sep 19 15:57:04 HA1 heartbeat: [26862]: info: Local Resource acquisition completed.

Sep 19 15:57:04 HA1 harc[26941]: info: Running /usr/etc/ha.d//rc.d/ip-request-resp ip-request-resp

Sep 19 15:57:04 HA1 ip-request-resp[26941]: received ip-request-resp IPaddr::172.16.6.66/21/eth0 OK yes

Sep 19 15:57:04 HA1 ResourceManager[26964]: info: Acquiring resource group: ha1 IPaddr::172.16.6.66/21/eth0 test

Sep 19 15:57:05 HA1 IPaddr[26992]: INFO:  Resource is stopped

Sep 19 15:57:05 HA1 ResourceManager[26964]: info: Running /etc/ha.d/resource.d/IPaddr 172.16.6.66/21/eth0 start

Sep 19 15:57:05 HA1 IPaddr[27077]: INFO: Using calculated netmask for 172.16.6.66: 255.255.248.0

Sep 19 15:57:05 HA1 IPaddr[27077]: INFO: eval ifconfig eth0:0 172.16.6.66 netmask 255.255.248.0 broadcast 172.16.7.255

Sep 19 15:57:05 HA1 IPaddr[27051]: INFO:  Success

Sep 19 15:57:05 HA1 logger: /etc/ha.d/resource.d/test called with status

Sep 19 15:57:05 HA1 ResourceManager[26964]: info: Running /etc/ha.d/resource.d/test  start

Sep 19 15:57:05 HA1 logger: /etc/ha.d/resource.d/test called with start

 

 

能夠看到HA1和HA2都啓動起來了  咱們的test腳本也運行了。咱們的 172.16.6.66的IP也啓來了。

而後再去 ha-log裏面看一下

 

[root@HA1 ~]# cat /var/log/ha-log

Sep 19 15:56:37 HA1 heartbeat: [26814]: info: Version 2 support: false

Sep 19 15:56:37 HA1 heartbeat: [26814]: WARN: Logging daemon is disabled --enabling logging daemon is recommended

Sep 19 15:56:37 HA1 heartbeat: [26814]: info: **************************

Sep 19 15:56:37 HA1 heartbeat: [26814]: info: Configuration validated. Starting heartbeat 3.0.2

Sep 19 15:56:37 HA1 heartbeat: [26815]: info: heartbeat: version 3.0.2

Sep 19 15:56:37 HA1 heartbeat: [26815]: info: Heartbeat generation: 1284708296

Sep 19 15:56:37 HA1 heartbeat: [26815]: info: glib: UDP Broadcast heartbeat started on port 694 (694) interface eth1

Sep 19 15:56:37 HA1 heartbeat: [26815]: info: glib: UDP Broadcast heartbeat closed on port 694 interface eth1 - Status: 1

Sep 19 15:56:37 HA1 heartbeat: [26815]: info: G_main_add_TriggerHandler: Added signal manual handler

Sep 19 15:56:37 HA1 heartbeat: [26815]: info: G_main_add_TriggerHandler: Added signal manual handler

Sep 19 15:56:37 HA1 heartbeat: [26815]: info: G_main_add_SignalHandler: Added signal handler for signal 17

Sep 19 15:56:37 HA1 heartbeat: [26815]: info: Local status now set to: 'up'

Sep 19 15:56:41 HA1 heartbeat: [26815]: info: Link ha1:eth1 up.

Sep 19 15:56:47 HA1 heartbeat: [26815]: info: Link ha2:eth1 up.

Sep 19 15:56:47 HA1 heartbeat: [26815]: info: Status update for node ha2: status up

harc[26822]:    2010/09/19_15:56:47 info: Running /usr/etc/ha.d//rc.d/status status

Sep 19 15:56:47 HA1 heartbeat: [26815]: info: Comm_now_up(): updating status to active

Sep 19 15:56:47 HA1 heartbeat: [26815]: info: Local status now set to: 'active'

Sep 19 15:56:48 HA1 heartbeat: [26815]: info: Status update for node ha2: status active

harc[26842]:    2010/09/19_15:56:48 info: Running /usr/etc/ha.d//rc.d/status status

Sep 19 15:57:04 HA1 heartbeat: [26815]: info: remote resource transition completed.

Sep 19 15:57:04 HA1 heartbeat: [26815]: info: remote resource transition completed.

Sep 19 15:57:04 HA1 heartbeat: [26815]: info: Initial resource acquisition complete (T_RESOURCES(us))

IPaddr[26898]:         2010/09/19_15:57:04 INFO:  Resource is stopped

Sep 19 15:57:04 HA1 heartbeat: [26862]: info: Local Resource acquisition completed.

harc[26941]:    2010/09/19_15:57:04 info: Running /usr/etc/ha.d//rc.d/ip-request-resp ip-request-resp

ip-request-resp[26941]:  2010/09/19_15:57:04 received ip-request-resp IPaddr::172.16.6.66/21/eth0 OK yes

ResourceManager[26964]:     2010/09/19_15:57:04 info: Acquiring resource group: ha1 IPaddr::172.16.6.66/21/eth0 test

IPaddr[26992]:         2010/09/19_15:57:05 INFO:  Resource is stopped

ResourceManager[26964]:     2010/09/19_15:57:05 info: Running /etc/ha.d/resource.d/IPaddr 172.16.6.66/21/eth0 start

IPaddr[27077]:         2010/09/19_15:57:05 INFO: Using calculated netmask for 172.16.6.66: 255.255.248.0

IPaddr[27077]:         2010/09/19_15:57:05 INFO: eval ifconfig eth0:0 172.16.6.66 netmask 255.255.248.0 broadcast 172.16.7.255

IPaddr[27051]:         2010/09/19_15:57:05 INFO:  Success

ResourceManager[26964]:     2010/09/19_15:57:05 info: Running /etc/ha.d/resource.d/test  start

 

內容和messages裏面的差很少。

 

HA2裏面的日誌

Sep 19 23:57:24 HA2 heartbeat: [14041]: info: Version 2 support: false

Sep 19 23:57:24 HA2 heartbeat: [14041]: WARN: Logging daemon is disabled --enabling logging daemon is recommended

Sep 19 23:57:24 HA2 heartbeat: [14041]: info: **************************

Sep 19 23:57:24 HA2 heartbeat: [14041]: info: Configuration validated. Starting heartbeat 3.0.2

Sep 19 23:57:24 HA2 heartbeat: [14042]: info: heartbeat: version 3.0.2

Sep 19 23:57:24 HA2 heartbeat: [14042]: info: Heartbeat generation: 1284893027

Sep 19 23:57:24 HA2 heartbeat: [14042]: info: glib: UDP Broadcast heartbeat started on port 694 (694) interface eth1

Sep 19 23:57:24 HA2 heartbeat: [14042]: info: glib: UDP Broadcast heartbeat closed on port 694 interface eth1 - Status: 1

Sep 19 23:57:24 HA2 heartbeat: [14042]: info: G_main_add_TriggerHandler: Added signal manual handler

Sep 19 23:57:24 HA2 heartbeat: [14042]: info: G_main_add_TriggerHandler: Added signal manual handler

Sep 19 23:57:24 HA2 heartbeat: [14042]: info: G_main_add_SignalHandler: Added signal handler for signal 17

Sep 19 23:57:24 HA2 heartbeat: [14042]: info: Local status now set to: 'up'

Sep 19 23:57:26 HA2 heartbeat: [14042]: info: Link ha1:eth1 up.

Sep 19 23:57:26 HA2 heartbeat: [14042]: info: Status update for node ha1: status up

Sep 19 23:57:26 HA2 heartbeat: [14042]: info: Link ha2:eth1 up.

Sep 19 23:57:26 HA2 harc[14049]: info: Running /usr/etc/ha.d//rc.d/status status

Sep 19 23:57:26 HA2 heartbeat: [14042]: info: Comm_now_up(): updating status to active

Sep 19 23:57:26 HA2 heartbeat: [14042]: info: Local status now set to: 'active'

Sep 19 23:57:26 HA2 heartbeat: [14042]: info: Status update for node ha1: status active

Sep 19 23:57:26 HA2 harc[14067]: info: Running /usr/etc/ha.d//rc.d/status status

Sep 19 23:57:42 HA2 heartbeat: [14042]: info: local resource transition completed.

Sep 19 23:57:42 HA2 heartbeat: [14042]: info: Initial resource acquisition complete (T_RESOURCES(us))

Sep 19 23:57:42 HA2 heartbeat: [14086]: info: No local resources [/usr/share/heartbeat/ResourceManager listkeys ha2] to acquire.

Sep 19 23:57:43 HA2 heartbeat: [14042]: info: remote resource transition completed.

 

 

會看到 info: No local resources [/usr/share/heartbeat/ResourceManager listkeys ha2] to acquire.

說明沒有任何本地資源，該機器將做爲備份服務器並閒置，它只監聽來自主服務器的心跳直到主服務器失敗爲止。

 

# tcpdump -i eth1 -n -p udp port 694

能夠查看到 eth1過來的心跳廣播。以下所示：

23:06:49.576155 IP 10.0.0.1.40661 > 10.0.0.255.ha-cluster: UDP, length 174

23:06:49.734999 IP 10.0.0.2.50487 > 10.0.0.255.ha-cluster: UDP, length 174

23:06:50.324281 IP 10.0.0.1.40661 > 10.0.0.255.ha-cluster: UDP, length 167

23:06:50.324283 IP 10.0.0.1.40661 > 10.0.0.255.ha-cluster: UDP, length 174

23:06:50.486151 IP 10.0.0.2.50487 > 10.0.0.255.ha-cluster: UDP, length 174

 

10) 模擬故障

咱們如今把主服務器的電源直接拔掉 模擬宕機 再找一臺機器一直ping 172.16.6.66

 

咱們會在ha2的ha-log裏面看到以下信息

Sep 20 00:03:47 HA2 heartbeat: [14042]: WARN: node ha1: is dead

Sep 20 00:03:47 HA2 heartbeat: [14042]: WARN: No STONITH device configured.

Sep 20 00:03:47 HA2 heartbeat: [14042]: WARN: Shared disks are not protected.

Sep 20 00:03:47 HA2 heartbeat: [14042]: info: Resources being acquired from ha1.

Sep 20 00:03:47 HA2 heartbeat: [14042]: info: Link ha1:eth1 dead.

harc[14105]:    2010/09/20_00:03:48 info: Running /usr/etc/ha.d//rc.d/status status

Sep 20 00:03:48 HA2 heartbeat: [14106]: info: No local resources [/usr/share/heartbeat/ResourceManager listkeys ha2] to acquire.

mach_down[14135]:        2010/09/20_00:03:48 info: Taking over resource group IPaddr::172.16.6.66/21/eth0

ResourceManager[14162]:     2010/09/20_00:03:48 info: Acquiring resource group: ha1 IPaddr::172.16.6.66/21/eth0 test

IPaddr[14190]:         2010/09/20_00:03:48 INFO:  Resource is stopped

ResourceManager[14162]:     2010/09/20_00:03:48 info: Running /etc/ha.d/resource.d/IPaddr 172.16.6.66/21/eth0 start

IPaddr[14275]:         2010/09/20_00:03:48 INFO: Using calculated netmask for 172.16.6.66: 255.255.248.0

IPaddr[14275]:         2010/09/20_00:03:48 INFO: eval ifconfig eth0:0 172.16.6.66 netmask 255.255.248.0 broadcast 172.16.7.255

IPaddr[14249]:         2010/09/20_00:03:48 INFO:  Success

ResourceManager[14162]:     2010/09/20_00:03:48 info: Running /etc/ha.d/resource.d/test  start

mach_down[14135]:        2010/09/20_00:03:48 info: /usr/share/heartbeat/mach_down: nice_failback: foreign resources acquired

mach_down[14135]:        2010/09/20_00:03:49 info: mach_down takeover complete for node ha1.

Sep 20 00:03:49 HA2 heartbeat: [14042]: info: mach_down takeover complete.

 

 

咱們會看到

node ha1: is dead

說明 ha1以經宕機 

資源腳本首先用status參數調用 而後用start參數啓動test 腳本。以完成故障轉移。

也能夠從messages 裏面看到

logger: /etc/ha.d/resource.d/test called with start

說明咱們的test腳本以經在HA2中運行了。

再看一下IP地址

HA2的eht0:0以經有了 是172.16.6.66

 

一旦完成故障轉移，則備份服務器會再次監視主服務器的心跳 若是主服務器啓動則會再將服務轉移回主服務器。

 

試驗成功。

 若是不須要 主服務器恢復後自動得到主權限 要在ha.cf中 加入一條 auto_failback on