corosync+drbd+mysql實現的高可用
要求：html
1、可以在同一網段內直接通訊node
2、節點名稱，要和uname的結果同樣，並保證能夠根據節點名稱解析到節點的IP地址，配置本地/etc/hostsmysql
3、SSH互信通訊linux
4、保證時間同步算法
環境準備配置：sql
test1,192.168.10.55配置數據庫
一、配置IP地址bootstrap
[root@test1 ~]# vim /etc/sysconfig/network-scripts/ifcfg-eth0
二、配置主機名vim
[root@test1 ~]# uname -n [root@test1 ~]# hostname master1.local#臨時生效 [root@test1 ~]# vim /etc/sysconfig/network#永久生效
三、配置主機名解析centos
[root@test1 ~]# vim /etc/hosts 添加： 192.168.10.55master1.local 192.168.10.56master2.local
3.二、測試主機名通訊
[root@test1 ~]# ping master1.local [root@test1 ~]# ping master2.local
四、配置SSH互信認證
[root@test1 ~]# ssh-keygen -t rsa -P '' [root@test1 ~]# ssh-copy-id -i .ssh/id_rsa.pub root@192.168.10.55
五、使用ntp同步時間
在crontab中加入每5分鐘執行一次ntpdate命令，用來保證服務器時間是同步的
[root@test1 ~]# crontab -e */5 * * * * /sbin/ntpdate 192.168.10.1 &> /dev/null
test2,192.168.10.56配置
一、配置IP地址
[root@test2 ~]# vim /etc/sysconfig/network-scripts/ifcfg-eth0
二、配置主機名
[root@test2 ~]# uname -n [root@test2 ~]# hostname test2.local#臨時生效 [root@test2 ~]# vim /etc/sysconfig/network#永久生效
三、配置主機名解析
[root@test2 ~]# vim /etc/hosts 添加： 192.168.10.55test1.localtest1 192.168.10.56test2.localtest2
3.二、測試主機名通訊
[root@test2 ~]# ping test1.local [root@test2 ~]# ping test1
四、配置SSH互信認證
[root@test2 ~]# ssh-keygen -t rsa -P '' [root@test2 ~]# ssh-copy-id -i .ssh/id_rsa.pub root@192.168.10.56
五、使用ntp同步時間
在crontab中加入每5分鐘執行一次ntpdate命令，用來保證服務器時間是同步的
[root@test2 ~]# crontab -e */5 * * * * /sbin/ntpdate 192.168.10.1 &> /dev/null
安裝配置heartbeat
CentOS直接yum安裝報錯，提示找不到可用的軟件包
解決辦法：
[root@test1 src]# wget http://mirrors.sohu.com/fedora-epel/6/i386/epel-release-6-8.noarch.rpm [root@test1 src]# rpm -ivh epel-release-6-8.noarch.rpm
6.一、安裝heartbeat：
[root@test1 src]# yum install heartbeat
6.二、copy配置文件：
[root@test1 src]# cp /usr/share/doc/heartbeat-3.0.4/{ha.cf,authkeys,haresources} /etc/ha.d/
6.三、配置認證文件：
[root@test1 src]# dd if=/dev/random count=1 bs=512 |md5sum #生成隨機數 [root@test1 src]# vim /etc/ha.d/authkeys auth 1 1 md5 d0f70c79eeca5293902aiamheartbeat [root@test1 src]# chmod 600 authkeys
test2節點的heartbeat安裝和test1同樣，此處略過。
6.四、heartbeat主配置文件參數：
[root@test2 ~]# vim /etc/ha.d/ha.cf #debugfile /var/log/ha-debug #排錯日誌 logfile #日誌位置 keepalive 2 #多長時間發送一次心跳檢測，默認2秒，可使用ms deadtime 30 #多長時間檢測不到主機就認爲掛掉 warntime 10 #若是沒有收到心跳信息，那麼在等待多長時間就認爲對方掛掉 initdead 120 #第一個節點起來後，等待其餘節點的時間 baud 19200 #串行線纜的發送速率是多少 auto_failback on #故障恢復後是否轉移回來 ping 10.10.10.254 #ping node，萬一節點主機不通，要ping哪一個主機 ping_group group1 10.10.10.254 10.10.10.253 #ping node group，只要組內有一臺主機能ping通就能夠 respawn hacluster /usr/lib/heartbeat/ipfail #當一個heartbeat服務中止了，會重啓對端的heartbeat服務 deadping 30 #ping nodes多長時間ping不通，就真的故障了 # serial serialportname ... #串行設備是什麼 serial /dev/ttyS0 # Linux serial /dev/cuaa0 # FreeBSD serial /dev/cuad0 # FreeBSD 6.x serial /dev/cua/a # Solaris # What interfaces to broadcast heartbeats over? #若是使用以太網，定義使用單播、組播仍是廣播發送心跳信息 bcast eth0 #廣播 mcast eth0 225.0.0.1 694 1 0 #組播 ucast eth0 192.168.1.2 #單播，只有兩個節點的時候才用單播 #定義stonith主機 stonith_host * baytech 10.0.0.3 mylogin mysecretpassword stonith_host ken3 rps10 /dev/ttyS1 kathy 0 stonith_host kathy rps10 /dev/ttyS1 ken3 0 # Tell what machines are in the cluster #告訴集羣中有多少個節點，每個節點用node和主機名寫一行，主機名要和uname -n保持一致 node ken3 node kathy 通常只要定義心跳信息的發送方式、和集羣中的節點就行。 bcasteth0 nodetest1.local nodetest2.local
6.五、定義haresources資源配置文件：
[root@test2 ~]# vim /etc/ha.d/haresources #node110.0.0.170Filesystem::/dev/sda1::/data1::ext2#默認用做主節點的主機名，要跟uname -n同樣。VIP是多少。自動掛載哪一個設備，到哪一個目錄下，文件類型是什麼。資源類型的參數要用雙冒號隔開 #just.linux-ha.org135.9.216.110http#和上面同樣，這裏使用的資源是在/etc/rc.d/init.d/下面的，默認先到/etc/ha.d/resource.d/目錄下找資源，找不到在到/etc/rc.d/init.d/目錄找 master1.localIPaddr::192.168.10.2/24/eth0 mysqld master1.localIPaddr::192.168.10.2/24/eth0 drbddisk::data Filesystem::/dev/drbd1::/data::ext3mysqld#使用IPaddr腳原本配置VIP
6.六、拷貝master1.local的配置文件到master2.local上
[root@test1 ~]# scp -p ha.cf haresources authkeys master2.local:/etc/ha.d/
七、啓動heartbeat
[root@test1 ~]# service heartbeat start [root@test1 ~]# ssh master2.local 'service heartbeat start'#必定要在test1上經過ssh的方式啓動test2節點的heartbeat
7.一、查看heartbeat啓動日誌
[root@test1 ~]# tail -f /var/log/messages Feb 16 15:12:45 test-1 heartbeat: [16056]: info: Configuration validated. Starting heartbeat 3.0.4 Feb 16 15:12:45 test-1 heartbeat: [16057]: info: heartbeat: version 3.0.4 Feb 16 15:12:45 test-1 heartbeat: [16057]: info: Heartbeat generation: 1455603909 Feb 16 15:12:45 test-1 heartbeat: [16057]: info: glib: UDP Broadcast heartbeat started on port 694 (694) interface eth0 Feb 16 15:12:45 test-1 heartbeat: [16057]: info: glib: UDP Broadcast heartbeat closed on port 694 interface eth0 - Status: 1 Feb 16 15:12:45 test-1 heartbeat: [16057]: info: glib: ping heartbeat started. Feb 16 15:12:45 test-1 heartbeat: [16057]: info: G_main_add_TriggerHandler: Added signal manual handler Feb 16 15:12:45 test-1 heartbeat: [16057]: info: G_main_add_TriggerHandler: Added signal manual handler Feb 16 15:12:45 test-1 heartbeat: [16057]: info: G_main_add_SignalHandler: Added signal handler for signal 17 Feb 16 15:12:45 test-1 heartbeat: [16057]: info: Local status now set to: 'up' Feb 16 15:12:45 test-1 heartbeat: [16057]: info: Link 192.168.10.1:192.168.10.1 up. Feb 16 15:12:45 test-1 heartbeat: [16057]: info: Status update for node 192.168.10.1: status ping Feb 16 15:12:45 test-1 heartbeat: [16057]: info: Link test1.local:eth0 up. Feb 16 15:12:51 test-1 heartbeat: [16057]: info: Link test2.local:eth0 up. Feb 16 15:12:51 test-1 heartbeat: [16057]: info: Status update for node test2.local: status up Feb 16 15:12:51 test-1 harc(default)[16068]: info: Running /etc/ha.d//rc.d/status status Feb 16 15:12:52 test-1 heartbeat: [16057]: WARN: 1 lost packet(s) for [test2.local] [3:5] Feb 16 15:12:52 test-1 heartbeat: [16057]: info: No pkts missing from test2.local! Feb 16 15:12:52 test-1 heartbeat: [16057]: info: Comm_now_up(): updating status to active Feb 16 15:12:52 test-1 heartbeat: [16057]: info: Local status now set to: 'active' Feb 16 15:12:52 test-1 heartbeat: [16057]: info: Status update for node test2.local: status active Feb 16 15:12:52 test-1 harc(default)[16086]: info: Running /etc/ha.d//rc.d/status status Feb 16 15:13:02 test-1 heartbeat: [16057]: info: local resource transition completed. Feb 16 15:13:02 test-1 heartbeat: [16057]: info: Initial resource acquisition complete (T_RESOURCES(us)) Feb 16 15:13:02 test-1 heartbeat: [16057]: info: remote resource transition completed. Feb 16 15:13:02 test-1 /usr/lib/ocf/resource.d//heartbeat/IPaddr(IPaddr_192.168.10.2)[16138]: INFO: Resource is stopped Feb 16 15:13:02 test-1 heartbeat: [16102]: info: Local Resource acquisition completed. Feb 16 15:13:02 test-1 harc(default)[16219]: info: Running /etc/ha.d//rc.d/ip-request-resp ip-request-resp Feb 16 15:13:02 test-1 ip-request-resp(default)[16219]: received ip-request-resp IPaddr::192.168.10.2/24/eth0 OK yes Feb 16 15:13:02 test-1 ResourceManager(default)[16238]: info: Acquiring resource group: test1.local IPaddr::192.168.10.2/24/eth0 mysqld Feb 16 15:13:02 test-1 /usr/lib/ocf/resource.d//heartbeat/IPaddr(IPaddr_192.168.10.2)[16264]: INFO: Resource is stopped Feb 16 15:13:03 test-1 ResourceManager(default)[16238]: info: Running /etc/ha.d/resource.d/IPaddr 192.168.10.2/24/eth0 start Feb 16 15:13:03 test-1 IPaddr(IPaddr_192.168.10.2)[16386]: INFO: Adding inet address 192.168.10.2/24 with broadcast address 192.168.10.255 to device eth0 Feb 16 15:13:03 test-1 IPaddr(IPaddr_192.168.10.2)[16386]: INFO: Bringing device eth0 up Feb 16 15:13:03 test-1 IPaddr(IPaddr_192.168.10.2)[16386]: INFO: /usr/libexec/heartbeat/send_arp -i 200 -r 5 -p /var/run/resource-agents/send_arp-192.168.10.2 eth0 192.168.10.2 auto not_used not_used Feb 16 15:13:03 test-1 /usr/lib/ocf/resource.d//heartbeat/IPaddr(IPaddr_192.168.10.2)[16360]: INFO: Success Feb 16 15:13:03 test-1 ResourceManager(default)[16238]: info: Running /etc/init.d/mysqld start Feb 16 15:13:04 test-1 ntpd[1605]: Listen normally on 15 eth0 192.168.10.2 UDP 123
說明：
一、Link test1.local:eth0 up、Link test2.local:eth0 up#兩個節點鏈接成功併爲UP狀態。
二、Link 192.168.10.1:192.168.10.1 up#ping節點的IP也已經啓動
三、info: Running /etc/init.d/mysqld start#mysql啓動成功
四、Listen normally on 15 eth0 192.168.10.2 UDP 123#VIP啓動成功
7.二、查看heartbeat的VIP
[root@test1 ha.d]# ip add |grep "10.2" inet 192.168.10.55/24 brd 192.168.10.255 scope global eth0 inet 192.168.10.2/24 brd 192.168.10.255 scope global secondary eth0
[root@test-2 ha.d]# ip add |grep "10.2" inet 192.168.10.56/24 brd 192.168.10.255 scope global eth0
注：能夠看到如今VIP是在master1.local主機上。而master2.local上沒有VIP
八、測試效果
8.一、正常狀況下鏈接mysql
[root@test3 ha.d]# mysql -uroot -h'192.168.10.2' -p Enter password: Welcome to the MySQL monitor. Commands end with ; or \g. Your MySQL connection id is 2 Server version: 5.5.44 Source distribution Copyright (c) 2000, 2013, Oracle and/or its affiliates. All rights reserved. Oracle is a registered trademark of Oracle Corporation and/or its affiliates. Other names may be trademarks of their respective owners. Type 'help;' or '\h' for help. Type '\c' to clear the current input statement. mysql> show variables like 'server_id'; +---------------+-------+ | Variable_name | Value | +---------------+-------+ | server_id | 1 | +---------------+-------+ 1 row in set (0.00 sec) mysql>
8.二、關閉master1.local上的heartbeat
[root@test1 ha.d]# service heartbeat stop Stopping High-Availability services: Done. [root@test1 ha.d]# ip add |grep "192.168.10.2" inet 192.168.10.55/24 brd 192.168.10.255 scope global eth0 [root@test2 ha.d]# ip add |grep "192.168.10.2" inet 192.168.10.56/24 brd 192.168.10.255 scope global eth0 inet 192.168.10.2/24 brd 192.168.10.255 scope global secondary eth0
注：這個時候VIP已經漂移到了master2.local主機上，咱們在來看看鏈接mysql的server_id
[root@test2 ha.d]# mysql -uroot -h'192.168.10.2' -p Enter password: Welcome to the MySQL monitor. Commands end with ; or \g. Your MySQL connection id is 2 Server version: 5.5.44 Source distribution Copyright (c) 2000, 2013, Oracle and/or its affiliates. All rights reserved. Oracle is a registered trademark of Oracle Corporation and/or its affiliates. Other names may be trademarks of their respective owners. Type 'help;' or '\h' for help. Type '\c' to clear the current input statement. mysql> show variables like 'server_id'; +---------------+-------+ | Variable_name | Value | +---------------+-------+ | server_id | 2 | +---------------+-------+ 1 row in set (0.00 sec) mysql>
注：server_id已經從1變成了2，證實此時訪問的是master2.local主機上的mysql服務
測試完畢。下面配置drbd讓兩臺mysql服務器之間使用同一個文件系統，以實現mysql的寫高可用。
九、配置DRBD
DRBD：（Distributed Replicated Block Device）分佈式複製塊設備，是linux內核中的一個模塊。DRBD做爲磁盤鏡像來說，它必定是主從架構的，它決不容許兩個節點同時讀寫，僅容許一個節點能讀寫，從節點不能讀寫和掛載，
可是DRDB有雙主的概念，主、從的角色能夠切換。DRBD分別將位於兩臺主機上的硬盤或磁盤分區作成鏡像設備，當咱們客戶端的程序向主節點發起存儲請求的時候，這個數據會在底層以TCP/IP協議按位同布一份到備節點，
因此這能保證只要咱們在主節點上存的數據，備節點上在按位必定有如出一轍的一份數據。這是在兩臺主機上實現的，這意味着DRBD是工做在內核模塊當中。不像RAID1的鏡像是在同一臺主機上實現的。
DRBD雙主模型的實現：一個節點在數據訪問的時候，它必定會將數據、元數據載入內存的，並且它對於某個文件內核中加鎖的操做，另外一個節點的內核是看不到的，那若是它能將它本身施加的鎖通知給另外一個節點的內核就能夠了。
在這種狀況下，咱們就只能經過message layer（heartbeat、corosync均可）、pathmaker（把DRBD定義成資源），而後把這兩個主機上對應的鏡像格式化成集羣文件系統（GFS2/OCFS2）。
這就是基於結合分佈式文件鎖管理器（DLM Distributed Lock Manager）以及集羣文件系統所完成的雙主模型。DRBD集羣只容許有兩個節點，要麼雙主，要麼主從。
9.一、DRBD的三種工做模型
A、數據在本地DRBD存儲完成後嚮應用程序返回存儲成功的消息，異步模型。效率高，性能好。數據不安全
B、數據在本地DRBD存儲完成後而且經過TCP/IP把全部數據發送到從DRBD中，才向本地的應用程序返回存儲成功的消息，半同步模型。通常不用。
C、數據在本地DRBD存儲完成後，經過TCP/IP把全部數據發送到從DRBD中，從DRBD存儲完成後才嚮應用程序返回成功的消息，同步模型。效率低，性能若，可是數據安全可靠性大，用的最多。
9.二、DRBD的資源
一、資源名稱，能夠是任意的ascii碼不包含空格的字符
二、DRBD設備，在雙方節點上，此DRBD設備的設備文件，通常爲/dev/drbdN，其主設備號相同都是147，此設備號用來標識不通的設備
三、磁盤配置，在雙方節點上，各自提供的存儲設備，能夠是個分區，能夠是任何類型的塊設備，也能夠是lvm
四、網絡配置，雙方數據同步時，所使用的網絡屬性
9.三、安裝DRBD
drbd在2.6.33開始，才整合進內核的。
9.3.一、下載drbd
[root@test1 ~]# wget -O /usr/local/src http://oss.linbit.com/drbd/8.4/drbd-8.4.3.tar.gz
9.3.二、安裝drbd軟件
[root@test1 ~]# cd /usr/local/src [root@test1 src]# tar -zxvf drbd-8.4.3.tar.gz [root@test1 src]# cd /usr/local/src/drbd-8.4.3 [root@test1 drbd-8.4.3]# ./configure --prefix=/usr/local/drbd --with-km [root@test1 drbd-8.4.3]# make KDIR=/usr/src/kernels/2.6.32-573.18.1.el6.x86_64
[root@test1 drbd-8.4.3]# make install [root@test1 drbd-8.4.3]# mkdir -p /usr/local/drbd/var/run/drbd [root@test1 drbd-8.4.3]# cp /usr/local/drbd/etc/rc.d/init.d/drbd /etc/rc.d/init.d/
9.3.三、安裝drbd模塊
[root@test1 drbd-8.4.3]# cd drbd/ [root@test1 drbd]# make clean [root@test1 drbd]# make KDIR=/usr/src/kernels/2.6.32-573.18.1.el6.x86_64 [root@test1 drbd]# cp drbd.ko /lib/modules/`uname -r`/kernel/lib/ [root@test1 drbd]# modprobe drbd [root@test1 drbd]# lsmod | grep drbd
9.3.四、爲drbd建立新分區
[root@test1 drbd]# fdisk /dev/sdb
WARNING: DOS-compatible mode is deprecated. It's strongly recommended to
switch off the mode (command 'c') and change display units to
sectors (command 'u').
Command (m for help): n
Command action
e extended
p primary partition (1-4)
p
Partition number (1-4): 1
First cylinder (1-1305, default 1):
Using default value 1
Last cylinder, +cylinders or +size{K,M,G} (1-1305, default 1305): +9G
Command (m for help): w
The partition table has been altered!
Calling ioctl() to re-read partition table.
AWRNING: Re-reading the partition table failed with error 16: Device or resource busy.
The kernel still uses the old table.
The new table will be used at the next reboot.
Syncing disks.
[root@test1 drbd]# partprobe /dev/sdb
test2節點的drbd安裝和分區配置步驟略過，和test1上同樣安裝，test2節點的drbd配置文件保證和test1節點同樣，使用scp傳到test2節點便可
十、配置drbd
10.一、配置drbd的通用配置文件
[root@test1 drbd.d]# cd /usr/local/drbd/etc/drbd.d
[root@test1 drbd.d]# vim global_common.conf
global { #global是全局配置
usage-count no; #官方用來統計有多少個客戶使用drbd的
# minor-count dialog-refresh disable-ip-verification
}
common { #通用配置，用來配置全部資源那些相同屬性的。爲drbd提供默認屬性的
protocol C; #默認使用協議C，即同步模型。
handlers { #處理器段，用來配置drbd的故障切換操做
# These are EXAMPLE handlers only.
# They may have severe implications,
# like hard resetting the node under certain circumstances.
# Be careful when chosing your poison.
pri-on-incon-degr "/usr/lib/drbd/notify-pri-on-incon-degr.sh; /usr/lib/drbd/notify-emergency-reboot.sh; echo b > /proc/sysrq-trigger ; reboot -f";#
pri-lost-after-sb "/usr/lib/drbd/notify-pri-lost-after-sb.sh; /usr/lib/drbd/notify-emergency-reboot.sh; echo b > /proc/sysrq-trigger ; reboot -f"; #腦裂以後的操做
local-io-error "/usr/lib/drbd/notify-io-error.sh; /usr/lib/drbd/notify-emergency-shutdown.sh; echo o > /proc/sysrq-trigger ; halt -f"; #本地i/o錯誤以後的操做
# fence-peer "/usr/lib/drbd/crm-fence-peer.sh";
# split-brain "/usr/lib/drbd/notify-split-brain.sh root";
# out-of-sync "/usr/lib/drbd/notify-out-of-sync.sh root";
# before-resync-target "/usr/lib/drbd/snapshot-resync-target-lvm.sh -p 15 -- -c 16k";
# after-resync-target /usr/lib/drbd/unsnapshot-resync-target-lvm.sh;
}
startup {
# wfc-timeout degr-wfc-timeout outdated-wfc-timeout wait-after-sb #設備啓動時，兩個節點要同步，設置節點的等待時間，超時時間等
}
options {
# cpu-mask on-no-data-accessible
}
disk {
on-io-error detach; #一旦發生i/o錯誤，就把磁盤卸載。不繼續進行同步
# size max-bio-bvecs on-io-error fencing disk-barrier disk-flushes
# disk-drain md-flushes resync-rate resync-after al-extents
# c-plan-ahead c-delay-target c-fill-target c-max-rate
# c-min-rate disk-timeout
}
net { #設置網絡的buffers/cache大小，初始化時間等
# protocol timeout max-epoch-size max-buffers unplug-watermark
# connect-int ping-int sndbuf-size rcvbuf-size ko-count
# allow-two-primaries cram-hmac-alg shared-secret after-sb-0pri
# after-sb-1pri after-sb-2pri always-asbp rr-conflict
# ping-timeout data-integrity-alg tcp-cork on-congestion
# congestion-fill congestion-extents csums-alg verify-alg
# use-rle
cram-hmac-alg "sha1"; #數據加密使用的算法
shared-secret "mydrbd1fa2jg8"; #驗證密碼
}
syncer {
rate 200M; #定義數據傳輸速率
}
}
10.二、配置資源文件，資源配置文件的名字要和資源文件中的同樣
[root@test1 drbd.d]# vim mydrbd.res
resource mydrbd { #資源名稱，能夠是任意的ascii碼不包含空格的字符
on test1.local { #節點1，每一個節點必需要能使用名稱的方式解析對方節點
device /dev/drbd0; #drbd設備的文件名叫什麼
disk /dev/sdb1; #分區設備是哪一個
address 192.168.10.55:7789;#節點ip和監聽的端口
meta-disk internal; #drbd的meta（原數據）放在什麼地方，internal是放在設備內部
}
on test2.local {
device /dev/drbd0;
disk /dev/sdb1;
address 192.168.10.56:7789;
meta-disk internal;
}
}
10.三、兩個節點的配置文件同樣，使用工具把配置文件傳到另外一個節點
[root@test1 drbd.d]# scp -r /usr/local/drbd/etc/drbd.* test2.local:/usr/local/drbd/etc/
10.四、在每一個節點上初始化已定義的資源
[root@test1 drbd.d]# drbdadm create-md mydrbd --== Thank you for participating in the global usage survey ==-- The server's response is: Writing meta data... initializing activity log NOT initialized bitmap New drbd meta data block successfully created. [root@test1 drbd.d]# [root@test2 drbd.d]# drbdadm create-md mydrbd --== Thank you for participating in the global usage survey ==-- The server's response is: Writing meta data... initializing activity log NOT initialized bitmap New drbd meta data block successfully created. [root@test2 drbd.d]#
10.五、分別啓動兩個節點的drbd服務
[root@test1 drbd.d]# service drbd start [root@test2 drbd.d]# service drbd start
十一、測試drbd的同步
11.一、查看drbd的啓動狀態
[root@test1 drbd.d]# cat /proc/drbd version: 8.4.3 (api:1/proto:86-101) GIT-hash: 89a294209144b68adb3ee85a73221f964d3ee515 build by root@test1.local, 2016-02-23 10:23:03 0: cs:Connected ro:Secondary/Secondary ds:Inconsistent/Inconsistent C r----- #兩個節點都是從，未來能夠把一個提高爲主。Inconsistent處於沒有同步的狀態 ns:0 nr:0 dw:0 dr:0 al:0 bm:0 lo:0 pe:0 ua:0 ap:0 ep:1 wo:b oos:0
11.二、提高一個節點爲主，並覆蓋從節點的drbd分區數據。在要提高爲主的節點上執行
[root@test1 drbd.d]# drbdadm -- --overwrite-data-of-peer primary mydrbd
11.三、查看主節點同步狀態
[root@test1 drbd.d]# watch -n 1 cat /proc/drbd Every 1.0s: cat /proc/drbd Tue Feb 23 17:10:55 2016 version: 8.4.3 (api:1/proto:86-101) GIT-hash: 89a294209144b68adb3ee85a73221f964d3ee515 build by root@test1.local, 2016-02-23 10:23:03 0: cs:SyncSource ro:Primary/Secondary ds:UpToDate/Inconsistent C r---n- ns:619656 nr:0 dw:0 dr:627840 al:0 bm:37 lo:1 pe:8 ua:64 ap:0 ep:1 wo:b oos:369144 [=============>.......] sync'ed: 10.3% (369144/987896)K finish: 0:00:12 speed: 25,632 (25,464) K/sec
11.四、查看從節點的狀態
[root@test2 drbd]# cat /proc/drbd version: 8.4.3 (api:1/proto:86-101) GIT-hash: 89a294209144b68adb3ee85a73221f964d3ee515 build by root@test2.local, 2016-02-22 16:05:34 0: cs:Connected ro:Secondary/Primary ds:UpToDate/UpToDate C r----- ns:4 nr:9728024 dw:9728028 dr:1025 al:1 bm:577 lo:0 pe:0 ua:0 ap:0 ep:1 wo:f oos:0
11.五、在主節點格式化分區並掛在寫入數據測試
[root@test1 drbd]# mke2fs -j /dev/drbd0 [root@test1 drbd]# mkdir /mydrbd [root@test1 drbd]# mount /dev/drdb0 /mydrbd [root@test1 drbd]# cd /mydrbd [root@test1 mydrbd]# touch drbd_test_file [root@test1 mydrbd]# ls /mydrbd/ drbd_test_file lost+found
11.六、把主節點降級爲從，把從節點提高爲主。查看數據是否同步
11.一、主節點操做
11.1.一、卸載分區，注意卸載的時候要退出掛在目錄，不然會顯示設備忙，不能卸載
[root@test1 mydrbd]# cd ~ [root@test1 ~]# umount /mydrbd [root@test1 ~]# drbdadm secondary mydrbd
11.1.二、查看如今的drbd狀態
[root@test1 ~]# cat /proc/drbd version: 8.4.3 (api:1/proto:86-101) GIT-hash: 89a294209144b68adb3ee85a73221f964d3ee515 build by root@test2.local, 2016-02-22 16:05:34 0: cs:Connected ro:Secondary/Secondary ds:UpToDate/UpToDate C r----- ns:4 nr:9728024 dw:9728028 dr:1025 al:1 bm:577 lo:0 pe:0 ua:0 ap:0 ep:1 wo:f oos:0
注：能夠看到，如今drbd的兩個節點的狀態都是secondary的，下面把從節點提高爲主
11.二、從節點操做
11.2.一、提高操做
[root@test2 ~]# drdbadm primary mydrbd
11.2.二、掛在drbd分區
[root@test2 ~]# mkdir /mydrbd [root@test2 ~]# mount /dev/drbd0 /mydrbd/
11.2.三、查看是否有數據
[root@test2 ~]# ls /myddrbd/ drbd_test_file lost+found
注：能夠看到從節點切換成主後，已經同步了數據。drbd搭建完成。下面結合corosync+mysql配置雙主高可用。
十二、結合corosync+drbd+mysql實現數據庫雙主高可用
將drbd配置爲corosync雙節點高可用集羣中的資源，可以實現主從角色的自動切換，注意，要把某一個服務配置爲高可用集羣的資源，必定不能讓這個服務開機自動啓動。
12.一、關閉兩臺節點的drbd開機自啓動
12.1.一、主節點操做
[root@test1 drbd.d]# chkconfig drbd off [root@test1 drbd.d]# chkconfig --list |grep drbd drbd 0:off 1:off 2:off 3:off 4:off 5:off 6:off
12.1.二、從節點操做
[root@test2 drbd.d]# chkconfig drbd off [root@test2 drbd.d]# chkconfig --list |grep drbd drbd 0:off 1:off 2:off 3:off 4:off 5:off 6:off
12.二、卸載drbd的文件系統並把主節點降級爲從節點
12.2.一、從節點操做，注意，這裏的從節點剛纔提高爲主了。如今把他降級
[root@test2 drbd]# umount /mydata/ [root@test2 drbd]# drbdadm secondary mydrbd [root@test2 drbd]# cat /proc/drbd version: 8.4.3 (api:1/proto:86-101) GIT-hash: 89a294209144b68adb3ee85a73221f964d3ee515 build by root@test2.local, 2016-02-22 16:05:34 0: cs:Connected ro:Secondary/Secondary ds:UpToDate/UpToDate C r----- ns:8 nr:9728024 dw:9728032 dr:1073 al:1 bm:577 lo:0 pe:0 ua:0 ap:0 ep:1 wo:f oos:0
注：確保兩個節點都是secondary
12.三、中止兩個節點的drbd服務
12.3.一、從節點操做
[root@test2 drbd]# service drbd stop Stopping all DRBD resources: . [root@test2 drbd]#
12.3.二、主節點操做
[root@test1 drbd.d]# service drbd stop Stopping all DRBD resources: . [root@test1 drbd.d]#
12.四、安裝corosync並建立日誌目錄
12.4.一、主節點操做
[root@test1 drbd.d]# wget -P /etc/yum.repos.d http://download.opensuse.org/repositories/network:/ha-clustering:/Stable/CentOS_CentOS-6/network:ha-clustering:Stable.repo [root@test1 drbd.d]# yum install corosync pacemaker crmsh [root@test1 drbd.d]# mkdir /var/log/cluster
12.4.二、從節點操做
[root@test1 drbd.d]# wget -P /etc/yum.repos.d http://download.opensuse.org/repositories/network:/ha-clustering:/Stable/CentOS_CentOS-6/network:ha-clustering:Stable.repo [root@test2 drbd.d]# mkdir /var/log/cluster [root@test2 drbd.d]# yum install corosync pacemaker crmsh
12.五、corosync配置文件
12.5.一、主節點操做
[root@test1 drbd.d]# cd /etc/corosync/ [root@test1 corosync]# cp corosync.conf.example corosync.conf
12.六、配置主節點配置文件，生成corosync祕鑰文件並複製給從節點（包括主配置文件）
12.6.一、主節點操做
[root@test1 corosync]# vim corosync.conf
# Please read the corosync.conf.5 manual page
compatibility: whitetank
totem {
version: 2# secauth: Enable mutual node authentication. If you choose to
# enable this ("on"), then do remember to create a shared
# secret with "corosync-keygen".
secauth: on
threads: 2
# interface: define at least one interface to communicate
# over. If you define more than one interface stanza, you must
# also set rrp_mode.
interface {
# Rings must be consecutively numbered, starting at 0.
ringnumber: 0
# This is normally the *network* address of the
# interface to bind to. This ensures that you can use
# identical instances of this configuration file
# across all your cluster nodes, without having to
# modify this option.
bindnetaddr: 192.168.10.0
# However, if you have multiple physical network
# interfaces configured for the same subnet, then the
# network address alone is not sufficient to identify
# the interface Corosync should bind to. In that case,
# configure the *host* address of the interface
# instead:
bindnetaddr: 192.168.10.0
# When selecting a multicast address, consider RFC
# 2365 (which, among other things, specifies that
# 239.255.x.x addresses are left to the discretion of
# the network administrator). Do not reuse multicast
# addresses across multiple Corosync clusters sharing
# the same network.
mcastaddr: 239.212.16.19
# Corosync uses the port you specify here for UDP
# messaging, and also the immediately preceding
# port. Thus if you set this to 5405, Corosync sends
# messages over UDP ports 5405 and 5404.
mcastport: 5405
# Time-to-live for cluster communication packets. The
# number of hops (routers) that this ring will allow
# itself to pass. Note that multicast routing must be
# specifically enabled on most network routers.
ttl: 1 #每個數據報文不容許通過路由
}
}
logging {
# Log the source file and line where messages are being
# generated. When in doubt, leave off. Potentially useful for
# debugging.
fileline: off
# Log to standard error. When in doubt, set to no. Useful when
# running in the foreground (when invoking "corosync -f")
to_stderr: no
# Log to a log file. When set to "no", the "logfile" option
# must not be set.
to_logfile: yes
logfile: /var/log/cluster/corosync.log
# Log to the system log daemon. When in doubt, set to yes.
to_syslog: no
# Log debug messages (very verbose). When in doubt, leave off.
debug: off
# Log messages with time stamps. When in doubt, set to on
# (unless you are only logging to syslog, where double
# timestamps can be annoying).
timestamp: on
logger_subsys {
subsys: AMF
debug: off
}
}
service {
ver:0
name:pacemaker
}
aisexec {
user:root
group:root
}
[root@test1 corosync]# corosync-keygen
[root@test1 corosync]# scp -p authkey corosync.conf test2.local:/etc/corosync/
12.七、啓動corosync
12.7.一、主節點操做（注意：兩個corosync的啓動操做都要在主節點上進行）
[root@test1 corosync]# service corosync start Starting Corosync Cluster Engine (corosync): [ OK ] [root@test1 corosync]# ssh test2.local 'service corosync start' root@test2.local's password: Starting Corosync Cluster Engine (corosync): [ OK ]
12.八、查看集羣狀態
12.8.一、問題：安裝以後系統沒有crm命令，不能使用crm的交互式模式
[root@test1 corosync]# crm status -bash: crm: command not found
解決辦法：安裝ha-cluster的yum源，在安裝crmsh軟件包
[root@test1 corosync]# wget -P /etc/yum.repos.d/ http://download.opensuse.org/repositories/network:/ha-clustering:/Stable/CentOS_CentOS-6/network:ha-clustering:Stable.repo
參考文檔：http://www.dwhd.org/20150530_014731.html 中的第二，安裝crmsh步驟
安裝完成後就可使用crm命令行模式了
12.8.二、查看集羣節點狀態
[root@test1 corosync]# crm status Last updated: Wed Feb 24 13:47:17 2016 Last change: Wed Feb 24 11:26:06 2016 Stack: classic openais (with plugin) Current DC: test2.local - partition with quorum Version: 1.1.11-97629de 2 Nodes configured, 2 expected votes 0 Resources configured Online: [ test1.local test2.local ] Full list of resources:
注：顯示有2個nodes已配置並處於在線狀態
12.8.三、查看pacemaker是否啓動
[root@test1 corosync]# grep pcmk_startup /var/log/cluster/corosync.log Feb 24 11:05:15 corosync [pcmk ] info: pcmk_startup: CRM: Initialized Feb 24 11:05:15 corosync [pcmk ] Logging: Initialized pcmk_startup
12.8.四、檢查集羣引擎是否啓動
[root@test1 corosync]# grep -e "Corosync Cluster Engine" -e "configuration file" /var/log/cluster/corosync.log
Feb 24 11:04:16 corosync [MAIN ] Corosync Cluster Engine ('1.4.7'): started and ready to provide service.
Feb 24 11:04:16 corosync [MAIN ] Successfully read main configuration file '/etc/corosync/corosync.conf'.
注：通過上面的步驟，能夠確認corosync的服務已經啓動並無問題了
12.九、配置corosync的屬性
12.9.一、禁用STONITH設備以保證verify不會出錯
[root@test1 corosync]# crm configure crm(live)configure# property stonith-enabled=false crm(live)configure# verify crm(live)configure# commit
12.9.二、配置當不具有法定票數的時候不能關閉服務
crm(live)configure# property no-quorum-policy=ignore crm(live)configure# verify crm(live)configure# commit
12.9.三、配置資源默認粘性值
crm(live)configure# rsc_defaults resource-stickiness=100 crm(live)configure# verify crm(live)configure# commit
12.9.三、查看當前的配置
crm(live)configure# show node test1.local node test2.local property cib-bootstrap-options: \ dc-version=1.1.11-97629de \ cluster-infrastructure="classic openais (with plugin)" \ expected-quorum-votes=2 \ stonith-enabled=false \ no-quorum-policy=ignore rsc_defaults rsc-options: \ resource-stickiness=100
12.十、配置集羣資源
12.10.一、定義drbd的資源
crm(live)configure# primitive mysqldrbd ocf:linbit:drbd params drbd_resource=mydrbd op start timeout=240 op stop timeout=100 op monitor role=Master interval=10 timeout=20 op monitor role=Slave interval=20 timeout=20
注：ocf:資源代理，這裏用linbit 代理drbd。 drbd_resource=mydrbd：drbd的資源名字。start timeout：啓動超時時間。stop timeout：中止超時時間。monitor role=Master：定義主節點的監控時間，interval：監控間隔，timeout：超時時間。monitor role=Slave：定義從節點的監控時間，interval：監控間隔，timeout：超時時間
crm(live)configure# show #查看配置 node test1.local node test2.local primitive mydrbd ocf:linbit:drbd \ params drbd_resource=mydrbd \ op start timeout=240 interval=0 \ op stop timeout=100 interval=0 \ op monitor role=Master interval=10s timeout=20s \ op monitor role=Slave interval=20s timeout=20s property cib-bootstrap-options: \ dc-version=1.1.11-97629de \ cluster-infrastructure="classic openais (with plugin)" \ expected-quorum-votes=2 \ stonith-enabled=false \ no-quorum-policy=ignore rsc_defaults rsc-options: \ resource-stickiness=100 crm(live)configure# verify #驗證配置
12.10.二、定義drbd的主從資源
crm(live)configure# ms ms_mysqldrbd mysqldrbd meta master-max=1 master-node-max=1 clone-max=2 clone-node-max=1 notify=true 注：ms ms_mydrbd mydrbd：把mydrbd作成主從，這個資源名稱叫ms_mydrbd。meta：定義元數據屬性。master-max=1：最多有1個主資源，master-node-max=1：最多有1個主節點，clone-max=2：最多有2個克隆，clone-node-max=1：每一個節點上，能夠啓動幾個克隆資源。
crm(live)configure# verify crm(live)configure# commit crm(live)configure# show node test1.local node test2.local primitive mydrbd ocf:linbit:drbd \ params drbd_resource=mydrbd \ op start timeout=240 interval=0 \ op stop timeout=100 interval=0 \ op monitor role=Master interval=10s timeout=20s \ op monitor role=Slave interval=20s timeout=20s ms ms_mydrbd mydrbd \ meta master-max=1 master-node-max=1 clone-max=2 clone-node-max=1 property cib-bootstrap-options: \ dc-version=1.1.11-97629de \ cluster-infrastructure="classic openais (with plugin)" \ expected-quorum-votes=2 \ stonith-enabled=false \ no-quorum-policy=ignore rsc_defaults rsc-options: \ resource-stickiness=100
注：能夠看到如今配置有一個primitive mydrbd的資源，一個ms的主從類型。
12.10.三、查看集羣節點狀態
crm(live)configure# cd crm(live)# status Last updated: Thu Feb 25 14:45:52 2016 Last change: Thu Feb 25 14:44:44 2016 Stack: classic openais (with plugin) Current DC: test1.local - partition with quorum Version: 1.1.11-97629de 2 Nodes configured, 2 expected votes 2 Resources configured Online: [ test1.local test2.local ] Full list of resources: Master/Slave Set: ms_mysqldrbd [mysqldrbd] Masters: [ test1.local ] Slaves: [ test2.local ]
注：正常狀態，已經能夠看到主從狀態，test1是master。
錯誤狀態：
Master/Slave Set: ms_mydrbd [mydrbd] mydrbd(ocf::linbit:drbd):FAILED test2.local (unmanaged) mydrbd(ocf::linbit:drbd):FAILED test1.local (unmanaged) Failed actions: mydrbd_stop_0 on test2.local 'not configured' (6): call=87, status=complete, last-rc-change='Thu Feb 25 14:17:34 2016', queued=0ms, exec=34ms mydrbd_stop_0 on test2.local 'not configured' (6): call=87, status=complete, last-rc-change='Thu Feb 25 14:17:34 2016', queued=0ms, exec=34ms mydrbd_stop_0 on test1.local 'not configured' (6): call=72, status=complete, last-rc-change='Thu Feb 25 14:17:34 2016', queued=0ms, exec=34ms mydrbd_stop_0 on test1.local 'not configured' (6): call=72, status=complete, last-rc-change='Thu Feb 25 14:17:34 2016', queued=0ms, exec=34ms 解決辦法：定義資源的時候要注意，drbd_resource=mydrbd的名字是drbd資源的名字且主從資源的名稱不能和drbd資源的名稱同樣，還有各類超時設置中不要加s。測試了好久，才找到這個問題。。。。若有不一樣見解，請各位大神賜教，謝謝。
12.10.四、驗證主從的切換
[root@test1 ~]# crm node standby test1.local #將主節點下線 [root@test1 ~]# crm status 注：查看狀態，顯示主節點已經不在線，而test2成爲了master Last updated: Thu Feb 25 14:51:58 2016 Last change: Thu Feb 25 14:51:44 2016 Stack: classic openais (with plugin) Current DC: test1.local - partition with quorum Version: 1.1.11-97629de 2 Nodes configured, 2 expected votes 2 Resources configured Node test1.local: standby Online: [ test2.local ] Full list of resources: Master/Slave Set: ms_mysqldrbd [mysqldrbd] Masters: [ test2.local ] Stopped: [ test1.local ] [root@test1 ~]# crm node online test1.local #從新將test1上線 [root@test1 ~]# crm status #查看狀態，顯示test依舊爲master，而test1成爲了slave Last updated: Thu Feb 25 14:52:55 2016 Last change: Thu Feb 25 14:52:39 2016 Stack: classic openais (with plugin) Current DC: test1.local - partition with quorum Version: 1.1.11-97629de 2 Nodes configured, 2 expected votes 2 Resources configured Online: [ test1.local test2.local ] Full list of resources: Master/Slave Set: ms_mysqldrbd [mysqldrbd] Masters: [ test2.local ] Slaves: [ test1.local ]
12.10.五、定義文件系統資源
[root@test1 ~]# crm crm(live)# configure crm(live)configure# primitive mystore ocf:heartbeat:Filesystem params device=/dev/drbd0 directory=/mydrbd fstype=ext3 op start timeout=60 op stop timeout=60 注：這裏的語句表示，primitive mystore：定義一個資源mystore，使用heartbeat的文件系統，params參數定義：drbd的設備名，掛在目錄，文件系統類型，和啓動中止超時時間 crm(live)configure# verify
12.10.六、定義排列約束以確保Filesystem和主節點在一塊兒。
crm(live)configure# colocation mystore_withms_mysqldrbd inf: mystore ms_mysqldrbd:Master crm(live)configure# verify
12.10.七、定義Order約束，以確保主從資源必需要先成爲主節點之後才能掛在文件系統
crm(live)configure# order mystore_after_ms_mysqldrbd mandatory: ms_mysqldrbd:promote mystore:start 注：這裏的語句表示，mystore_after_ms_mysqldrbd mandatory:，mystore在ms_mysqldrbd以後啓動，mandatory(表明強制的)，先啓動ms_mysqldrbd，promote(角色切換成功後)，在啓動mystore:start crm(live)configure# verify crm(live)configure# commit crm(live)configure# cd crm(live)# status #查看狀態，能夠看到文件系統已經自動掛在到主節點test1.local上了。 Last updated: Thu Feb 25 15:29:39 2016 Last change: Thu Feb 25 15:29:36 2016 Stack: classic openais (with plugin) Current DC: test1.local - partition with quorum Version: 1.1.11-97629de 2 Nodes configured, 2 expected votes 3 Resources configured Online: [ test1.local test2.local ] Full list of resources: Master/Slave Set: ms_mysqldrbd [mysqldrbd] Masters: [ test1.local ] Slaves: [ test2.local ] mystore(ocf::heartbeat:Filesystem):Started test1.local [root@test1 ~]# ls /mydrbd/#已經有以前建立的文件了。 inittab lost+found
12.10.八、切換主從節點，驗證文件系統是否會自動掛載
[root@test1 ~]# crm node standby test1.local [root@test1 ~]# crm node online test1.local [root@test1 ~]# crm status Last updated: Thu Feb 25 15:32:39 2016 Last change: Thu Feb 25 15:32:36 2016 Stack: classic openais (with plugin) Current DC: test1.local - partition with quorum Version: 1.1.11-97629de 2 Nodes configured, 2 expected votes 3 Resources configured Online: [ test1.local test2.local ] Full list of resources: Master/Slave Set: ms_mysqldrbd [mysqldrbd] Masters: [ test2.local ] Slaves: [ test1.local ] mystore(ocf::heartbeat:Filesystem):Started test2.local [root@test2 ~]# ls /mydrbd/ inittab lost+found
12.9.十二、配置mysql
12.9.12.一、關閉mysqld的開機自啓動
[root@test1 ~]# chkconfig mysqld off [root@test2 ~]# chkconfig mysqld off #必定要記住，只要是高可用集羣中的資源的服務，必定不能開機自啓動
12.9.12.二、配置主節點1的mysql服務（mysql安裝就不寫了，直接進入mysql的配置）
[root@test1 mysql]# mkdir /mydrbd/data [root@test1 mysql]# chown -R mysql.mysql /mydrbd/data/ [root@test1 mysql]# ./scripts/mysql_install_db --user=mysql --datadir=/mydrbd/data/ --basedir=/usr/local/mysql/ Installing MySQL system tables... 160225 16:07:12 [Note] /usr/local/mysql//bin/mysqld (mysqld 5.5.44) starting as process 18694 ... OK Filling help tables... 160225 16:07:18 [Note] /usr/local/mysql//bin/mysqld (mysqld 5.5.44) starting as process 18701 ... OK To start mysqld at boot time you have to copy support-files/mysql.server to the right place for your system PLEASE REMEMBER TO SET A PASSWORD FOR THE MySQL root USER ! To do so, start the server, then issue the following commands: /usr/local/mysql//bin/mysqladmin -u root password 'new-password' /usr/local/mysql//bin/mysqladmin -u root -h test1.local password 'new-password' Alternatively you can run: /usr/local/mysql//bin/mysql_secure_installation which will also give you the option of removing the test databases and anonymous user created by default. This is strongly recommended for production servers. See the manual for more instructions. You can start the MySQL daemon with: cd /usr/local/mysql/ ; /usr/local/mysql//bin/mysqld_safe & You can test the MySQL daemon with mysql-test-run.pl cd /usr/local/mysql//mysql-test ; perl mysql-test-run.pl Please report any problems at [root@test1 mysql]# service mysqld start Starting MySQL..... [ OK ] [root@test1 mysql]# mysql -uroot Welcome to the MySQL monitor. Commands end with ; or \g. Your MySQL connection id is 2 Server version: 5.5.44 Source distribution Copyright (c) 2000, 2013, Oracle and/or its affiliates. All rights reserved. Oracle is a registered trademark of Oracle Corporation and/or its affiliates. Other names may be trademarks of their respective owners. Type 'help;' or '\h' for help. Type '\c' to clear the current input statement. mysql> show databases; +--------------------+ | Database | +--------------------+ | information_schema | | mysql | | performance_schema | | test | +--------------------+ 4 rows in set (0.06 sec) mysql> create database drbd_mysql; Query OK, 1 row affected (0.00 sec) mysql> show databases; +--------------------+ | Database | +--------------------+ | information_schema | | drbd_mysql | | mysql | | performance_schema | | test | +--------------------+ 5 rows in set (0.00 sec) mysql>
12.9.12.三、配置從節點的mysql
注：由於剛纔已經在test1.local上的共享存儲上初始化了mysql的data目錄，在test2.local上就不用重複初始化了。
[root@test1 mysql]# crm status Last updated: Thu Feb 25 16:14:14 2016 Last change: Thu Feb 25 15:35:16 2016 Stack: classic openais (with plugin) Current DC: test1.local - partition with quorum Version: 1.1.11-97629de 2 Nodes configured, 2 expected votes 3 Resources configured Online: [ test1.local test2.local ] Full list of resources: Master/Slave Set: ms_mysqldrbd [mysqldrbd] Masters: [ test1.local ] Slaves: [ test2.local ] mystore(ocf::heartbeat:Filesystem):Started test1.local
一、先讓test2.local成爲master才能繼續操做
[root@test1 mysql]# crm node standby test1.local [root@test1 mysql]# crm node online test1.local [root@test1 mysql]# crm status Last updated: Thu Feb 25 16:14:46 2016 Last change: Thu Feb 25 16:14:30 2016 Stack: classic openais (with plugin) Current DC: test1.local - partition with quorum Version: 1.1.11-97629de 2 Nodes configured, 2 expected votes 3 Resources configured Node test1.local: standby Online: [ test2.local ] Full list of resources: Master/Slave Set: ms_mysqldrbd [mysqldrbd] Masters: [ test2.local ] Stopped: [ test1.local ] mystore(ocf::heartbeat:Filesystem):Started test2.local #要確保讓test2.local成爲master節點
二、test2.local的mysql配置
[root@test2 ~]# vim /etc/my.cnf 添加: datadir = /mydrbd/data [root@test2 ~]# service mysqld start Starting MySQL. [ OK ] [root@test2 ~]# mysql -uroot Welcome to the MySQL monitor. Commands end with ; or \g. Your MySQL connection id is 1 Server version: 5.5.44 Source distribution Copyright (c) 2000, 2013, Oracle and/or its affiliates. All rights reserved. Oracle is a registered trademark of Oracle Corporation and/or its affiliates. Other names may be trademarks of their respective owners. Type 'help;' or '\h' for help. Type '\c' to clear the current input statement. mysql> show databases; +--------------------+ | Database | +--------------------+ | information_schema | | drbd_mysql | | mysql | | performance_schema | | test | +--------------------+ 5 rows in set (0.10 sec) mysql>
12.9.1三、定義mysql資源
一、中止mysql
[root@test2 ~]# service mysqld stop Shutting down MySQL. [ OK ]
二、定義mysql資源
[root@test1 mysql]# crm configure crm(live)configure# primitive mysqld lsb:mysqld crm(live)configure# verify
三、定義mysql和主節點約束
crm(live)configure# colocation mysqld_with_mystore inf: mysqld mystore #注：由於mystore必定和主節點在一塊兒，那麼咱們就定義mysql和mystore的約束。 crm(live)configure# verify
四、定義mysql和mystore啓動次序約束
crm(live)configure# order mysqld_after_mystore mandatory: mystore mysqld #必定要弄清楚啓動的前後次序，mysql是在mystore以後啓動的。 crm(live)configure# verify crm(live)configure# commit crm(live)configure# cd crm(live)# status Last updated: Thu Feb 25 16:44:29 2016 Last change: Thu Feb 25 16:42:16 2016 Stack: classic openais (with plugin) Current DC: test1.local - partition with quorum Version: 1.1.11-97629de 2 Nodes configured, 2 expected votes 4 Resources configured Online: [ test1.local test2.local ] Full list of resources: Master/Slave Set: ms_mysqldrbd [mysqldrbd] Masters: [ test2.local ] Slaves: [ test1.local ] mystore(ocf::heartbeat:Filesystem):Started test2.local mysqld(lsb:mysqld):Started test2.local 注：如今主節點在test2.local上。
五、驗證test2.local上的mysql登陸是否正常和角色切換後是否正常
[root@test2 ~]# mysql -uroot Welcome to the MySQL monitor. Commands end with ; or \g. Your MySQL connection id is 2 Server version: 5.5.44 Source distribution Copyright (c) 2000, 2013, Oracle and/or its affiliates. All rights reserved. Oracle is a registered trademark of Oracle Corporation and/or its affiliates. Other names may be trademarks of their respective owners. Type 'help;' or '\h' for help. Type '\c' to clear the current input statement. mysql> show databases; #test2.local上已經啓動好了mysql並自動掛在了drbd資源 +--------------------+ | Database | +--------------------+ | information_schema | | drbd_mysql | | mysql | | performance_schema | | test | +--------------------+ 5 rows in set (0.07 sec) mysql> create database mydb; #建立一個數據庫，而後切換到test1.local節點上，看是否正常 Query OK, 1 row affected (0.00 sec) mysql> show databases; +--------------------+ | Database | +--------------------+ | information_schema | | drbd_mysql | | mydb | | mysql | | performance_schema | | test | +--------------------+ 6 rows in set (0.00 sec) mysql> exit [root@test1 mysql]# crm node standby test2.local #將test2.local主節點standby，讓test1.local自動成爲master [root@test1 mysql]# crm node online test2.local [root@test1 mysql]# crm status Last updated: Thu Feb 25 16:53:24 2016 Last change: Thu Feb 25 16:53:19 2016 Stack: classic openais (with plugin) Current DC: test1.local - partition with quorum Version: 1.1.11-97629de 2 Nodes configured, 2 expected votes 4 Resources configured Online: [ test1.local test2.local ] Full list of resources: Master/Slave Set: ms_mysqldrbd [mysqldrbd] Masters: [ test1.local ] Slaves: [ test2.local ] mystore(ocf::heartbeat:Filesystem):Started test1.local #test1.local已經成爲master mysqld(lsb:mysqld):Started test1.local [root@test1 mysql]# mysql -uroot #在test1.local上登陸mysql Welcome to the MySQL monitor. Commands end with ; or \g. Your MySQL connection id is 1 Server version: 5.5.44 Source distribution Copyright (c) 2000, 2013, Oracle and/or its affiliates. All rights reserved. Oracle is a registered trademark of Oracle Corporation and/or its affiliates. Other names may be trademarks of their respective owners. Type 'help;' or '\h' for help. Type '\c' to clear the current input statement. mysql> show databases; +--------------------+ | Database | +--------------------+ | information_schema | | drbd_mysql | | mydb | #已經有剛纔在test2.local上建立的mydb數據庫。目前，一切正常 | mysql | | performance_schema | | test | +--------------------+ 6 rows in set (0.15 sec) mysql>
12.9.1四、定義VIP資源及約束關係
crm(live)configure# primitive myip ocf:heartbeat:IPaddr params ip=192.168.10.3 nic=eth0 cidr_netmask=24 #這裏出了一個錯誤，浪費半天時間，是由於子網掩碼寫的255.255.255.0，應該寫成24，忘記命令的時候，要多使用table和help。 crm(live)configure# verify crm(live)configure# colocation myip_with_ms_mysqldrbd inf: ms_mysqldrbd:Master myip #定義vip和ms_mysqldrbd的約束關係 crm(live)configure# verify crm(live)configure# commit crm(live)configure# cd crm(live)# status#查看狀態 Last updated: Fri Feb 26 10:05:16 2016 Last change: Fri Feb 26 10:05:12 2016 Stack: classic openais (with plugin) Current DC: test1.local - partition with quorum Version: 1.1.11-97629de 2 Nodes configured, 2 expected votes 5 Resources configured Online: [ test1.local test2.local ] Full list of resources: Master/Slave Set: ms_mysqldrbd [mysqldrbd] Masters: [ test1.local ] Slaves: [ test2.local ] mystore(ocf::heartbeat:Filesystem):Started test1.local mysqld(lsb:mysqld):Started test1.local myip(ocf::heartbeat:IPaddr):Started test1.local #vip已經啓動在test1.local節點上了。
12.9.1五、測試鏈接VIP
[root@test1 ~]# ip addr #查看test1.local上是否綁定VIP 1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 inet 127.0.0.1/8 scope host lo inet6 ::1/128 scope host valid_lft forever preferred_lft forever 2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen 1000 link/ether 00:0c:29:34:7d:9f brd ff:ff:ff:ff:ff:ff inet 192.168.10.55/24 brd 192.168.10.255 scope global eth0 inet 192.168.10.3/24 brd 192.168.10.255 scope global secondary eth0 #VIP已經綁定到test1.local上的eth0接口上 inet6 fe80::20c:29ff:fe34:7d9f/64 scope link valid_lft forever preferred_lft forever [root@test-3 ~]# mysql -uroot -h192.168.10.3 -p #使用VIP鏈接mysql，這裏要給鏈接的客戶端受權，否則不能登陸 Enter password: Welcome to the MySQL monitor. Commands end with ; or \g. Your MySQL connection id is 6 Server version: 5.5.44 Source distribution Copyright (c) 2000, 2013, Oracle and/or its affiliates. All rights reserved. Oracle is a registered trademark of Oracle Corporation and/or its affiliates. Other names may be trademarks of their respective owners. Type 'help;' or '\h' for help. Type '\c' to clear the current input statement. mysql> show databases; #下面有咱們剛纔建立的兩個庫 +--------------------+ | Database | +--------------------+ | information_schema | | drbd_mysql | | mydb | | mysql | | performance_schema | | test | +--------------------+ 6 rows in set (0.08 sec)
模擬test1.local故障：
[root@test1 mysql]# crm node standby test1.local #將test1.local降級爲slave，看vip是否會自動切換 [root@test1 mysql]# crm node online test1.local [root@test1 mysql]# crm status Last updated: Fri Feb 26 10:20:38 2016 Last change: Fri Feb 26 10:20:35 2016 Stack: classic openais (with plugin) Current DC: test1.local - partition with quorum Version: 1.1.11-97629de 2 Nodes configured, 2 expected votes 5 Resources configured Online: [ test1.local test2.local ] Full list of resources: Master/Slave Set: ms_mysqldrbd [mysqldrbd] Masters: [ test2.local ] Slaves: [ test1.local ] mystore(ocf::heartbeat:Filesystem):Started test2.local mysqld(lsb:mysqld):Started test2.local myip(ocf::heartbeat:IPaddr):Started test2.local #vip已經在test2.local節點上了。
查看test2的IP信息:
[root@test2 ~]# ip addr #在test2.local上查看VIP綁定信息 1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 inet 127.0.0.1/8 scope host lo inet6 ::1/128 scope host valid_lft forever preferred_lft forever 2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen 1000 link/ether 00:0c:29:fd:7f:e5 brd ff:ff:ff:ff:ff:ff inet 192.168.10.56/24 brd 192.168.10.255 scope global eth0 inet 192.168.10.3/24 brd 192.168.10.255 scope global secondary eth0 #VIP已經在test2.local上的eth0接口上。 inet6 fe80::20c:29ff:fefd:7fe5/64 scope link valid_lft forever preferred_lft forever
測試鏈接MySQL:
[root@test-3 ~]# mysql -uroot -h192.168.10.3 -p #鏈接mysql Enter password: Welcome to the MySQL monitor. Commands end with ; or \g. Your MySQL connection id is 1 Server version: 5.5.44 Source distribution Copyright (c) 2000, 2013, Oracle and/or its affiliates. All rights reserved. Oracle is a registered trademark of Oracle Corporation and/or its affiliates. Other names may be trademarks of their respective owners. Type 'help;' or '\h' for help. Type '\c' to clear the current input statement. mysql> show databases; #一切正常。 +--------------------+ | Database | +--------------------+ | information_schema | | drbd_mysql | | mydb | | mysql | | performance_schema | | test | +--------------------+ 6 rows in set (0.06 sec)
到此，corosync+drbd+mysql已經配置完畢，文檔不夠詳細，沒有corosync、heartbeat、drbd、mysql結合的原理講解，沒有發生腦裂後的處理辦法和預防方法。之後有時間在加上吧。
這篇文檔參照了http://litaotao.blog.51cto.com/6224470/1303307，
DRBD安裝參照了：http://88fly.blog.163.com/blog/static/12268039020131113452222/
相關文章
- 1. 實現MooseFS的高可用
- 2. Nginx+Keepalived 實現高可用
- 3. keepalived+MySQL實現高可用
- 4. keepalived實現高可用
- 5. Sentinel實現Redis高可用
- 6. keepalived實現nginx高可用
- 7. Keepalived 實現 nginx 高可用
- 8. Keepalived實現高可用
- 9. nginx+keepalived實現高可用
- 10. 實現redis高可用
- 更多相關文章...
- • 現實生活中的 XML - XML 教程
- • PHP 可用的函數 - PHP參考手冊
- • ☆基於Java Instrument的Agent實現
- • Git可視化極簡易教程 — Git GUI使用方法
相關標籤/搜索
每日一句
-
每一个你不满意的现在，都有一个你没有努力的曾经。
歡迎關注本站公眾號,獲取更多信息
