DRBD腦裂問題故障處理

    環境:Mysql+heartbeat+drbd 架構node

    問題:mysql-主 宕機 mysql-從接替以後,再恢復Mysql-主以後,發現主的drbd啓動不了,並且從上面也沒法辨識對方,從的狀態是Primary/Unknown ,主的狀態是Secondary/Unknown mysql

         查看從節點的日誌發現腦裂:
[root@node2 ~]# tail -n 12 /var/log/messages 
Sep 21 11:19:53 lab4 kernel: block drbd0: Split-Brain detected but unresolved, dropping connection! 
Sep 21 11:19:53 lab4 kernel: block drbd0: helper command: /sbin/drbdadm split-brain minor-0 
Sep 21 11:19:53 lab4 kernel: block drbd0: helper command: /sbin/drbdadm split-brain minor-0 exit code 0 (0x0) 
Sep 21 11:19:53 lab4 kernel: block drbd0: conn( WFReportParams -> Disconnecting ) 
Sep 21 11:19:53 lab4 kernel: block drbd0: error receiving ReportState, l: 4! 
Sep 21 11:19:53 lab4 kernel: block drbd0: meta connection shut down by peer. 
Sep 21 11:19:53 lab4 kernel: block drbd0: asender terminated 
Sep 21 11:19:53 lab4 kernel: block drbd0: Terminating drbd0_asender 
Sep 21 11:19:53 lab4 kernel: block drbd0: Connection closed 
Sep 21 11:19:53 lab4 kernel: block drbd0: conn( Disconnecting -> StandAlone ) 
Sep 21 11:19:53 lab4 kernel: block drbd0: receiver terminated 
Sep 21 11:19:53 lab4 kernel: block drbd0: Terminating drbd0_receiver
sql

           此時的環境是從節點的heartbeat、mysql啓動,頂替主的工做,從節點的drbd狀態是Primary/Unknown,而且已經掛載着  /dev/drbd0  /data  ,主節點的狀態是未啓動heartbeat和mysql 而且啓動drbd的時候報錯。centos

             

恢復過程:
api

               

將主節點設置爲從節點並丟棄資源數據
[root@node1 ~]# drbdadm secondary r0
[root@node1 ~]# drbdadm -- --discard-my-data connect r0架構

在從節點上手動鏈接資源
[root@node2 ~]# drbdadm connect r0ide


最後查看節點狀態,鏈接已恢復正常
[root@node1 ~]# cat /proc/drbd        
version: 8.3.15 (api:88/proto:86-97) 
GIT-hash: 0ce4d235fc02b5c53c1c52c53433d11a694eab8c build by mockbuild@builder17.centos.org, 2013-03-27 16:04:08 
 0: cs:Connected ro:Primary/Secondary ds:UpToDate/UpToDate C r----- 
    ns:36 nr:0 dw:24 dr:185 al:0 bm:1 lo:0 pe:0 ua:0 ap:0 ep:1 wo:b oos:0ui


此時再啓動主節點的heartbeat 便可!spa

相關文章
相關標籤/搜索