1、描述
mysql 8.0.19版本,三節點mgr架構,zabbix告警member_mode爲offline。mysql
2、 處理過程
一、查看日誌
查看告警主機數據庫的錯誤日誌,梳理報錯過程sql
二、查看MGR集羣狀態
查看現有集羣是否運行正常,由於告警只提示單實例異常,因此能夠先看日誌。數據庫
三、根據分析,進行操做
處理過程很簡單,使用start group_replication;等待便可架構
3、啓動組複製告警信息
2021-02-24T03:09:00.813707+08:00 0 [Note] [MY-013471] [Repl] Plugin group_replication reported: 'Distributed recovery will transfer data using: Incremental recovery from a group donor'
2021-02-24T03:09:00.814035+08:00 585 [Note] [MY-011576] [Repl] Plugin group_replication reported: 'Establishing group recovery connection with a possible donor. Attempt 1/10'
2021-02-24T03:09:00.814054+08:00 0 [Note] [MY-011503] [Repl] Plugin group_replication reported: 'Group membership changed to mgr2:3311, mgr1:3311, mgr3:3311 on view 16023164214094886:13.'
2021-02-24T03:09:00.845097+08:00 585 [System] [MY-010597] [Repl] 'CHANGE MASTER TO FOR CHANNEL 'group_replication_recovery' executed'. Previous state master_host='', master_port= 3306, master_log_file='', master_log_pos= 4, master_bind=''. New state master_host='mgr3', master_port= 3311, master_log_file='', master_log_pos= 4, master_bind=''.
2021-02-24T03:09:00.874041+08:00 585 [Note] [MY-011580] [Repl] Plugin group_replication reported: 'Establishing connection to a group replication recovery donor 4a84d2c2-0ac9-11eb-8a31-fa163e1a972a at mgr3 port: 3311.'
2021-02-24T03:09:00.874524+08:00 586 [Warning] [MY-010897] [Repl] Storing MySQL user name or password information in the master info repository is not secure and is therefore not recommended. Please consider using the USER and PASSWORD connection options for START SLAVE; see the 'START SLAVE Syntax' in the MySQL Manual for more information.
2021-02-24T03:09:00.877341+08:00 586 [System] [MY-010562] [Repl] Slave I/O thread for channel 'group_replication_recovery': connected to master 'repl@mgr3:3311',replication started in log 'FIRST' at position 4
2021-02-24T03:09:00.932437+08:00 587 [Note] [MY-010581] [Repl] Slave SQL thread for channel 'group_replication_recovery' initialized, starting replication in log 'FIRST' at position 0, relay log '/zxlog/my3311/log/relay-log-group_replication_recovery.000001' position: 4
2021-02-24T03:11:00.016033+08:00 587 [Note] [MY-010559] [Repl] Multi-threaded slave statistics for channel 'group_replication_recovery': seconds elapsed = 120; events assigned = 211969; worker queues filled over overrun level = 0; waited due a Worker queue full = 0; waited due the total size = 0; waited at clock conflicts = 114749941900 waited (count) when Workers occupied = 0 waited when Workers occupied = 0
2021-02-24T03:13:00.200716+08:00 587 [Note] [MY-010559] [Repl] Multi-threaded slave statistics for channel 'group_replication_recovery': seconds elapsed = 120; events assigned = 459777; worker queues filled over overrun level = 0; waited due a Worker queue full = 0; waited due the total size = 0; waited at clock conflicts = 232870937300 waited (count) when Workers occupied = 0 waited when Workers occupied = 0
2021-02-24T03:15:00.228929+08:00 587 [Note] [MY-010559] [Repl] Multi-threaded slave statistics for channel 'group_replication_recovery': seconds elapsed = 120; events assigned = 735233; worker queues filled over overrun level = 0; waited due a Worker queue full = 0; waited due the total size = 0; waited at clock conflicts = 350703540000 waited (count) when Workers occupied = 0 waited when Workers occupied = 0
2021-02-24T03:17:00.348988+08:00 587 [Note] [MY-010559] [Repl] Multi-threaded slave statistics for channel 'group_replication_recovery': seconds elapsed = 120; events assigned = 1006593; worker queues filled over overrun level = 0; waited due a Worker queue full = 0; waited due the total size = 0; waited at clock conflicts = 468687630500 waited (count) when Workers occupied = 0 waited when Workers occupied = 0
2021-02-24T03:19:00.272722+08:00 587 [Note] [MY-010559] [Repl] Multi-threaded slave statistics for channel 'group_replication_recovery': seconds elapsed = 120; events assigned = 1295361; worker queues filled over overrun level = 0; waited due a Worker queue full = 0; waited due the total size = 0; waited at clock conflicts = 586516461900 waited (count) when Workers occupied = 0 waited when Workers occupied = 0
2021-02-24T03:19:58.442469+08:00 585 [Note] [MY-011585] [Repl] Plugin group_replication reported: 'Terminating existing group replication donor connection and purging the corresponding logs.'
2021-02-24T03:19:58.454104+08:00 587 [Note] [MY-010587] [Repl] Slave SQL thread for channel 'group_replication_recovery' exiting, replication stopped in log 'mysql-bin.001168' at position 346695824
2021-02-24T03:19:58.456004+08:00 586 [Note] [MY-011026] [Repl] Slave I/O thread killed while reading event for channel 'group_replication_recovery'.
2021-02-24T03:19:58.456034+08:00 586 [Note] [MY-010570] [Repl] Slave I/O thread exiting for channel 'group_replication_recovery', read up to log 'mysql-bin.001168', position 364987891
2021-02-24T03:19:58.594325+08:00 585 [System] [MY-010597] [Repl] 'CHANGE MASTER TO FOR CHANNEL 'group_replication_recovery' executed'. Previous state master_host='mgr3', master_port= 3311, master_log_file='', master_log_pos= 4, master_bind=''. New state master_host='<NULL>', master_port= 0, master_log_file='', master_log_pos= 4, master_bind=''.
2021-02-24T03:19:58.894546+08:00 572 [Note] [MY-010559] [Repl] Multi-threaded slave statistics for channel 'group_replication_applier': seconds elapsed = 662; events assigned = 1025; worker queues filled over overrun level = 0; waited due a Worker queue full = 0; waited due the total size = 0; waited at clock conflicts = 257603800 waited (count) when Workers occupied = 2 waited when Workers occupied = 6565400
2021-02-24T03:20:11.483440+08:00 0 [Note] [MY-011490] [Repl] Plugin group_replication reported: 'This server was declared online within the replication group.'app
4、定位
根據錯誤日誌分析,大機率是Bug引發致使,後來又把相關信息提交給官方,收到的回覆也是Bug,須要版本升級。目前已經開始在作升級的測試工做。ide