mysql 主從複製以及binlog 測試

###mysql查看binlog日誌內容

 

https://blog.csdn.net/nuli888/article/details/52106910

mysql的binlog日誌位置可經過show variables like '%datadir%';查看，直接打開沒法查看，要看其內容2個辦法：


一、登陸到mysql查看binlog
只查看第一個binlog文件的內容
mysql> show binlog events;


查看指定binlog文件的內容
mysql> show binlog events in 'mysql-bin.000002';

mysql> show binlog events in 'mysql-bin.000001';
+------------------+------+-------------+-----------+-------------+-----------------------------------------------------------+
| Log_name         | Pos  | Event_type  | Server_id | End_log_pos | Info                                                      |
+------------------+------+-------------+-----------+-------------+-----------------------------------------------------------+
| mysql-bin.000001 |    4 | Format_desc |       195 |         106 | Server ver: 5.1.73-log, Binlog ver: 4                     |
| mysql-bin.000001 |  106 | Query       |       195 |         198 | use `hadoop`; delete from user where id=3                 |
| mysql-bin.000001 |  198 | Intvar      |       195 |         226 | INSERT_ID=4                                               |
| mysql-bin.000001 |  226 | Query       |       195 |         332 | use `hadoop`; INSERT INTO user (id,name)VALUES (NULL,1)   |
| mysql-bin.000001 |  332 | Query       |       195 |         424 | use `hadoop`; delete from user where id=3                 |
| mysql-bin.000001 |  424 | Intvar      |       195 |         452 | INSERT_ID=5                                               |
| mysql-bin.000001 |  452 | Query       |       195 |         560 | use `hadoop`; INSERT INTO user (id,name)VALUES (NULL,222) |
| mysql-bin.000001 |  560 | Query       |       195 |         660 | use `hadoop`; DELETE FROM `user` WHERE (`id`='1')         |
| mysql-bin.000001 |  660 | Intvar      |       195 |         688 | INSERT_ID=6                                               |
| mysql-bin.000001 |  688 | Query       |       195 |         795 | use `hadoop`; INSERT INTO `user` (`name`) VALUES ('555')  |
| mysql-bin.000001 |  795 | Intvar      |       195 |         823 | INSERT_ID=7                                               |
| mysql-bin.000001 |  823 | Query       |       195 |         930 | use `hadoop`; INSERT INTO `user` (`name`) VALUES ('555')  |
| mysql-bin.000001 |  930 | Intvar      |       195 |         958 | INSERT_ID=8                                               |
| mysql-bin.000001 |  958 | Query       |       195 |        1065 | use `hadoop`; INSERT INTO `user` (`name`) VALUES ('555')  |
| mysql-bin.000001 | 1065 | Intvar      |       195 |        1093 | INSERT_ID=9                                               |
| mysql-bin.000001 | 1093 | Query       |       195 |        1200 | use `hadoop`; INSERT INTO `user` (`name`) VALUES ('555')  |
| mysql-bin.000001 | 1200 | Query       |       195 |        1300 | use `hadoop`; DELETE FROM `user` WHERE (`id`='9')         |
| mysql-bin.000001 | 1300 | Query       |       195 |        1400 | use `hadoop`; DELETE FROM `user` WHERE (`id`='8')         |
| mysql-bin.000001 | 1400 | Query       |       195 |        1500 | use `hadoop`; DELETE FROM `user` WHERE (`id`='7')         |
| mysql-bin.000001 | 1500 | Query       |       195 |        1600 | use `hadoop`; DELETE FROM `user` WHERE (`id`='4')         |
| mysql-bin.000001 | 1600 | Query       |       195 |        1700 | use `hadoop`; DELETE FROM `user` WHERE (`id`='5')         |
| mysql-bin.000001 | 1700 | Query       |       195 |        1800 | use `hadoop`; DELETE FROM `user` WHERE (`id`='6')         |
| mysql-bin.000001 | 1800 | Intvar      |       195 |        1828 | INSERT_ID=10                                              |
| mysql-bin.000001 | 1828 | Query       |       195 |        1935 | use `hadoop`; INSERT INTO `user` (`name`) VALUES ('555')  |
| mysql-bin.000001 | 1935 | Intvar      |       195 |        1963 | INSERT_ID=11                                              |
| mysql-bin.000001 | 1963 | Query       |       195 |        2070 | use `hadoop`; INSERT INTO `user` (`name`) VALUES ('666')  |
| mysql-bin.000001 | 2070 | Intvar      |       195 |        2098 | INSERT_ID=12                                              |
| mysql-bin.000001 | 2098 | Query       |       195 |        2205 | use `hadoop`; INSERT INTO `user` (`name`) VALUES ('777')  |
+------------------+------+-------------+-----------+-------------+-----------------------------------------------------------+

查看當前正在寫入的binlog文件
mysql> show master status\G

mysql> show master status\G
*************************** 1. row ***************************
            File: mysql-bin.000002
        Position: 106
    Binlog_Do_DB:
Binlog_Ignore_DB: mysql,information_schema,performance_schema
1 row in set (0.00 sec)

獲取binlog文件列表
mysql> show binary logs;

mysql> show binary logs;
+------------------+-----------+
| Log_name         | File_size |
+------------------+-----------+
| mysql-bin.000001 |      3548 |
| mysql-bin.000002 |       106 |
+------------------+-----------+
2 rows in set (0.00 sec)

二、用mysqlbinlog工具查看
基於開始/結束時間
[root@hd3 ~]# mysqlbinlog --start-datetime='2016-08-02 00:00:00' --stop-datetime='2016-08-03 23:01:01' -d hadoop /var/lib/mysql/mysql-bin.000001


基於pos值，注：hadoop是庫名，/var/lib/mysql/mysql-bin.000001是二進制文件路徑
[root@hd3 ~]# mysqlbinlog --start-position=2098 --stop-position=2205 -d hadoop /var/lib/mysql/mysql-bin.000001

/*!40019 SET @@session.max_insert_delayed_threads=0*/;
/*!50003 SET @OLD_COMPLETION_TYPE=@@COMPLETION_TYPE,COMPLETION_TYPE=0*/;
DELIMITER /*!*/;
# at 4
#160803 17:49:51 server id 195  end_log_pos 106         Start: binlog v 4, server v 5.1.73-log created 160803 17:49:51 at startup
# Warning: this binlog is either in use or was not closed properly.
ROLLBACK/*!*/;
BINLOG '
P76hVw/DAAAAZgAAAGoAAAABAAQANS4xLjczLWxvZwAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAA/vqFXEzgNAAgAEgAEBAQEEgAAUwAEGggAAAAICAgC
'/*!*/;
# at 2098
#160803 18:53:56 server id 195  end_log_pos 2205        Query   thread_id=1481  exec_time=115   error_code=0
use `hadoop`/*!*/;
SET TIMESTAMP=1470221636/*!*/;
SET @@session.pseudo_thread_id=1481/*!*/;
SET @@session.foreign_key_checks=1, @@session.sql_auto_is_null=1, @@session.unique_checks=1, @@session.autocommit=1/*!*/;
SET @@session.sql_mode=0/*!*/;
SET @@session.auto_increment_increment=1, @@session.auto_increment_offset=1/*!*/;
/*!\C utf8 *//*!*/;
SET @@session.character_set_client=33,@@session.collation_connection=33,@@session.collation_server=8/*!*/;
SET @@session.lc_time_names=0/*!*/;
SET @@session.collation_database=DEFAULT/*!*/;
INSERT INTO `user` (`name`) VALUES ('777')
/*!*/;
DELIMITER ;
# End of log file
ROLLBACK /* added by mysqlbinlog */;
/*!50003 SET COMPLETION_TYPE=@OLD_COMPLETION_TYPE*/;

 

 

#############12

 

查看是否開啓
mysql> show variables like 'log_bin%';


二、5.7及之後的版本
在5.6的版本狀況下須要多添加一個參數
server-id=123456        #123456是惟一的值就好

 

############sample 主從複製準備  5.6 版本以前 老的方式 

配置主數據庫：
my.cnf：
server-id       = 1
log-bin 
重啓數據庫
登陸並查看：
[root@Mysql-server ~]# mysql -uroot -p199429
mysql> show variables like ‘log_bin‘;  
+---------------+-------+
| Variable_name | Value |
+---------------+-------+
| log_bin       | ON    |
+---------------+-------+
mysql> show variables like ‘server_id‘;
+---------------+-------+
| Variable_name | Value |
+---------------+-------+
| server_id     | 1     |
+---------------+-------+
創建主從複製帳號：
mysql> grant replication slave on *.* to ‘rep‘@‘192.1.1.%‘ identified by ‘199429‘;
mysql> select user,host from mysql.user;
+-----------+-----------+
| user      | host      |
+-----------+-----------+
| root      | 127.0.0.1 |
| bbs       | 192.1.1.% |
| keer      | 192.1.1.% |
| rep       | 192.1.1.% |
| wordpress | 192.1.1.% |
| root      | localhost |
+-----------+-----------+
實現對主數據庫鎖表只讀：
mysql> flush table with read lock;
Query OK, 0 rows affected (0.00 sec)

mysql> show variables like ‘%timeout%‘;
+----------------------------+----------+
| Variable_name              | Value    |
+----------------------------+----------+
| connect_timeout            | 10       |
| delayed_insert_timeout     | 300      |
| innodb_lock_wait_timeout   | 50       |
| innodb_rollback_on_timeout | OFF      |
| interactive_timeout        | 28800    |#####
| lock_wait_timeout          | 31536000 |
| net_read_timeout           | 30       |
| net_write_timeout          | 60       |
| slave_net_timeout          | 3600     |
| wait_timeout               | 28800    |#####
+----------------------------+----------+
10 rows in set (0.00 sec)
查看主庫狀態：
mysql> show master status;
+-------------------------+----------+--------------+------------------+
| File                    | Position | Binlog_Do_DB | Binlog_Ignore_DB |
+-------------------------+----------+--------------+------------------+
| Mysql-server-bin.000001 |      962 |              |                  |
+-------------------------+----------+--------------+------------------+
1 row in set (0.00 sec)
新開窗口備份導出數據：
[root@Mysql-server ~]# mkdir -p /server/backup/ 
[root@Mysql-server ~]# mysqldump -uroot -p199429 --events -A -B |gzip >/server/backup/mysql_bak.$(date +%F).sql.gz
在此查看主庫狀態是否變化：
mysql> show master status;
+-------------------------+----------+--------------+------------------+
| File                    | Position | Binlog_Do_DB | Binlog_Ignore_DB |
+-------------------------+----------+--------------+------------------+
| Mysql-server-bin.000001 |      962 |              |                  |
+-------------------------+----------+--------------+------------------+
1 row in set (0.00 sec)



從數據庫：
配置文件my.cnf
server-id       = 2  ####保證惟一性
重啓從數據庫
登陸從數據庫：
[root@Mysql-server_02 backup]# mysql -uroot -p199429 -S /data/3306/mysql.sock 
mysql> show variables like ‘log_bin‘;
+---------------+-------+
| Variable_name | Value |
+---------------+-------+
| log_bin       | OFF   |
+---------------+-------+
1 row in set (0.00 sec)
mysql> show variables like ‘server_id‘;
+---------------+-------+
| Variable_name | Value |
+---------------+-------+
| server_id     | 3     |
+---------------+-------+
恢復MySQLdump的數據
[root@Mysql-server_02 backup]# cd /server/backup/
[root@Mysql-server_02 backup]# gzip -d mysql_bak.2017-03-23.sql.gz
[root@Mysql-server_02 backup]# mysql -uroot -p‘199429‘ -S /data/3306/mysql.sock <mysql_bak.2017-03-23.sql 

登陸數據庫配置複製參數
[root@Mysql-server_02 backup]# mysql -uroot -p‘199429‘ -S /data/3306/mysql.sock
mysql> CHANGE MASTER TO
    -> MASTER_HOST=‘192.1.1.11‘,
    -> MASTER_PORT=3306,
    -> MASTER_USER=‘rep‘,
    -> MASTER_PASSWORD=‘199429‘,
    -> MASTER_LOG_FILE=‘Mysql-server-bin.000001‘,
    -> MASTER_LOG_POS=962;
費登陸狀態執行方法：
[root@Mysql-server_02 backup]# mysql -uroot -p‘199429‘ -S /data/3306/mysql.sock<< EOF
CHANGE MASTER TO
    MASTER_HOST=‘192.1.1.11‘,
    MASTER_PORT=3306,
    MASTER_USER=‘rep‘,
    MASTER_PASSWORD=‘199429‘,
    MASTER_LOG_FILE=‘Mysql-server-bin.000001‘,
    MASTER_LOG_POS=962;
EOF
實際修改從庫中的master.info文件
[root@Mysql-server_02 backup]# cat /data/3306/data/master.info 
18
mysql-server-bin.000001
962
192.1.1.11
rep
199429
3306
60
0
，。。。。。。。。

啓動主從複製：
mysql> start slave;                                               
Query OK, 0 rows affected (0.00 sec)
mysql> show slave status\G;
*************************** 1. row ***************************
               Slave_IO_State: Waiting for master to send event
                  Master_Host: 192.1.1.11
                  Master_User: rep
                  Master_Port: 3306
                Connect_Retry: 60
              Master_Log_File: Mysql-server-bin.000001
          Read_Master_Log_Pos: 962
               Relay_Log_File: relay-bin.000002
                Relay_Log_Pos: 260
        Relay_Master_Log_File: Mysql-server-bin.000001
             Slave_IO_Running: Yes
            Slave_SQL_Running: Yes
              Replicate_Do_DB: 
          Replicate_Ignore_DB: 
           Replicate_Do_Table: 
       Replicate_Ignore_Table: 
      Replicate_Wild_Do_Table: 
  Replicate_Wild_Ignore_Table: 
                   Last_Errno: 0
                   Last_Error: 
                 Skip_Counter: 0
          Exec_Master_Log_Pos: 962
              Relay_Log_Space: 410
              Until_Condition: None
               Until_Log_File: 
                Until_Log_Pos: 0
           Master_SSL_Allowed: No
           Master_SSL_CA_File: 
           Master_SSL_CA_Path: 
              Master_SSL_Cert: 
            Master_SSL_Cipher: 
               Master_SSL_Key: 
        Seconds_Behind_Master: 0
Master_SSL_Verify_Server_Cert: No
                Last_IO_Errno: 0
                Last_IO_Error: 
               Last_SQL_Errno: 0
               Last_SQL_Error: 
  Replicate_Ignore_Server_Ids: 
             Master_Server_Id: 1
1 row in set (0.00 sec)


測試主從複製功能（省略）

 

 

 

#############sampe 2  測試經過   5.6 版本以前 老的方式 

 

https://www.linuxidc.com/Linux/2016-08/134669.htm

mysql主從複製
mysql支持單向 雙向 鏈式級聯 實時 異步複製，在複製過程當中，一臺服務器充當主服務器（Master）,而一個或多個其餘服務器充當從服務器(Slave)

mysql主從複製的應用場景
一、主從服務器互爲備份
二、主從服務器讀寫分離分擔網站壓力

讀寫分離
中大型公司：經過程序(php,java)
測試環境：代理軟件(mysql-proxy,amoeba)
門戶網站：分佈式dbproxy（讀寫分離，hash負載均衡，健康檢查）
主從同步實踐操做（多實例環境） 
  
一、主庫上面設置server-id值並開啓binlog參數 
[root@CentOS03 ~]# egrep "log-bin|server-id" /data/3306/my.cnf  
log-bin = /data/3306/mysql-bin
server-id = 1 
  
  
檢查實際配置效果 
[root@centos03 ~]# mysql -uroot -p123456 -S /data/3306/mysql.sock -e "show variables like 'log_bin';"  
+---------------+-------+ 
| Variable_name | Value | 
+---------------+-------+ 
| log_bin      | ON    | 
+---------------+-------+ 
  
二、創建用於同步的帳號 
mysql>  grant replication slave  on *.* to rep@'172.16.80.%' identified by '123456'; 
說明：replication slave 是mysql同步的必須權限，此處不要受權all 
  
mysql> flush privileges; 
  
查看受權後的結果 
mysql> show grants for rep@'172.16.80.%'; 
+--------------------------------------------------------------------------------------------------------------------------+ 
| Grants for rep@172.16.80.%                                                                                              | 
+--------------------------------------------------------------------------------------------------------------------------+ 
| GRANT REPLICATION SLAVE ON *.* TO 'rep'@'172.16.80.%' IDENTIFIED BY PASSWORD '*6BB4837EB74329105EE4568DDA7DC67ED2CA2AD9' | 
+--------------------------------------------------------------------------------------------------------------------------+ 
1 row in set (0.00 sec) 
  
  
三、鎖表，導出數據庫 
mysql> flush table with read lock;    #該窗口不能斷，新開一個窗口作數據庫導出操做 
Query OK, 0 rows affected (0.00 sec) 
  
mysql> show master status; 
+------------------+----------+--------------+------------------+ 
| File            | Position | Binlog_Do_DB | Binlog_Ignore_DB | 
+------------------+----------+--------------+------------------+ 
| mysql-bin.000002 |      332 |              |                  | 
+------------------+----------+--------------+------------------+ 
1 row in set (0.00 sec) 
  
[root@centos03 ~]# mysqldump -uroot -p123456 -S /data/3306/mysql.sock -A -B --events --master-data=2 > /opt/rep.sql #導出全部數據庫 
[root@centos03 ~]# vim /opt/rep.sql 
-- CHANGE MASTER TO MASTER_LOG_FILE='mysql-bin.000002', MASTER_LOG_POS=332;  #能夠看到該語句的記錄位置和上面show master status是同樣的，註釋狀態 
  
  
四、數據庫導出後，解鎖 
mysql> show master status;  #再次查看位置點，以驗證上面的鎖表操做是否有效 
+------------------+----------+--------------+------------------+ 
| File            | Position | Binlog_Do_DB | Binlog_Ignore_DB | 
+------------------+----------+--------------+------------------+ 
| mysql-bin.000002 |      332 |              |                  | 
+------------------+----------+--------------+------------------+ 
1 row in set (0.00 sec) 
  
mysql> unlock tables; 
Query OK, 0 rows affected (0.00 sec) 
  
  
五、從庫上面 
[root@centos03 ~]# egrep "log-bin|server-id" /data/3307/my.cnf      
#log-bin = /data/3307/mysql-bin    #log-bin無需開啓 
server-id = 3  #server-id的值不能和主庫上面的值同樣 
  
[root@centos03 ~]# mysql -uroot -p123456 -S /data/3307/mysql.sock < /opt/rep.sql  #導入從主庫備份的數據庫 
  
[root@centos03 ~]# mysql -uroot -phello123 -S /data/3307/mysql.sock              
mysql> change master to \ 
    -> master_host='172.16.80.118',\ 
    -> master_user='rep',\ 
    -> master_password='123456',\ 
    -> master_log_file='mysql-bin.000002',\ 
    -> master_log_pos=332; 
Query OK, 0 rows affected (0.03 sec) 
  
驗證一下 
[root@centos03 ~]# cat /data/3307/data/master.info  
18 
mysql-bin.000002 
332 
172.16.80.118 
rep 
123456 
3306 
60 
0 
  
mysql> start slave;      
Query OK, 0 rows affected (0.01 sec) 
  
mysql> show slave status\G;    #觀察Slave_IO和Slave_SQL 這兩個線程的狀態是不是yes 
*************************** 1. row *************************** 
              Slave_IO_State: Waiting for master to send event 
                  Master_Host: 172.16.80.118 
                  Master_User: rep 
                  Master_Port: 3306 
                Connect_Retry: 60 
              Master_Log_File: mysql-bin.000002 
          Read_Master_Log_Pos: 332 
              Relay_Log_File: relay-bin.000002 
                Relay_Log_Pos: 253 
        Relay_Master_Log_File: mysql-bin.000002 
            Slave_IO_Running: Yes 
            Slave_SQL_Running: Yes 
              Replicate_Do_DB:  
          Replicate_Ignore_DB: mysql 
          Replicate_Do_Table:  
      Replicate_Ignore_Table:  
      Replicate_Wild_Do_Table:  
  Replicate_Wild_Ignore_Table:  
                  Last_Errno: 0 
                  Last_Error:  
                Skip_Counter: 0 
          Exec_Master_Log_Pos: 332 
              Relay_Log_Space: 403 
              Until_Condition: None 
              Until_Log_File:  
                Until_Log_Pos: 0 
          Master_SSL_Allowed: No 
          Master_SSL_CA_File:  
          Master_SSL_CA_Path:  
              Master_SSL_Cert:  
            Master_SSL_Cipher:  
              Master_SSL_Key:  
        Seconds_Behind_Master: 0 
Master_SSL_Verify_Server_Cert: No 
                Last_IO_Errno: 0 
                Last_IO_Error:  
              Last_SQL_Errno: 0 
              Last_SQL_Error:  
  Replicate_Ignore_Server_Ids:  
            Master_Server_Id: 1 
1 row in set (0.00 sec)

六、登陸主庫建立數據庫，看是否會同步到從庫上面 
[root@centos03 ~]# mysql -uroot -p123456 -S /data/3306/mysql.sock  
mysql> create database martin;  
Query OK, 1 row affected (0.01 sec) 
  
  
觀察從庫，能夠看到已經同步過來 
mysql> show databases; 
+--------------------+ 
| Database          | 
+--------------------+ 
| information_schema | 
| martin            | 
| mysql              | 
| performance_schema | 
| test              | 
+--------------------+ 
5 rows in set (0.00 sec) 
  
  
[root@centos03 ~]# cd /data/3307 
[root@centos03 3307]# ls 
data  my.cnf  mysql  mysqld.pid  mysql_martin3307.err  mysql.sock  relay-bin.000001  relay-bin.000002  relay-bin.index  relay-log.info 
[root@centos03 3307]# cat data/master.info  
18 
mysql-bin.000002 
419 
172.16.80.118 
rep 
123456 
3306 
60 
0 
  
[root@centos03 3307]# mysqlbinlog  relay-bin.000002 
/*!\C utf8 *//*!*/; 
SET @@session.character_set_client=33,@@session.collation_connection=33,@@session.collation_server=33/*!*/; 
SET @@session.lc_time_names=0/*!*/; 
SET @@session.collation_database=DEFAULT/*!*/; 
create database martin

mysql主從複製原理總結：

一、異步同步方式

二、邏輯同步模式，多種模式，默認是經過sql語句執行

三、主庫經過記錄bin-log實現對從庫的同步，bin-log記錄數據庫更新的語句

四、主庫一個IO線程，從庫一個IO線程和一個SQL線程

五、從庫關鍵文件master.info  relay-log  relay-info 

六、若是從庫還須要作級聯從庫，從庫須要打開log-bin和log-slave-updates參數

 

監控mysql主從狀態（這裏咱們簡單監控從庫上面io和sql線程yes總數爲不是2就認爲主從出現問題了）

在客戶端編寫腳本

[root@centos03 tools]# cat /tmp/mysql-replication.sh 

#!/bin/bash

/application/mysql/bin/mysql -uroot -p123456 -e 'show slave status\G' -S /data/3307/mysql.sock|grep -Ei "Slave_IO_Running|Slave_SQL_Running"|awk '{print $2}'|grep -c Yes

 

修改zabbix-agent配置文件

UnsafeUserParameters=1

UserParameter=mysql.replication,/tmp/mysql-replication.sh

 

在服務器端添加監控項--觸發器--圖形

 

 

 

正常狀況下的圖形以下

此時咱們模擬主庫掛掉

[root@centos03 tools]# /data/3306/mysql stop

 

本文永久更新連接地址：http://www.linuxidc.com/Linux/2016-08/134669.htm

 

 

#### 5.6 版本能夠參考  GTID 方式+mysqldump搭建準備複製模式  測試經過(數據庫比較小推薦這種方式)

https://dev.mysql.com/doc/refman/5.6/en/replication-gtids-howto.html

 

------=========master----------
1、 啓用gtid:

一、 備份my.cnf文件：
cp /mydb/mysql/app/mysql/my.cnf /mydb/mysql/app/mysql/my.cnf0502
二、master端vi /mydb/mysql/app/mysql/my.cnf加入如下參數：
#GTID
gtid_mode = on
enforce_gtid_consistency = 1
log_slave_updates = 1

log_bin=mysql-bin

 

2.2 create rep user

二、創建用於同步的帳號
mysql> grant replication slave on *.* to rep@'172.16.80.%' identified by '123456';
說明：replication slave 是mysql同步的必須權限，此處不要受權all

 

2、 停應用後處理會話
一、 應用中止

mysql> SET @@global.read_only = ON;

二、 以下查詢若有結果則kill掉：
mysql>select concat('kill connection ', id, ';') from information_schema.processlist;

 

3、重啓

shell> mysqladmin -uusername -p shutdown

 

mysqld_safe  --defaults-file=/**/**/my.cnf &

 

 

2.2 use coyp or Mysqldump backup db. 

##Mysqldump方式在備份過程當中會鎖myisam，搭建的時候須要注意主庫的活動事物狀況。、

##若是是大庫，不適用直接拷貝，使用mysqldump比較好。

### master node :

mysqldump -uroot -p123456 -S /data/3306/mysql.sock -A -B --events --master-data=2 > /opt/rep.sql #導出全部數據庫 

 

 

4、驗證參數

mysql> show variables like 'log_bin%';
mysql> show variables like 'enforce%';
+--------------------------+-------+
| Variable_name | Value |
+--------------------------+-------+
| enforce_gtid_consistency | ON |
+--------------------------+-------+
1 row in set (0.00 sec)

mysql> show variables like 'log_slave_updates';
+-------------------+-------+
| Variable_name | Value |
+-------------------+-------+
| log_slave_updates | ON |
+-------------------+-------+
1 row in set (0.00 sec)

mysql> show variables like 'gtid_mode';
+---------------+-------+
| Variable_name | Value |
+---------------+-------+
| gtid_mode | ON |
+---------------+-------+
1 row in set (0.00 sec)

--============slave：

1、 啓用gtid並修改server_id

一、備份my.cnf文件：
cp /mydb/mysql/app/mysql/my.cnf /mydb/mysql/app/mysql/my.cnf0502
二、slave端vi /mydb/mysql/app/mysql/my.cnf加入如下參數：
#GTID
gtid_mode = on
enforce_gtid_consistency = 1
log_slave_updates = 1

log_bin=mysql-bin

三、vi /mydb/mysql/app/mysql/my.cnf：
server_id=6

2、重啓

shell> mysqladmin -uusername -p shutdown

 

mysqld_safe  --defaults-file=/**/**/my.cnf &

 

 mysql -uroot -p123456 -S /data/3307/mysql.sock < /opt/rep.sql  #導入從主庫備份的數據庫 

 

3、驗證參數

mysql> show variables like 'log_bin%';

mysql> show variables like 'enforce%';
+--------------------------+-------+
| Variable_name | Value |
+--------------------------+-------+
| enforce_gtid_consistency | ON |
+--------------------------+-------+
1 row in set (0.00 sec)

mysql> show variables like 'log_slave_updates';
+-------------------+-------+
| Variable_name | Value |
+-------------------+-------+
| log_slave_updates | ON |
+-------------------+-------+
1 row in set (0.00 sec)

mysql> show variables like 'gtid_mode';
+---------------+-------+
| Variable_name | Value |
+---------------+-------+
| gtid_mode | ON |
+---------------+-------+
1 row in set (0.00 sec)

4、change master到指定源
change master to
master_host='xxx',
master_user='root',
master_password='',
master_port=xxx,
master_auto_position = 1;

5、啓動slave同步
一、start slave；

 2. show slave status\G

 

6.Master node

mysql> SET @@global.read_only = off;

 

 

############## 5.6 版本能夠參考  GTID 方式+ mysqlbackup 搭建準備複製模式  測試經過( 數據庫比較大 推薦這種方式)

目前個人環境都仍是mysql5.6的，因此選擇用3.12的版本

mysqlbackup企業版在mysql的開源社區是下載不了的，須要有metalink的帳號才能下載。（早期是能夠下載的，估計後來用戶的體驗不錯，須要受權購買oracle的服務才能夠了）

目前個人環境都仍是mysql5.6的，因此選擇用3.12的版本（3.12的版本修復了幾個重要的bug，建議用這個版本）

其餘設置命令使用的快捷方式：ln -s /opt/mysql/meb-3.12/bin/mysqlbackup /usr/bin/mysqlbackup

3.1.3 建立單獨的備份用戶

其餘業務分開帳號的使用，數據庫做爲重要的系統，通常都會每一年變動管理密碼的，這裏單獨使用一個避免後期還須要變動；
備份用戶所須要的基本權限

GRANT CREATE, INSERT, DROP, UPDATE ON mysql.backup_progress TO 'mysqlbackup'@'localhost' identified by '123456';;
GRANT CREATE, INSERT, SELECT, DROP, UPDATE ON mysql.backup_history TO 'mysqlbackup'@'localhost';
GRANT REPLICATION CLIENT ON *.* TO 'mysqlbackup'@'localhost';
GRANT SUPER ON *.* TO 'mysqlbackup'@'localhost';
GRANT LOCK TABLES, SELECT, CREATE, DROP, FILE ON *.* TO 'mysqlbackup'@'localhost';
GRANT reload on *.* to 'mysqlbackup'@'localhost';

 

###machine 1 primay database:
全備的腳本

#####
[root@db02 backup]# mysqlbackup --user=mysqlbackup --password=123456 --backup-dir=/tmp/dba --with-timestamp --socket /db/mysql/data/mysqltmp/mysql_3307.sock backup

####output
180518 08:26:24 mysqlbackup: INFO: MEB logfile created at /tmp/dba/2018-05-18_08-26-24/meta/MEB_2018-05-18.08-26-24_backup.log

--------------------------------------------------------------------
Server Repository Options:
--------------------------------------------------------------------
datadir = /db/mysql/data_3307/
innodb_data_home_dir =
innodb_data_file_path = ibdata1:12M:autoextend
innodb_log_group_home_dir = /db/mysql/data_3307/
innodb_log_files_in_group = 3
innodb_log_file_size = 536870912
innodb_page_size = 16384
innodb_checksum_algorithm = innodb
innodb_undo_directory = /db/mysql/data_3307/
innodb_undo_tablespaces = 0
innodb_undo_logs = 128

--------------------------------------------------------------------
Backup Config Options:
--------------------------------------------------------------------
datadir = /tmp/dba/2018-05-18_08-26-24/datadir
innodb_data_home_dir = /tmp/dba/2018-05-18_08-26-24/datadir
innodb_data_file_path = ibdata1:12M:autoextend
innodb_log_group_home_dir = /tmp/dba/2018-05-18_08-26-24/datadir
innodb_log_files_in_group = 3
innodb_log_file_size = 536870912
innodb_page_size = 16384
innodb_checksum_algorithm = innodb
innodb_undo_directory = /tmp/dba/2018-05-18_08-26-24/datadir
innodb_undo_tablespaces = 0
innodb_undo_logs = 128

mysqlbackup: INFO: Unique generated backup id for this is 15266031843454002

mysqlbackup: INFO: Creating 14 buffers each of size 16777216.
180518 08:26:26 mysqlbackup: INFO: Full Backup operation starts with following threads
1 read-threads 6 process-threads 1 write-threads
180518 08:26:26 mysqlbackup: INFO: System tablespace file format is Antelope.
180518 08:26:26 mysqlbackup: INFO: Starting to copy all innodb files...
180518 08:26:26 mysqlbackup: INFO: Copying /db/mysql/data_3307/ibdata1 (Antelope file format).
180518 08:26:26 mysqlbackup: INFO: Found checkpoint at lsn 1626047.
180518 08:26:26 mysqlbackup: INFO: Starting log scan from lsn 1625600.
180518 08:26:26 mysqlbackup: INFO: Copying log...
180518 08:26:26 mysqlbackup: INFO: Log copied, lsn 1626047.
180518 08:26:26 mysqlbackup: INFO: Copying /db/mysql/data_3307/mysql/innodb_index_stats.ibd (Antelope file format).
180518 08:26:26 mysqlbackup: INFO: Copying /db/mysql/data_3307/mysql/innodb_table_stats.ibd (Antelope file format).
180518 08:26:26 mysqlbackup: INFO: Copying /db/mysql/data_3307/mysql/slave_master_info.ibd (Antelope file format).
180518 08:26:26 mysqlbackup: INFO: Copying /db/mysql/data_3307/mysql/slave_relay_log_info.ibd (Antelope file format).
180518 08:26:26 mysqlbackup: INFO: Copying /db/mysql/data_3307/mysql/slave_worker_info.ibd (Antelope file format).
180518 08:26:26 mysqlbackup: INFO: Completing the copy of innodb files.
180518 08:26:26 mysqlbackup: INFO: Starting to copy Binlog files...
180518 08:26:26 mysqlbackup: INFO: Copying /db/mysql/data_3307/mysql-bin.000001.
180518 08:26:26 mysqlbackup: INFO: Copying /db/mysql/data_3307/mysql-bin.000002.
180518 08:26:26 mysqlbackup: INFO: Copying /db/mysql/data_3307/mysql-bin.000003.
180518 08:26:26 mysqlbackup: INFO: Copying /db/mysql/data_3307/mysql-bin.000004.
180518 08:26:26 mysqlbackup: INFO: Copying /db/mysql/data_3307/mysql-bin.000005.
180518 08:26:26 mysqlbackup: INFO: Copying /db/mysql/data_3307/mysql-bin.000006.
180518 08:26:26 mysqlbackup: INFO: Copying /db/mysql/data_3307/mysql-bin.000007.
180518 08:26:27 mysqlbackup: INFO: Preparing to lock tables: Connected to mysqld server.
180518 08:26:27 mysqlbackup: INFO: Starting to lock all the tables...
180518 08:26:27 mysqlbackup: INFO: All tables are locked and flushed to disk
180518 08:26:27 mysqlbackup: INFO: Copying /db/mysql/data_3307/mysql-bin.000008.
180518 08:26:28 mysqlbackup: INFO: Completed the copy of binlog files...
180518 08:26:28 mysqlbackup: INFO: Opening backup source directory '/db/mysql/data_3307/'
180518 08:26:28 mysqlbackup: INFO: Starting to backup all non-innodb files in
subdirectories of '/db/mysql/data_3307/'
180518 08:26:28 mysqlbackup: INFO: Copying the database directory 'mysql'
180518 08:26:28 mysqlbackup: INFO: Copying the database directory 'peng'
180518 08:26:28 mysqlbackup: INFO: Copying the database directory 'performance_schema'
180518 08:26:29 mysqlbackup: INFO: Copying the database directory 'test'
180518 08:26:29 mysqlbackup: INFO: Copying the database directory 'tmp'
180518 08:26:29 mysqlbackup: INFO: Completing the copy of all non-innodb files.
180518 08:26:30 mysqlbackup: INFO: A copied database page was modified at 1626047.
(This is the highest lsn found on page)
Scanned log up to lsn 1626047.
Was able to parse the log up to lsn 1626047.
Maximum page number for a log record 0
180518 08:26:30 mysqlbackup: INFO: All tables unlocked
180518 08:26:30 mysqlbackup: INFO: All MySQL tables were locked for 2.818 seconds.
180518 08:26:30 mysqlbackup: INFO: Reading all global variables from the server.
180518 08:26:30 mysqlbackup: INFO: Completed reading of all global variables from the server.
180518 08:26:30 mysqlbackup: INFO: Creating server config files server-my.cnf and server-all.cnf in /tmp/dba/2018-05-18_08-26-24
180518 08:26:30 mysqlbackup: INFO: Full Backup operation completed successfully.
180518 08:26:30 mysqlbackup: INFO: Backup created in directory '/tmp/dba/2018-05-18_08-26-24'
180518 08:26:30 mysqlbackup: INFO: MySQL binlog position: filename mysql-bin.000008, position 1659

-------------------------------------------------------------
Parameters Summary
-------------------------------------------------------------
Start LSN : 1625600
End LSN : 1626047
-------------------------------------------------------------

mysqlbackup completed OK!
[mysql@localhost dba]$

#######

##[root@db02 backup]# mysqlbackup --backup-dir=/backup/2017-09-18_13-49-11 apply-log #由於在備份期間數據庫還在讀寫，把這期間的log進行應用，達到數據的一致性

mysqlbackup --backup-dir=/tmp/dba/2018-05-18_08-26-24 apply-log

####

180518 08:30:51 mysqlbackup: INFO: MEB logfile created at /tmp/dba/2018-05-18_08-26-24/meta/MEB_2018-05-18.08-30-51_apply_log.log

--------------------------------------------------------------------
Backup Config Options:
--------------------------------------------------------------------
datadir = /tmp/dba/2018-05-18_08-26-24/datadir
innodb_data_home_dir = /tmp/dba/2018-05-18_08-26-24/datadir
innodb_data_file_path = ibdata1:12M:autoextend
innodb_log_group_home_dir = /tmp/dba/2018-05-18_08-26-24/datadir
innodb_log_files_in_group = 3
innodb_log_file_size = 536870912
innodb_page_size = 16384
innodb_checksum_algorithm = innodb

mysqlbackup: INFO: Creating 14 buffers each of size 65536.
180518 08:30:51 mysqlbackup: INFO: Apply-log operation starts with following threads
1 read-threads 1 process-threads
mysqlbackup: INFO: Using up to 100 MB of memory.
180518 08:30:51 mysqlbackup: INFO: ibbackup_logfile's creation parameters:
start lsn 1625600, end lsn 1626047,
start checkpoint 1626047.
InnoDB: Doing recovery: scanned up to log sequence number 1626047
mysqlbackup: INFO: InnoDB: Starting an apply batch of log records to the database...
InnoDB: Progress in percent: 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99
mysqlbackup: INFO: InnoDB: Setting log file size to 536870912
InnoDB: Progress in MB: 100 200 300 400 500
mysqlbackup: INFO: InnoDB: Setting log file size to 536870912
InnoDB: Progress in MB: 100 200 300 400 500
mysqlbackup: INFO: InnoDB: Setting log file size to 536870912
InnoDB: Progress in MB: 100 200 300 400 500
180518 08:30:55 mysqlbackup: INFO: We were able to parse ibbackup_logfile up to
lsn 1626047.
mysqlbackup: INFO: Last MySQL binlog file position 0 1659, file name mysql-bin.000008:1659
180518 08:30:55 mysqlbackup: INFO: The first data file is '/tmp/dba/2018-05-18_08-26-24/datadir/ibdata1'
and the new created log files are at '/tmp/dba/2018-05-18_08-26-24/datadir'
180518 08:30:55 mysqlbackup: INFO: Apply-log operation completed successfully.
180518 08:30:55 mysqlbackup: INFO: Full backup prepared for recovery successfully.

mysqlbackup completed OK!
########

從庫還原 （in standby db） ：

 

machine 2:

mysqlbackup --force --socket=/var/lib/mysql/mysql_3307.sock --datadir=/mysqldb/data_3307 --backup-dir=/tmp/dba/2018-05-18_08-26-24 copy-back

[root@localhost 2018-05-18_08-26-24]# mysqlbackup --force --socket=/var/lib/mysql/mysql_3307.sock --datadir=/mysqldb/data_3307 --backup-dir=/tmp/dba/2018-05-18_08-26-24 copy-back
MySQL Enterprise Backup version 3.12.0 Linux-3.8.13-35.2.1.el7uek.x86_64-x86_64 [2015/03/10]
Copyright (c) 2003, 2015, Oracle and/or its affiliates. All Rights Reserved.

mysqlbackup: INFO: Starting with following command line ...
mysqlbackup --force --socket=/var/lib/mysql/mysql_3307.sock
--datadir=/mysqldb/data_3307 --backup-dir=/tmp/dba/2018-05-18_08-26-24
copy-back

mysqlbackup: INFO:
IMPORTANT: Please check that mysqlbackup run completes successfully.
At the end of a successful 'copy-back' run mysqlbackup
prints "mysqlbackup completed OK!".

180518 10:18:57 mysqlbackup: INFO: MEB logfile created at /tmp/dba/2018-05-18_08-26-24/meta/MEB_2018-05-18.10-18-57_copy_back.log

mysqlbackup: WARNING: If you restore to a server of a different version, the innodb_data_file_path parameter might have a different default. In that case you need to add 'innodb_data_file_path=ibdata1:12M:autoextend' to the target server configuration.
mysqlbackup: WARNING: If you restore to a server of a different version, the innodb_log_files_in_group parameter might have a different default. In that case you need to add 'innodb_log_files_in_group=3' to the target server configuration.
mysqlbackup: WARNING: If you restore to a server of a different version, the innodb_log_file_size parameter might have a different default. In that case you need to add 'innodb_log_file_size=536870912' to the target server configuration.
--------------------------------------------------------------------
Server Repository Options:
--------------------------------------------------------------------
datadir = /mysqldb/data_3307
innodb_data_home_dir = /mysqldb/data_3307
innodb_data_file_path = ibdata1:12M:autoextend
innodb_log_group_home_dir = /mysqldb/data_3307
innodb_log_files_in_group = 3
innodb_log_file_size = 536870912
innodb_page_size = Null
innodb_checksum_algorithm = innodb

--------------------------------------------------------------------
Backup Config Options:
--------------------------------------------------------------------
datadir = /tmp/dba/2018-05-18_08-26-24/datadir
innodb_data_home_dir = /tmp/dba/2018-05-18_08-26-24/datadir
innodb_data_file_path = ibdata1:12M:autoextend
innodb_log_group_home_dir = /tmp/dba/2018-05-18_08-26-24/datadir
innodb_log_files_in_group = 3
innodb_log_file_size = 536870912
innodb_page_size = 16384
innodb_checksum_algorithm = innodb

mysqlbackup: INFO: Creating 14 buffers each of size 16777216.
180518 10:18:57 mysqlbackup: INFO: Copy-back operation starts with following threads
1 read-threads 1 write-threads
180518 10:18:57 mysqlbackup: INFO: Copying /tmp/dba/2018-05-18_08-26-24/datadir/ibdata1.
180518 10:18:57 mysqlbackup: INFO: Copying /tmp/dba/2018-05-18_08-26-24/datadir/mysql/innodb_index_stats.ibd.
180518 10:18:57 mysqlbackup: INFO: Copying /tmp/dba/2018-05-18_08-26-24/datadir/mysql/innodb_table_stats.ibd.
180518 10:18:57 mysqlbackup: INFO: Copying /tmp/dba/2018-05-18_08-26-24/datadir/mysql/slave_master_info.ibd.
180518 10:18:57 mysqlbackup: INFO: Copying /tmp/dba/2018-05-18_08-26-24/datadir/mysql/slave_relay_log_info.ibd.
180518 10:18:57 mysqlbackup: INFO: Copying /tmp/dba/2018-05-18_08-26-24/datadir/mysql/slave_worker_info.ibd.
180518 10:18:57 mysqlbackup: INFO: Starting to copy Binlog files...
180518 10:18:57 mysqlbackup: INFO: Copying /tmp/dba/2018-05-18_08-26-24/datadir/mysql-bin.000001.
180518 10:18:57 mysqlbackup: INFO: Copying /tmp/dba/2018-05-18_08-26-24/datadir/mysql-bin.000002.
180518 10:18:57 mysqlbackup: INFO: Copying /tmp/dba/2018-05-18_08-26-24/datadir/mysql-bin.000003.
180518 10:18:57 mysqlbackup: INFO: Copying /tmp/dba/2018-05-18_08-26-24/datadir/mysql-bin.000004.
180518 10:18:57 mysqlbackup: INFO: Copying /tmp/dba/2018-05-18_08-26-24/datadir/mysql-bin.000005.
180518 10:18:57 mysqlbackup: INFO: Copying /tmp/dba/2018-05-18_08-26-24/datadir/mysql-bin.000006.
180518 10:18:57 mysqlbackup: INFO: Copying /tmp/dba/2018-05-18_08-26-24/datadir/mysql-bin.000007.
180518 10:18:57 mysqlbackup: INFO: Copying /tmp/dba/2018-05-18_08-26-24/datadir/mysql-bin.000008.
180518 10:18:58 mysqlbackup: INFO: Completed the copy of binlog files...
180518 10:18:58 mysqlbackup: INFO: Copying the database directory 'mysql'
180518 10:18:58 mysqlbackup: INFO: Copying the database directory 'peng'
180518 10:18:58 mysqlbackup: INFO: Copying the database directory 'performance_schema'
180518 10:18:58 mysqlbackup: INFO: Copying the database directory 'test'
180518 10:18:58 mysqlbackup: INFO: Copying the database directory 'tmp'
180518 10:18:58 mysqlbackup: INFO: Completing the copy of all non-innodb files.
180518 10:18:58 mysqlbackup: INFO: Copying the log file 'ib_logfile0'
mysqlbackup: Progress in MB: 200 400
180518 10:19:12 mysqlbackup: INFO: Copying the log file 'ib_logfile1'
mysqlbackup: Progress in MB: 600 800 1000
180518 10:19:15 mysqlbackup: INFO: Copying the log file 'ib_logfile2'
mysqlbackup: Progress in MB: 1200 1400
180518 10:19:19 mysqlbackup: INFO: Creating server config files server-my.cnf and server-all.cnf in /mysqldb/data_3307
180518 10:19:19 mysqlbackup: INFO: Copy-back operation completed successfully.
180518 10:19:19 mysqlbackup: INFO: Finished copying backup files to '/mysqldb/data_3307'

mysqlbackup completed OK! with 3 warnings

#############

in standby database 

chown -R mysql:mysql /mysqldb/data_3307
mysqld_safe --defaults-file=/mysqldb/data_3307/my.cnf &

  

mysql> show master status;
mysql> reset master;
mysql> show master status;

 

4、在slave端進行GTID_PURGED，執行backup_gtid_executed.sql中的SET @@GLOBAL.GTID_PURGED語句：

mysql> SET @@GLOBAL.GTID_PURGED='4e461dbc-54a7-11e8-8589-005056a41701:1-9';

 

mysql> change master to
-> master_host='10.241.95.221',
-> master_user='rep',
-> master_password='123456',
-> master_port=3307,
-> master_auto_position = 1;

 

6、啓動slave同步

1.start slave；

2. show slave status\G

####

 

#####more  info about gtids

17.1.3.2 Setting Up Replication Using GTIDs
This section describes a process for configuring and starting GTID-based replication in MySQL 5.6. This is a 「cold start」 procedure that assumes either that you are starting the replication master for the first time, or that it is possible to stop it; for information about provisioning replication slaves using GTIDs from a running master, see Section 17.1.3.3, 「Using GTIDs for Failover and Scaleout」.

The key steps in this startup process for the simplest possible GTID replication topology—consisting of one master and one slave—are as follows:

If replication is already running, synchronize both servers by making them read-only.

Stop both servers.

Restart both servers with GTIDs, binary logging, and slave update logging enabled, and with statements that are unsafe for GTID-based replication disabled. In addition, the servers should be started in read-only mode, and the slave SQL and I/O threads should be prevented from starting on the slave.

The mysqld options necessary to start the servers as described are discussed in the example that follows later in this section.

Instruct the slave to use the master as the replication data source and to use auto-positioning. The SQL statements needed to accomplish this step are described in the example that follows later in this section.

Take a new backup. Binary logs containing transactions without GTIDs cannot be used on servers where GTIDs are enabled, so backups taken before this point cannot be used with your new configuration.

Start the slave, then disable read-only mode again on both servers, so that they can accept updates.

In the following example, two servers are already running as master and slave, using MySQL's 「classic」 file-based replication protocol.

Most of the steps that follow require the use of the MySQL root account or another MySQL user account that has the SUPER privilege. mysqladmin shutdown requires either the SUPER privilege or the SHUTDOWN privilege.

Step 1: Synchronize the servers. Make the servers read-only. To do this, enable the read_only system variable by executing the following statement on both servers:

mysql> SET @@global.read_only = ON;
Wait for all ongoing transactions to commit or roll back. Then, allow the slave to catch up with the master. It is extremely important that you make sure the slave has processed all updates before continuing.

If you use binary logs for anything other than replication, for example to do point in time backup and restore, wait until you do not need the old binary logs containing transactions without GTIDs. Ideally, wait for the server to purge all binary logs, and wait for any existing backup to expire.

Important
It is important to understand that logs containing transactions without GTIDs cannot be used on servers where GTIDs are enabled. Before proceeding, you must be sure that transactions without GTIDs do not exist anywhere in the topology.

Step 2: Stop both servers. Stop each server using mysqladmin as shown here, where username is the user name for a MySQL user having sufficient privileges to shut down the server:

shell> mysqladmin -uusername -p shutdown
Then supply this user's password at the prompt.

Step 3: Restart both servers with GTIDs enabled. To enable binary logging with global transaction identifiers, each server must be started with GTID mode, binary logging, slave update logging enabled, and with statements that are unsafe for GTID-based replication disabled. In addition, you should prevent unwanted or accidental updates from being performed on either server by starting both in read-only mode. This means that both servers must be started with (at least) the options shown in the following invocation of mysqld_safe:

shell> mysqld_safe --gtid_mode=ON --log-bin --log-slave-updates --enforce-gtid-consistency &
Note
Prior to MySQL 5.6.9, --enforce-gtid-consistency was named --disable-gtid-unsafe-statements.

In addition, you should start the slave with the --skip-slave-start option along with the other server options specified in the example just shown.

Note
--gtid-mode is not a boolean, but an enumeration. Use one of the values ON or OFF only, when setting this option. Using a numeric value such as 0 or 1 can lead to unexpected results.

For more information about the --gtid-mode and --enforce-gtid-consistency server options, see Section 17.1.4.5, 「Global Transaction ID Options and Variables」.

Depending on your configuration, supply additional options to mysqld_safe or other mysqld startup script.

Step 4: Direct the slave to use the master. Tell the slave to use the master as the replication data source, and to use GTID-based auto-positioning rather than file-based positioning. Execute a CHANGE MASTER TO statement on the slave, using the MASTER_AUTO_POSITION option to tell the slave that transactions will be identified by GTIDs.

You may also need to supply appropriate values for the master's host name and port number as well as the user name and password for a replication user account which can be used by the slave to connect to the master; if these have already been set prior to Step 1 and no further changes need to be made, the corresponding options can safely be omitted from the statement shown here.

mysql> CHANGE MASTER TO
> MASTER_HOST = host,
> MASTER_PORT = port,
> MASTER_USER = user,
> MASTER_PASSWORD = password,
> MASTER_AUTO_POSITION = 1;
Neither the MASTER_LOG_FILE option nor the MASTER_LOG_POS option may be used with MASTER_AUTO_POSITION set equal to 1. Attempting to do so causes the CHANGE MASTER TO statement to fail with an error.

Step 5: Take a new backup. Existing backups that were made before you enabled GTIDs can no longer be used on these servers now that you have enabled GTIDs. Take a new backup at this point, so that you are not left without a usable backup.

For instance, you can execute FLUSH LOGS on the server where you are taking backups. Then either explicitly take a backup or wait for the next iteration of any periodic backup routine you may have set up.

Step 6: Start the slave and disable read-only mode. Start the slave like this:

mysql> START SLAVE;
Allow the master to begin accepting updates once again by running the following statement:

mysql> SET @@global.read_only = OFF;
GTID-based replication should now be running, and you can begin (or resume) activity on the master as before. Section 17.1.3.3, 「Using GTIDs for Failover and Scaleout」, discusses creation of new slaves when using GTIDs.

PREV HOME UP NEXT
User Comments
Sign UpLogin You must be logged in to post a comment.

 

 

###############22

https://www.cnblogs.com/1477717815fuming/p/8006143.html

 

公司規模已經造成，用戶數據已成爲公司的核心命脈，一次老王一不當心把數據庫文件刪除，經過mysqldump備份策略恢復用了兩個小時，在這兩小時中，公司業務中斷，損失100萬，老王作出深入檢討，公司也所以對於數據庫的性能和可靠性提出更高要求。要求對數據庫進行改造，使其承載力進行提高，故障修復時間減小，有沒有能實現的方案呢？

 

數據庫常遇到的問題

1、性能問題

一、向上拓展 scale up ：針對單臺服務器，提升服務器的硬件性能，好比：內存，cpu等，個體自己 容易達到極限

二、向外拓展 scale out ：多臺服務器造成集羣，共同完成一件事情

2、可用性問題

一、數據庫服務中斷

二、誤操做數據損壞

三、硬件故障

四、數據庫升級測試遭遇bug

五、黑客攻擊

 

數據庫高可用技術說明

高可用架構對於互聯網服務基本是標配，不管是應用服務仍是數據庫服務都須要作到高可用。雖然互聯網服務號稱7*24小時不間斷服務，但多多少少有一些時候服務不可用，好比某些時候網頁打不開，百度不能搜索或者沒法發微博，發微信等。通常而言，衡量高可用作到什麼程度能夠經過一年內服務不可用時間做爲參考，要作到3個9的可用性，一年內只能累計有8個小時不可服務，而若是要作到5個9的可用性，則一年內只能累計5分鐘服務中斷。因此雖然說每一個公司都說本身的服務是7*24不間斷的，但實際上能作到5個9的屈指可數，甚至根本作不到，國內互聯網巨頭BAT(百度，阿里巴巴，騰訊)都有由於故障致使的停服問題。對於一個系統而言，可能包含不少模塊，好比前端應用，緩存，數據庫，搜索，消息隊列等，每一個模塊都須要作到高可用，才能保證整個系統的高可用。對於數據庫服務而言，高可用可能更復雜，對用戶的服務可用，不只僅是能訪問，還須要有正確性保證，所以，對於實現數據庫高可用，對互聯網公司來講極其重要！

 

企業級數據庫高可用架構圖

 

Mysql主從架構技術說明

Mysql內建的複製功能是構建大型，高性能應用程序的基礎。將Mysql的數據分佈到多個系統上去，這種分佈的機制，是經過將Mysql的某一臺主機（Master）的數據複製到其它主機（slaves）上，並從新執行一遍來實現的。複製過程當中一個服務器充當主服務器，而一個或多個其它服務器充當從服務器。主服務器將更新寫入二進制日誌文件，這些日誌能夠記錄發送到從服務器的更新。當一個從服務器鏈接主服務器時，它通知主服務器從服務器在日誌中讀取的最後一次成功更新的位置。從服務器接收從那時起發生的任何更新，而後封鎖並等待主服務器通知新的更新。

主從複製架構圖

 

數據庫複製特性

Mysql複製解決的問題

MySQL複製技術有如下一些特色：

(1) 數據分佈 (Data distribution )

(2) 負載平衡(load balancing)

(3) 備份(Backups)

(4) 高可用性和容錯性 High availabilityand failover

 

Mysql複製如何工做

Mysql的複製功能主要有3個步驟：

(1) 主服務器（master）將改變記錄到二進制日誌(binarylog)中（這些記錄叫作二進制日誌事件，binary log events）

(2) 從服務器（slave）將主服務器master的binary logevents拷貝到它的中繼日誌(relay log)

(3) slave重作中繼日誌中的事件，將改變反映它本身的數據。

一、該過程的第一部分就是master記錄二進制日誌。在每一個事務更新數據完成以前，master在二進制日誌記錄這些改變。MySQL將事務串行的寫入二進制日誌，在事件寫入二進制日誌完成後，master通知存儲引擎提交事務。此後可接收slave的請求

二、下一步就是slave將master的binary log拷貝到它本身的中繼日誌。首先，slave開始一個工做線程——I/O線程。I/O線程在master上打開一個普通的鏈接，而後開始在主節點上binlog dump process（二進制轉存線程）。Binlog dump process從master的二進制日誌中讀取事件，若是已經跟上master，它會睡眠並等待master產生新的事件。I/O線程將這些事件寫入中繼日誌。

三、 SQL slave thread（SQL從線程）處理該過程的最後一步。SQL線程從中繼日誌讀取事件，並重放其中的事件而更新slave的數據，使其與master中的數據一致。只要該線程與I/O線程保持一致，中繼日誌一般會位於OS的緩存中，因此中繼日誌的開銷很小。

I/O線程：將master數據庫二進制日誌拉到slave數據庫上，並將二進制日誌寫到中繼日誌，寫完以後，他會睡眠並等待master數據庫二進制日誌更新，一旦更新，就會寫入slave數據庫的中繼日誌中

SQL線程：讀取中繼日誌的事件，並在數據庫中執行，寫入到內存中，使slave數據庫的數據與master數據庫中的數據一致

Mysql實現企業級數據庫主從複製架構實戰

注意：slave數據庫只能是可讀的，不能是可寫的，若是改變了slave數據庫的數據，master不能從slave數據庫上同步數據，致使主從數據庫數據不一致。

實戰演練

1、環境準備

centos系統服務器2臺、一臺用戶作Mysql主服務器，一臺用於作Mysql從服務器，都在同一個網段中，配置好yum源、防火牆關閉、各節點時鐘服務同步、各節點之間能夠經過主機名互相通訊

2、準備步驟：

一、iptables -F && setenforce 清空防火牆策略，關閉selinux

二、拿兩臺服務器都使用yum方式安裝Mysql服務，要求版本一致

三、分別啓動兩臺服務器mysql服務，確保服務正常

3、實現步驟:

一、配置master主服務器

對master進行配置，包括打開二進制日誌，指定惟一的servr ID。例如，在配置文件加入以下值

vim /etc/my.cnf

server-id=1 #配置server-id，讓主服務器有惟一ID號（讓從服務器知道他的主服務器是誰）

log-bin=mysql-bin #打開Mysql日誌，日誌格式爲二進制

skip-name-resolve#關閉名稱解析，（非必須）

而後重啓數據庫服務

systemctl restart mariadb

2．建立複製賬號

在Master的數據庫中創建一個備份賬戶：每一個slave使用標準的MySQL用戶名和密碼鏈接master

。進行復制操做的用戶會授予REPLICATION SLAVE權限。（給從服務器受權，讓他能從主服務器拷貝二進制日誌）

mysql

GRANT REPLICATION SLAVE, REPLICATION CLIENT ON *.* TO slave@'192.168.10.%' IDENTIFIED BY 'magedu';

3．查看主服務器狀態

在Master的數據庫執行show master status，查看主服務器二進制日誌狀態

四、配置slave從服務器

對slave進行配置，打開中繼日誌，指定惟一的servr ID，設置只讀權限。在配置文件加入以下值

vim /etc/my.cnf

server-id=2 #配置server-id，讓從服務器有惟一ID號

relay_log = mysql-relay-bin #打開Mysql日誌，日誌格式爲二進制

read_only = 1 #設置只讀權限

log_bin = mysql-bin #開啓從服務器二進制日誌

log_slave_updates = 1 #使得更新的數據寫進二進制日誌中

而後重啓數據庫服務

systemctl restart mariadb

5．啓動從服務器複製線程

讓slave鏈接master，並開始重作master二進制日誌中的事件。

mysql

CHANGE MASTER TO MASTER_HOST='192.168.10.190',

MASTER_USER='slave',

MASTER_PASSWORD='magedu',

MASTER_LOG_FILE='mysql-bin.000001',

MASTER_LOG_POS=278;

執行start slave；# 啓動複製線程。

六、查看從服務器狀態

可以使用SHOW SLAVE STATUS\G查看從服務器狀態，以下所示，也可用show processlist \G查看前複製狀態：

mysql

SHOW SLAVE STATUS\G

Slave_IO_Running: Yes #IO線程正常運行

Slave_SQL_Running: Yes #SQL線程正常運行

7.測試

理想的結果是在主服務器上添加的數據，在從服務器上也會同步

在主服務器上

 

在從服務器上

4、添加新slave服務器

假如master已經運行好久了，想對新安裝的slave進行數據同步，甚至它沒有master的數據。

此時，有幾種方法可使slave從另外一個服務開始，例如，從master拷貝數據，從另外一個slave克隆，從最近的備份開始一個slave。爲了加快Slave與master同步，可用如下方式先進行數據同步：

(1)master的某個時刻的數據快照；

(2)數據庫的備份數據

(3)master的二進制日誌文件。

 

實現主從從架構

也能夠搭建主從從架構，讓從服務器之間進行復制

就是在從服務器也開啓二進制日誌，而後從的從I/O線程再將從的二進制日誌給拷貝過來寫入到本身的relay log中，而後sql線程再讀取relay log中的事件，在數據庫中執行，寫入到內存中。

 

Mysql複製過濾器

複製過濾器：

僅複製有限一個或幾個數據庫相關的數據，而非全部；由複製過濾器進行；

有兩種實現思路：

(1) 主服務器

主服務器僅向二進制日誌中記錄有關特定數據庫相關的寫操做；

binlog_do_db=

binlog_ignore_db=

(2) 從服務器

從服務器的SQL THREAD僅重放關注的數據庫或表相關的事件，並將其應用於本地；

Replicate_Do_DB=

Replicate_Ignore_DB=

 

企業常見數據庫架構

1、單一master和多slave

在實際應用場景中，MySQL複製90%以上都是一個Master複製到一個或者多個Slave的架構模式，主要用於讀壓力比較大的應用的數據庫端廉價擴展解決方案。由於只要Master和Slave的壓力不是太大（尤爲是Slave端壓力）的話，異步複製的延時通常都不多不多。尤爲是自從Slave端的複製方式改爲兩個線程處理以後，更是減少了Slave端的延時問題。而帶來的效益是，對於數據實時性要求不是特別高的應用，只須要經過廉價的pcserver來擴展Slave的數量，將讀壓力分散到多臺Slave的機器上面，便可經過分散單臺數據庫服務器的讀壓力來解決數據庫端的讀性能瓶頸，畢竟在大多數數據庫應用系統中的讀壓力仍是要比寫壓力大不少。這在很大程度上解決了目前不少中小型網站的數據庫壓力瓶頸問題，甚至有些大型網站也在使用相似方案解決數據庫瓶頸。

單一master和多slave架構圖

(1) 不一樣的slave扮演不一樣的做用(例如使用不一樣的索引，或者不一樣的存儲引擎)；

(2) 用一個slave做爲備用master，只進行復制；#主服務器掛了以後，可在從服務器執行

1> 在備機上執行STOP SLAVE 和RESET MASTER

2> 查看show slave status \G;

3> 而後修改應用的鏈接地址。

(3) 用一個遠程的slave，用於災難恢復；

2、互爲主從Master-Master(Master-Master in Active-Active Mode)

Master-Master複製的兩臺服務器，既是master，又是另外一臺服務器的slave。這樣，任何一方所作的變動，都會經過複製應用到另一方的數據庫中。

即：在兩臺服務器上既執行master的操做又執行slave的操做（注意：兩臺數據庫都必須是可寫的）

互爲主從複製過程

互爲主從：兩個節點各自都要開啓binlog和relay log；

一、數據不一致；

二、自動增加id；

什麼是自增加ID？

對於某些惟一性的字段，能夠經過設置自增加ID來實現,自增加ID的數據，表明這個表中存在一條惟一的記錄；而自增加id是確定不會重複的；

建立表，設置ID爲自增加

create table userInfo (id int PRIMARY KEY AUTO_INCREMENT,name varchar(50) NOT NULL)；

兩邊插入數據看數據增加

insert into userInfo(name) value('xiao'),('da'),('lao');

定義一個節點使用奇數id

auto_increment_increment=2 #表示自增加字段每次遞增的量

auto_increment_offset=1 #表示自增加字段從那個數開始

另外一個節點使用偶數id

auto_increment_increment=2

auto_increment_offset=2

 

配置：

一、server_id必需要使用不一樣值；

二、均啓用binlog和relay log； read only = 0(由於互爲主從，因此必須是可寫的)

三、存在自動增加id的表，爲了使得id不相沖突，須要定義其自動增加方式；

服務啓動後執行以下兩步：

四、都受權有複製權限的用戶帳號；

五、各把對方指定爲主節點；

 

實驗：數據庫互爲主從複製步驟

1.修改mysql配置文件

一臺數據庫服務器上

vim /etc/my.cnf

server-id = 1

log_bin = mysql_bin

relay_log = relay-log

skip-name-resolve = on

log_slave_updates = 1

auto_increment_increment=2

auto_increment_offset=1

另外一臺數據庫服務器上

vim /etc/my.cnf

server-id = 2

relay_log = relay-log

log_bin = mysql-log

skip-name-resolve = on

log_slave_updates = 1

auto_increment_increment=2

auto_increment_offset=2

修改完配置文件後，重啓數據庫服務

systemctl restart mariadb

2．建立複製賬號

分別在兩臺數據庫服務器上建立複製帳號

mysql

GRANT REPLICATION SLAVE, REPLICATION CLIENT ON *.* TO slave@'192.168.10.%' IDENTIFIED BY 'magedu';

3．啓動從服務器複製線程

讓slave鏈接master，並開始重作master二進制日誌中的事件。

mysql

CHANGE MASTER TO MASTER_HOST='192.168.10.190',

MASTER_USER='slave',

MASTER_PASSWORD='magedu',

MASTER_LOG_FILE='mysql-bin.000001',

MASTER_LOG_POS=278;

執行start slave；# 啓動複製線程。

另外一臺數據庫服務器也是如此

四、查看從服務器狀態

可以使用SHOW SLAVE STATUS\G查看從服務器狀態，以下所示，也可用show processlist \G查看前複製態：

mysql

SHOW SLAVE STATUS\G

Slave_IO_Running: Yes #IO線程正常運行

Slave_SQL_Running: Yes #SQL線程正常運行

兩臺數據庫服務器都顯示如上結果就ok。

5.建立表，設置ID爲自增加,兩邊插入數據看數據增加

在一臺數據庫服務器上

mysql

create database dnf;

use dnf;

create table userinfo (id int PRIMARY KEY AUTO_INCREMENT,name varchar(20) NOT NULL);

insert into userinfo (name) values('ni'),('wo'),('ta');

而後查看錶，由於是自增加id，從1開始，步長爲2，因此添加的數據id爲1,3,5

而後在另外一臺數據庫服務器插入數據，由於是自增加id，從2開始，步長爲2，因此新添加的數據id爲6,8,10

 

排錯：當配置文件中配置中繼日誌格式不當心配置錯了，或者讓slave鏈接master，執行sql語句不當心寫錯了，都有可能致使start slave；報錯，此時能夠show slave status\G;會出現一大串信息，裏面會提示錯誤。找到錯誤之後，重置slave，reset slave;從新設置，而後再start slave；

注意：mysql的錯誤日誌很是重要，能夠提供錯誤信息，從而找到錯誤緣由。

互爲主從容易致使數據不一致，此時咱們能夠用兩個實例來互爲主從

 

三種複製方式

異步複製（Asynchronous replication）

MySQL默認的複製便是異步的，主庫在執行完客戶端提交的事務後會當即將結果返給給客戶端，並不關心從庫是否已經接收並處理，這樣就會有一個問題，主若是crash掉了，此時主上已經提交的事務可能並無傳到從上，若是此時，強行將從提高爲主，可能致使新主上的數據不完整

全同步複製（Fully synchronous replication）

指當主庫執行完一個事務，全部的從庫都執行了該事務才返回給客戶端。由於須要等待全部從庫執行完該事務才能返回，因此全同步複製的性能必然會收到嚴重的影響。須要有超時時間。

半同步複製（Semisynchronous replication）

介於異步複製和全同步複製之間，主庫在執行完客戶端提交的事務後不是馬上返回給客戶端，而是等待至少一個從庫接收到並寫到relay log中才返回給客戶端。相對於異步複製，半同步複製提升了數據的安全性，同時它也形成了必定程度的延遲，這個延遲最少是一個TCP/IP往返的時間。因此，半同步複製最好在低延時的網絡中使用。

 

半同步複製

支持多種插件：/usr/lib64/mysql/plugins/

須要安裝方可以使用：

mysql> INSTALL PLUGIN plugin_name SONAME 'shared_library_name';

半同步複製：

semisync_master.so

semisync_slave.so

主節點：

INSTALL PLUGIN rpl_semi_sync_master SONAME 'semisync_master.so';

MariaDB [mydb]> SHOW GLOBAL VARIABLES LIKE 'rpl_semi%';

+------------------------------------+-------+

| Variable_name | Value |

+------------------------------------+-------+

| rpl_semi_sync_master_enabled | OFF |

| rpl_semi_sync_master_timeout | 10000 |

| rpl_semi_sync_master_trace_level | 32 |

| rpl_semi_sync_master_wait_no_slave | ON |

+------------------------------------+-------+

MariaDB [mydb]> SET GLOBAL rpl_semi_sync_master_enabled=ON/1;

stop slave;

start slave;

從節點：

INSTALL PLUGIN rpl_semi_sync_slave SONAME 'semisync_slave.so';

MariaDB [mydb]> SHOW GLOBAL VARIABLES LIKE 'rpl_semi%';

+---------------------------------+-------+

| Variable_name | Value |

+---------------------------------+-------+

| rpl_semi_sync_slave_enabled | OFF |

| rpl_semi_sync_slave_trace_level | 32 |

+---------------------------------+-------+

MariaDB [mydb]> STOP SLAVE IO_THREAD;

MariaDB [mydb]> SET GLOBAL rpl_semi_sync_slave_enabled = ON ;

MariaDB [mydb]> SHOW GLOBAL VARIABLES LIKE 'rpl_semi%';

MariaDB [mydb]> START SLAVE IO_THREAD;

stop slave;

start slave;

可查看從庫錯誤日誌觀察是否生效

master錯誤日誌

slave錯誤日誌

 

mysql優化：

1.能夠用數據緩存，常見的memcache

2.數據庫自己有不少緩存機制，可以使用對應的緩存策略

3.對數據來講，竟可能使用索引

4.對請求而言，能夠實現讀寫分離，對讀請求負載均衡

5.對大數據庫或者表，可根據業務邏輯進行分庫分表

6.多有的優化，儘量網內存中存放

分庫分表

分庫：當數據庫的數據很是龐大，能夠把數據庫分紅幾個數據庫，每一個數據庫當一類數據，最後在拼接起來

分表：有兩種分法：

1.水平拆分：一個表中有10億條記錄，將這10億條記錄分紅每10萬條記錄爲一個表

2.垂直拆分：將一個字段或者多個字段分紅一個表

http://tieba.baidu.com/p/4558183228