ProxySQL實現讀寫分離

時間  2019-11-06
標籤 proxysql 實現 讀寫 分離 简体版
原文   原文鏈接

ProxySQL實現讀寫分離

環境：

192.168.205.37: as ProxySQL server 
192.168.205.47: as Master server 
192.168.205.57: as Slave server

版本：

OS: centos 7 1810 with mini install 
mariadb-server 5.5.60
proxysql-1.4.15-1-centos7.x86_64.rpm

目地：

proxysql能夠實現讀寫分離，它主要讀取數據庫的read_only變量來判斷那些是主從服務器，當建立讀寫組時它會跟據狀態自動將節點劃分到相應的組中，如寫組10和讀組20，並跟據創建好的規則和對應的組名，從而實現讀寫的分離操做，proxysql是基於一個輕量級的數據庫，全部的設置都要在數據庫中進行更改，步聚有點繁瑣。mysql

步驟：

  1. 使用以下的腳本安將兩個數據庫 
    [root@master data]#cat maridb_yum.sh 
#!/bin/bash
ID=`ip a show dev eth0 | sed -r '3!d;s@(.*inet)(.*)(/.*)@\2@' | cut -d. -f4`
rpm -q mariadb-server ||yum install -y mariadb-server
[ -d /data/mysql ] || mkdir -p /data/mysql
[ -d /data/logs ] || mkdir -p /data/logs
chown mysql:mysql /data/{mysql,logs}
sed -i 's@datadir=/var/lib/mysql@datadir=/data/mysql@' /etc/my.cnf
grep "log-bin" /etc/my.cnf || sed -i '/\[mysqld\]/a log-bin=/data/logs/bin' /etc/my.cnf
grep "innodb_file_per_table" /etc/my.cnf || sed -i '/\[mysqld\]/a innodb_file_per_table = on' /etc/my.cnf
grep "skip_name_resolve" /etc/my.cnf || sed -i '/\[mysqld\]/a skip_name_resolve = on' /etc/my.cnf
grep "server-id" /etc/my.cnf || sed -i "/\[mysqld\]/a server-id=$ID" /etc/my.cnf
service mariadb restart
  2. 修改從節點57的數據庫爲read-only 
    [root@slave data]#vi /etc/my.cnf
[mysqld]
read-only
  3. 記錄主節點的複製位置並建立複製帳號 
    MariaDB [(none)]> show master logs;
+------------+-----------+
| Log_name   | File_size |
+------------+-----------+
| bin.000001 |       245 |
+------------+-----------+
1 row in set (0.00 sec)
[root@master data]#mysql -e "grant replication slave on *.* to repluser@'192.168.205.%' identified by 'centos'"

  4. 修改slave的change master to,並啓動i/o線程sql

    MariaDB [(none)]> CHANGE MASTER TO
    ->   MASTER_HOST='192.168.205.47',
    ->   MASTER_USER='repluser',
    ->   MASTER_PASSWORD='centos',
    ->   MASTER_PORT=3306,
    ->   MASTER_LOG_FILE='bin.000001',
    ->   MASTER_LOG_POS=245;
Query OK, 0 rows affected (0.00 sec)

MariaDB [(none)]> start slave;
Query OK, 0 rows affected (0.00 sec)

MariaDB [(none)]> show slave status\G
*************************** 1. row ***************************
               Slave_IO_State: Waiting for master to send event
                  Master_Host: 192.168.205.47
                  Master_User: repluser
                  Master_Port: 3306
                Connect_Retry: 60
              Master_Log_File: bin.000001
          Read_Master_Log_Pos: 401
               Relay_Log_File: mariadb-relay-bin.000002
                Relay_Log_Pos: 679
        Relay_Master_Log_File: bin.000001
             Slave_IO_Running: Yes
            Slave_SQL_Running: Yes
  5. 導入庫測試，並在從節點上查看數據庫成功 
    [root@master ~]#mysql < hellodb_innodb.sql 
MariaDB [(none)]> show databases;
+--------------------+
| Database           |
+--------------------+
| information_schema |
| hellodb            |
| mysql              |
| performance_schema |
| test               |
+--------------------+
5 rows in set (0.00 sec)

    安裝ProxySQL

  6. 建立yum源，經過yum安裝proxySQL 
    cat <<EOF | tee /etc/yum.repos.d/proxysql.repo
[proxysql_repo]
name= ProxySQL YUM repository
#baseurl=https://repo.proxysql.com/ProxySQL/proxysql-2.0.x/centos/\$releasever
baseurl=https://repo.proxysql.com/ProxySQL/proxysql-1.4.x/centos/\$releasever
gpgcheck=1
gpgkey=https://repo.proxysql.com/ProxySQL/repo_pub_key
EOF
[root@Proxy ~]#yum install proxysql
  7. 安裝一下mariadb client用來鏈接ProxySQL使用 
    [root@Proxy ~]#yum install mariadb
  8. 查看一下裝了那些文件呢？好像很少呀 
    [root@Proxy ~]#rpm -ql proxysql
/etc/init.d/proxysql
/etc/proxysql.cnf
/usr/bin/proxysql
/usr/share/proxysql/tools/proxysql_galera_checker.sh
/usr/share/proxysql/tools/proxysql_galera_writer.pl
  9. 啓用一下服務，他會監聽6032用來管理的，6033是用來用戶鏈接的端口 
    [root@Proxy ~]#service proxysql start
Starting ProxySQL: 2019-08-12 17:00:02 [INFO] Using config file /etc/proxysql.cnf
DONE!
[root@Proxy ~]#ss -ntl
State       Recv-Q Send-Q                    Local Address:Port                                   Peer Address:Port              
LISTEN      0      128                                   *:6032                                              *:*                  
LISTEN      0      128                                   *:6033                                              *:*                  
LISTEN      0      128                                   *:6033                                              *:*                  
LISTEN      0      128                                   *:6033                                              *:*                  
LISTEN      0      128                                   *:6033                                              *:*                  
[root@Proxy ~]#
  10. 改一下咱們熟悉的鏈接端口吧 
    [root@Proxy ~]#vi /etc/proxysql.cnf
  interfaces="0.0.0.0:3306"
  11. 發現沒有改過來？咱們還用SQL命令來改吧 
    [root@Proxy ~]#service proxysql restart
Shutting down ProxySQL: DONE!
Starting ProxySQL: 2019-08-12 17:03:54 [INFO] Using config file /etc/proxysql.cnf
DONE!
[root@Proxy ~]#ss -ntl                                   
State       Recv-Q Send-Q                    Local Address:Port                                   Peer Address:Port              
LISTEN      0      128                                   *:6032                                              *:*                  
LISTEN      0      128                                   *:6033                                              *:*                  
LISTEN      0      128                                   *:6033                                              *:*                  
LISTEN      0      128                                   *:6033                                              *:*                  
LISTEN      0      128                                   *:6033                                              *:*
  12. 咱們使用默認的賬號來鏈接proxysqL,能夠看到它裏面一些系統庫文件 
    [root@Proxy ~]#mysql -uadmin -padmin -P6032 -h127.0.0.1 
MySQL [(none)]> show databases;
+-----+---------------+-------------------------------------+
| seq | name          | file                                |
+-----+---------------+-------------------------------------+
| 0   | main          |                                     |
| 2   | disk          | /var/lib/proxysql/proxysql.db       |
| 3   | stats         |                                     |
| 4   | monitor       |                                     |
| 5   | stats_history | /var/lib/proxysql/proxysql_stats.db |
+-----+---------------+-------------------------------------+
5 rows in set (0.00 sec)
  13. show tables; 默認的是show main庫的表與show tables from main;同樣 
    MySQL [(none)]> show tables;
+--------------------------------------------+
| tables                                     |
+--------------------------------------------+
| global_variables                           |
| mysql_collations                           |
| mysql_group_replication_hostgroups         |
| mysql_query_rules                          |
| mysql_query_rules_fast_routing             |
| mysql_replication_hostgroups               |
| mysql_servers                              |
| mysql_users                                |
| proxysql_servers                           |
| runtime_checksums_values                   |
| runtime_global_variables                   |
| runtime_mysql_group_replication_hostgroups |
| runtime_mysql_query_rules                  |
| runtime_mysql_query_rules_fast_routing     |
| runtime_mysql_replication_hostgroups       |
| runtime_mysql_servers                      |
| runtime_mysql_users                        |
| runtime_proxysql_servers                   |
| runtime_scheduler                          |
| scheduler                                  |
+--------------------------------------------+
20 rows in set (0.00 sec)
  14. 查看一 
    MySQL [(none)]> select * from sqlite_master where name='mysql_servers'\G
*************************** 1. row ***************************
    type: table
    name: mysql_servers
tbl_name: mysql_servers
rootpage: 2
     sql: CREATE TABLE mysql_servers (hostgroup_id INT CHECK (hostgroup_id>=0) NOT NULL DEFAULT 0 , hostname VARCHAR NOT NULL , port INT NOT NULL DEFAULT 3306 , status VARCHAR CHECK (UPPER(status) IN ('ONLINE','SHUNNED','OFFLINE_SOFT', 'OFFLINE_HARD')) NOT NULL DEFAULT 'ONLINE' , weight INT CHECK (weight >= 0) NOT NULL DEFAULT 1 , compression INT CHECK (compression >=0 AND compression <= 102400) NOT NULL DEFAULT 0 , max_connections INT CHECK (max_connections >=0) NOT NULL DEFAULT 1000 , max_replication_lag INT CHECK (max_replication_lag >= 0 AND max_replication_lag <= 126144000) NOT NULL DEFAULT 0 , use_ssl INT CHECK (use_ssl IN(0,1)) NOT NULL DEFAULT 0 , max_latency_ms INT UNSIGNED CHECK (max_latency_ms>=0) NOT NULL DEFAULT 0 , comment VARCHAR NOT NULL DEFAULT '' , PRIMARY KEY (hostgroup_id, hostname, port) )
1 row in set (0.00 sec)
  15. 修改主一下節點的記錄，無論主仍是從有幾個節點主機添加幾個，所有插入到數據庫中監控 
    MySQL [(none)]> insert into mysql_servers(hostgroup_id,hostname,port) values(10,'192.168.205.47',3306);
Query OK, 1 row affected (0.00 sec)
MySQL [(none)]> insert into mysql_servers(hostgroup_id,hostname,port) values(10,'192.168.205.57',3306);  
Query OK, 1 row affected (0.00 sec
  16. 能夠看一兩個節點已經插入到數據庫中 
    MySQL [(none)]> select * from mysql_servers;
+--------------+----------------+------+--------+--------+-------------+-----------------+---------------------+---------+----------------+---------+
| hostgroup_id | hostname       | port | status | weight | compression | max_connections | max_replication_lag | use_ssl | max_latency_ms | comment |
+--------------+----------------+------+--------+--------+-------------+-----------------+---------------------+---------+----------------+---------+
| 10           | 192.168.205.47 | 3306 | ONLINE | 1      | 0           | 1000            | 0                   | 0       | 0              |         |
| 10           | 192.168.205.57 | 3306 | ONLINE | 1      | 0           | 1000            | 0                   | 0       | 0              |         |
+--------------+----------------+------+--------+--------+-------------+-----------------+---------------------+---------+----------------+---------+
2 rows in set (0.00 sec)
  17. 加載配置，使其生效 
    MySQL [(none)]> load mysql servers to runtime;
Query OK, 0 rows affected (0.00 sec)
  18. 保存到磁盤裏 
    MySQL [(none)]> save mysql servers to disk;
Query OK, 0 rows affected (0.01 sec)
  19. 因爲proxysql是查看主和從的數據的read_only來判讀誰是主誰是從的，因此創建一帳號用來鏈接到主和從服務器上，咱們要在主節點上創建這個賬號，它會複製到從節點上 
    MariaDB [(none)]> show variables like 'read_only';
+---------------+-------+
| Variable_name | Value |
+---------------+-------+
| read_only     | OFF   |
+---------------+-------+
1 row in set (0.00 sec)
MariaDB [(none)]> show variables like 'read_only';
+---------------+-------+
| Variable_name | Value |
+---------------+-------+
| read_only     | ON    |
+---------------+-------+
1 row in set (0.00 sec)
MariaDB [(none)]> grant replication client on *.* to monitor@'192.168.205.%' identified by 'centos';
Query OK, 0 rows affected (0.00 sec
  20. 受權proxy用這個建立的帳號鏈接數據庫，設置監控帳號和口令 
    MySQL [(none)]> set mysql-monitor_username='monitor';
Query OK, 1 row affected (0.00 sec)
MySQL [(none)]> set mysql-monitor_password='centos'; 
Query OK, 1 row affected (0.00 sec)

  21. 存盤並生效數據庫

    MySQL [(none)]> load mysql variables to runtime;                
Query OK, 0 rows affected (0.00 sec)

MySQL [(none)]> save mysql variables to disk;                   
Query OK, 97 rows affected (0.00 sec)

  22. 查看一下相關的日誌，之前出錯的緣由是由於默認沒使用monitor密碼是monitor進行鏈接(在proxysql.cnf中能夠看到)，因此會出錯，當你添加完賬號就成功了,centos

    [root@Proxy ~]#vi /etc/proxysql.cnf
  monitor_username="monitor"
  monitor_password="monitor"

MySQL [(none)]> select * from mysql_server_connect_log;
+----------------+------+------------------+-------------------------+-------------------------------------------------------------------------+
| hostname       | port | time_start_us    | connect_success_time_us | connect_error                                                           |
+----------------+------+------------------+-------------------------+-------------------------------------------------------------------------+
| 192.168.205.47 | 3306 | 1565603995322153 | 0                       | Access denied for user 'monitor'@'192.168.205.37' (using password: YES) |
| 192.168.205.47 | 3306 | 1565604055779260 | 0                       | Access denied for user 'monitor'@'192.168.205.37' (using password: YES) |
| 192.168.205.57 | 3306 | 1565604159035893 | 3871                    | NULL                                                                    |
| 192.168.205.47 | 3306 | 1565604159905593 | 3563                    | NULL                                                                    |
+----------------+------+------------------+-------------------------+-------------------------------------------------------------------------+
22 rows in set (0.00 sec)
  23. 查看ping的結果 
    MySQL [(none)]> select * from mysql_server_ping_log;
+----------------+------+------------------+----------------------+-------------------------------------------------------------------------+
| hostname       | port | time_start_us    | ping_success_time_us | ping_error                                                              |
+----------------+------+------------------+----------------------+-------------------------------------------------------------------------+
| 192.168.205.57 | 3306 | 1565604094739272 | 0                    | Access denied for user 'monitor'@'192.168.205.37' (using password: YES) |
| 192.168.205.47 | 3306 | 1565604094919486 | 0                    | Access denied for user 'monitor'@'192.168.205.37' (using password: YES) |
| 192.168.205.57 | 3306 | 1565604099107658 | 745                  | NULL                                                                    |
| 192.168.205.47 | 3306 | 1565604099295895 | 358                  | NULL                                                                    |
+----------------+------+------------------+----------------------+-------------------------------------------------------------------------+
122 rows in set (0.00 sec)
  24. 查看read_only的值爲空，由於沒有分組 
    MySQL [(none)]> select * from mysql_server_read_only_log;
Empty set (0.00 sec)
  25. 查看複製結果爲空 
    MySQL [(none)]> select * from mysql_server_replication_lag_log;
Empty set (0.00 sec)
  26. 須要修改的是main庫中的mysql_replication_hostgroups表，該表有3個字段：writer_hostgroup, reader_hostgroup，comment, 指定寫組的id爲10，讀組的id爲20 
    MySQL [(none)]> insert into mysql_replication_hostgroups values(10,20,"test");
Query OK, 1 row affected (0.00 sec)

  27. 保存生效bash

    MySQL [(none)]> load mysql servers to runtime;                                
Query OK, 0 rows affected (0.00 sec)

MySQL [(none)]> save mysql servers to disk;                                   
Query OK, 0 rows affected (0.02 sec)
  28. proxySQL會跟據剛纔鏈接賬號判斷read_only並自動的把兩個服務器加到這個表中了 
    MySQL [(none)]>  select hostgroup_id,hostname,port,status,weight from mysql_servers;
+--------------+----------------+------+--------+--------+
| hostgroup_id | hostname       | port | status | weight |
+--------------+----------------+------+--------+--------+
| 20           | 192.168.205.57 | 3306 | ONLINE | 1      |
| 10           | 192.168.205.47 | 3306 | ONLINE | 1      |
+--------------+----------------+------+--------+--------+
2 rows in set (0.00 sec)
MySQL [(none)]> select * from mysql_server_read_only_log;
  29. 查看read_only的值已經有記錄 
    +----------------+------+------------------+-----------------+-----------+-------+
| hostname       | port | time_start_us    | success_time_us | read_only | error |
+----------------+------+------------------+-----------------+-----------+-------+
| 192.168.205.57 | 3306 | 1565605365323639 | 565             | 1         | NULL  |
| 192.168.205.47 | 3306 | 1565605365353823 | 1595            | 0         | NULL  |
| 192.168.205.57 | 3306 | 1565605366824223 | 1275            | 1         | NULL  |
| 192.168.205.47 | 3306 | 1565605366844952 | 1607            | 0         | NULL  |
+----------------+------+------------------+-----------------+-----------+-------+
280 rows in set (0.02 sec)
  30. 此時proxysql仍是不知道那些sql語句算讀，那些爲寫，咱們要定義好，讓它來區別併發送到不一樣的服務器上。在主節點建立一個能夠訪問的賬號 
    MariaDB [(none)]> grant all on *.* to sqluser@'192.168.205.%' identified by 'centos'; 
Query OK, 0 rows affected (0.00 sec)
  31. 在proxysql上定義鏈接時使用這個賬號去寫，也就是使用10組，也就是主服務器上寫 
    MySQL [(none)]>  insert into mysql_users(username,password,default_hostgroup) values('sqluser','centos',10);
Query OK, 1 row affected (0.00 sec)

  32. 保存生效服務器

    MySQL [(none)]> load mysql users to runtime;
Query OK, 0 rows affected (0.00 sec)

MySQL [(none)]> save mysql users to disk;
Query OK, 0 rows affected (0.01 sec)
  33. 如今咱們使用剛纔的賬號鏈接一下數據庫(注意如今鏈接測試的是proxysql)，他會默認只會發送到主服務器，由於沒有定義從服務器。 
    [root@Proxy ~]#mysql -usqluser -pcentos -P6033 -h127.0.0.1 -e 'select @@server_id'
+-------------+
| @@server_id |
+-------------+
|          47 |
+-------------+
[root@Proxy ~]#mysql -usqluser -pcentos -P6033 -h127.0.0.1 -e 'create database testdb'
[root@Proxy ~]#mysql -usqluser -pcentos testdb -P6033 -h127.0.0.1 -e 'create table t(id int)'
  34. 在主服務器和從服務器上查看一下剛纔建立的庫和表，同步過去了 
    MariaDB [(none)]> show databases;
+--------------------+
| Database           |
+--------------------+
| information_schema |
| hellodb            |
| mysql              |
| performance_schema |
| test               |
| testdb             |
+--------------------+
6 rows in set (0.00 sec)
MariaDB [(none)]> select * from testdb.t;
Empty set (0.00 sec)

  35. 目前proxysql仍是不知道什麼是讀，什麼是寫，因此全部的操做都會發送到10組中，也就是主服務器上，那麼如今咱們來建SQL語句規則併發

    #select.* for updata爲寫向10組裏發，而其它的select開頭的向20組裏發，也就是讀，那麼沒定義的呢？默認會往10上發，
MySQL [(none)]> insert into mysql_query_rules
    -> (rule_id,active,match_digest,destination_hostgroup,apply)VALUES
    -> (1,1,'^SELECT.*FOR UPDATE$',10,1),(2,1,'^SELECT',20,1);  
Query OK, 2 rows affected (0.00 sec)

#查看一下你添加的規則
MySQL [(none)]> select rule_id,active,match_digest,destination_hostgroup,apply from mysql_query_rules; 
+---------+--------+----------------------+-----------------------+-------+
| rule_id | active | match_digest         | destination_hostgroup | apply |
+---------+--------+----------------------+-----------------------+-------+
| 1       | 1      | ^SELECT.*FOR UPDATE$ | 10                    | 1     |
| 2       | 1      | ^SELECT              | 20                    | 1     |
+---------+--------+----------------------+-----------------------+-------+
2 rows in set (0.00 sec)

  36. 保存生效app

    MySQL [(none)]> load mysql query rules to runtime;
Query OK, 0 rows affected (0.00 sec)

MySQL [(none)]> save mysql query rules to disk;    
Query OK, 0 rows affected (0.01 sec)
  37. 接着再次測試咱們上面運行的命令,select語句就會往20上的從服務器57上發了 
    [root@Proxy ~]#mysql -usqluser -pcentos -P6033 -h127.0.0.1 -e 'select @@server_id'           
+-------------+
| @@server_id |
+-------------+
|          57 |
+-------------+
  38. 若是是以一個事務來執行呢，咱們發現會發送到主節點上，由於begin開始的即不是select updata開頭也是select開頭，因此發送到了默認節點 
    [root@Proxy ~]#mysql -usqluser -pcentos -P6033 -h127.0.0.1 -e 'begin;select @@server_id;commit'       
+-------------+
| @@server_id |
+-------------+
|          47 |
+-------------+
  39. 若是是從建表操做是發送到主節點上，而後主節點再複製到從節點 
    [root@Proxy ~]#mysql -usqluser -pcentos testdb -P6033 -h127.0.0.1 -e 'create table t1(id int)'
在主節點上看這個表，發現已經建成
Database changed
MariaDB [testdb]> show tables;
+------------------+
| Tables_in_testdb |
+------------------+
| t                |
| t1               |
+------------------+
2 rows in set (0.00 sec)
在從節點上看也複製過去了
MariaDB [testdb]> show tables;
+------------------+
| Tables_in_testdb |
+------------------+
| t                |
| t1               |
+------------------+
2 rows in set (0.00 sec)
  40. 咱們能夠使用proxsql查看是否調度成功 
    MySQL [(none)]> select hostgroup hg,sum_time,count_star,digest_text from stats_mysql_query_digest order by sum_time desc; 
+----+----------+------------+----------------------------------+
| hg | sum_time | count_star | digest_text                      |
+----+----------+------------+----------------------------------+
| 10 | 18692    | 1          | create table t(id int)           |
| 10 | 5704     | 1          | create table t1(id int)          |
| 10 | 2002     | 2          | select @@server_id               |
| 20 | 1546     | 1          | select @@server_id               |
| 10 | 819      | 1          | begin                            |
| 10 | 717      | 1          | create database testdb           |
| 10 | 240      | 1          | commit                           |
| 10 | 0        | 2          | select @@version_comment limit ? |
| 10 | 0        | 4          | select @@version_comment limit ? |
+----+----------+------------+----------------------------------+
相關文章
相關標籤/搜索
每日一句
    每一个你不满意的现在，都有一个你没有努力的曾经。
本站公眾號
   歡迎關注本站公眾號,獲取更多信息
聯繫我們 最近搜索 最新文章 沪ICP备13005482号-10 MyBatis教程 SQL 教程 MySQL教程 Java 教程 Thymeleaf 教程 Hibernate教程 Spring教程 Redis教程