MySQL ProxySQL讀寫分離實踐
目的
在上一篇文章MySQL ProxySQL讀寫分離使用初探裏初步介紹了ProxySQL的使用，本文繼續介紹它的一些特色和DBProxy的性能差別。深刻一些去了解ProxySQL，經過例子來講明ProxySQL的一些特性和用sysbench對其進行測試來對比分析說明。html
環境：
Distributor ID： Ubuntu Description ： Ubuntu 14.04.5 LTS Release ： 14.04 Codename ： trusty
MySQL Master ：192.168.200.202 MySQL Slave ：192.168.200.132 APP IP ：192.168.200.25/64
測試
本文測試環境是在上一篇文章的基礎上進行的，因此已經作了一主一從的讀寫分離。在此基礎上，若從庫掛了，會怎麼樣？這裏先把從庫（132）shutdown掉，看看讀去了哪裏。mysql
tips：如何修改管理接口的用戶名密碼？除了初始化時候修改配置文件，還有一個方法是在管理端口設置：git
admin@127.0.0.1 : (none) 12:52:53>set admin-admin_credentials='zjy:zjy'; Query OK, 1 row affected (0.00 sec)
admin模塊修改（select * from global_variables where variable_name like 'admin%';）須要用admin加載：github
admin@127.0.0.1 : (none) 12:53:02>load admin variables to runtime; Query OK, 0 rows affected (0.00 sec) admin@127.0.0.1 : (none) 12:53:33>save admin variables to disk; Query OK, 9 rows affected (0.00 sec)
1，從庫不可用算法
① 關閉從庫sql
當前ProxySQL下後端MySQL的運行狀況：數據庫
admin@127.0.0.1 : (none) 11:25:26>select hostgroup_id,hostname,port,status from runtime_mysql_servers; +--------------+-----------------+------+--------+ | hostgroup_id | hostname | port | status | +--------------+-----------------+------+--------+ | 100 | 192.168.200.202 | 3306 | ONLINE | | 1000 | 192.168.200.132 | 3306 | ONLINE | +--------------+-----------------+------+--------+
shutdown從庫（132）後，後端MySQL的運行狀況：後端
admin@127.0.0.1 : (none) 11:33:24>select hostgroup_id,hostname,port,status from runtime_mysql_servers; +--------------+-----------------+------+---------+ | hostgroup_id | hostname | port | status | +--------------+-----------------+------+---------+ | 100 | 192.168.200.202 | 3306 | ONLINE | | 1000 | 192.168.200.132 | 3306 | SHUNNED | +--------------+-----------------+------+---------+
此時讀的操做會報超時：安全
sbuser@192.168.200.24 : sbtest 11:30:40>select * from x; ERROR 9001 (HY000): Max connect timeout reached while reaching hostgroup 1000 after 10000ms
緣由是proxysql的核心都在規則，shutdown從以後，proxysql仍是想路由到 hostgroup=1000，它不會自動選擇默認的100(mysql_users裏配置的default_hostgroup) 。服務器
這裏解決的辦法是：在mysql_servers的hostgroup 1000 裏面要插一條主庫的記錄，而後把weight設小，當讀不到從庫，回去主庫查詢。
admin@127.0.0.1 : (none) 11:50:13>insert into mysql_servers(hostgroup_id,hostname,port,weight,max_connections,max_replication_lag,comment) values(1000,'192.168.200.202',3306,1,1000,10,'test proxysql'); Query OK, 1 row affected (0.00 sec) admin@127.0.0.1 : (none) 11:50:42>update mysql_servers set weight=9 where hostgroup_id=1000 and hostname='192.168.200.132'; Query OK, 1 row affected (0.00 sec) admin@127.0.0.1 : (none) 11:53:41>select hostgroup_id,hostname,port,weight from mysql_servers; +--------------+-----------------+------+--------+ | hostgroup_id | hostname | port | weight | +--------------+-----------------+------+--------+ | 100 | 192.168.200.202 | 3306 | 1 | | 1000 | 192.168.200.132 | 3306 | 9 | | 1000 | 192.168.200.202 | 3306 | 1 | +--------------+-----------------+------+--------+ admin@127.0.0.1 : (none) 11:54:03>load mysql servers to runtime; Query OK, 0 rows affected (0.01 sec) admin@127.0.0.1 : (none) 11:54:28>save mysql servers to disk; Query OK, 0 rows affected (0.00 sec) admin@127.0.0.1 : (none) 11:54:38>select hostgroup_id,hostname,port,status from runtime_mysql_servers; +--------------+-----------------+------+---------+ | hostgroup_id | hostname | port | status | +--------------+-----------------+------+---------+ | 100 | 192.168.200.202 | 3306 | ONLINE | | 1000 | 192.168.200.132 | 3306 | SHUNNED | | 1000 | 192.168.200.202 | 3306 | ONLINE | +--------------+-----------------+------+---------+
此時讀的操做正常：
sbuser@192.168.200.24 : sbtest 11:52:37>select * from x; +------+ | id | +------+ | 123 | | 123 | | 123 | +------+ 3 rows in set (0.01 sec)
說明從關閉了以後讀操做確實去主上執行了。當從庫恢復以後，之後的讀操做主庫也能夠處理 1/10 的讀請求。
② 從庫延遲/從庫中止複製
在上一篇文章中已經創建了監控帳號：proxysql，因爲須要執行show slave status的命令來得到延遲時間，因此須要權限SUPER 和 REPLICATION CLIENT。而且須要設置mysql_servers.max_replication_lag的值，因爲mysql_servers.max_replication_lag僅適用於從，但也能夠將其配置爲全部主機，不管是從仍是主（不會有任何影響）。
-- 設置監控帳號權限 dba@192.168.200.202 : sbtest 10:44:38>GRANT SUPER, REPLICATION CLIENT ON *.* TO 'proxysql'@'192.168.200.24' IDENTIFIED BY PASSWORD '*BF27B4C7AAD278126E228AA8427806E870F64F39'; Query OK, 0 rows affected (0.00 sec) -- 設置延遲的閾值 admin@127.0.0.1 : (none) 11:04:50>UPDATE mysql_servers SET max_replication_lag=5; Query OK, 3 rows affected (0.00 sec) -- 應用配置 admin@127.0.0.1 : (none) 11:04:54>load mysql servers to runtime; Query OK, 0 rows affected (0.01 sec) admin@127.0.0.1 : (none) 11:05:04>save mysql servers to disk; Query OK, 0 rows affected (0.01 sec)
主從複製正常的狀況下，後端MySQL的狀況：
admin@127.0.0.1 : (none) 11:05:13>select hostgroup_id,hostname,port,status,max_replication_lag from runtime_mysql_servers; +--------------+-----------------+------+--------+---------------------+ | hostgroup_id | hostname | port | status | max_replication_lag | +--------------+-----------------+------+--------+---------------------+ | 1000 | 192.168.200.132 | 3306 | ONLINE | 5 | | 1000 | 192.168.200.202 | 3306 | ONLINE | 5 | | 100 | 192.168.200.202 | 3306 | ONLINE | 5 | +--------------+-----------------+------+--------+---------------------+ 3 rows in set (0.00 sec)
從庫執行stop slave以後，後端MySQL的狀況：
admin@127.0.0.1 : (none) 11:06:52>select hostgroup_id,hostname,port,status,max_replication_lag from runtime_mysql_servers; +--------------+-----------------+------+---------+---------------------+ | hostgroup_id | hostname | port | status | max_replication_lag | +--------------+-----------------+------+---------+---------------------+ | 1000 | 192.168.200.132 | 3306 | SHUNNED | 5 | | 1000 | 192.168.200.202 | 3306 | ONLINE | 5 | | 100 | 192.168.200.202 | 3306 | ONLINE | 5 | +--------------+-----------------+------+---------+---------------------+ 3 rows in set (0.00 sec)
此時，132從庫不可用，讀都到了HG 1000的202上去了，能夠自行測試。 也能夠在日誌裏看到：
2017-05-11 11:06:43 MySQL_HostGroups_Manager.cpp:934:replication_lag_action(): [WARNING] Shunning server 192.168.200.132:3306 with replication lag of 60 second
日誌顯示延遲60s，這個是怎麼回事？這裏須要說明下幾個變量：
mysql-monitor_replication_lag_interval：主從延遲檢測時間，默認10秒。
mysql-monitor_slave_lag_when_null：當爲null時，設置的延遲值，默認爲60。
admin@127.0.0.1 : (none) 11:08:35>select * from global_variables where variable_name like 'mysql-monitor%lag%'; +----------------------------------------+----------------+ | variable_name | variable_value | +----------------------------------------+----------------+ | mysql-monitor_replication_lag_interval | 10000 | | mysql-monitor_replication_lag_timeout | 1000 | | mysql-monitor_slave_lag_when_null | 60 | +----------------------------------------+----------------+ 3 rows in set (0.00 sec)
根據mysql_servers.max_replication_lag設置的閾值，這2個參數能夠根據本身的狀況來設置，好比設置檢測時間爲1500。延遲的記錄也能夠經過表來查看：
admin@127.0.0.1 : (none) 11:19:47>select * from mysql_server_replication_lag_log limit 3; +-----------------+------+------------------+-----------------+----------+-------+ | hostname | port | time_start_us | success_time_us | repl_lag | error | +-----------------+------+------------------+-----------------+----------+-------+ | 192.168.200.132 | 3306 | 1494472189886932 | 411 | 0 | NULL | | 192.168.200.202 | 3306 | 1494472189887224 | 372 | NULL | NULL | | 192.168.200.202 | 3306 | 1494472189887640 | 325 | NULL | NULL | +-----------------+------+------------------+-----------------+----------+-------+ 3 rows in set (0.00 sec)
主從延遲的狀況和stop slave的狀況同樣，只是stop slave是把延遲設置成了60s。
小結：經過上面的測試說明ProxySQL能夠在從庫不可用時進行下線，不須要人爲再進行干預，等到恢復正常以後自動上線提供服務。
2，多路由規則
① 根據庫路由
在現有基礎上再增長一個主從：
M：192.168.200.97 S：192.168.200.245
受權帳號：程序和監控帳號
dba@192.168.200.97 : proxysql 12:39:39>GRANT SUPER, REPLICATION CLIENT ON *.* TO 'proxysql'@'192.168.200.24' IDENTIFIED BY PASSWORD '*BF27B4C7AAD278126E228AA8427806E870F64F39'; Query OK, 0 rows affected (0.01 sec) dba@192.168.200.97 : proxysql 12:42:50>grant select,insert,update,delete on proxysql.* to proxysql@192.168.200.24 identified by 'proxysql'; Query OK, 0 rows affected (0.00 sec)
配置ProxySQL：
admin@127.0.0.1 : (none) 12:43:35>insert into mysql_servers(hostgroup_id,hostname,port,weight,max_connections,max_replication_lag,comment) values(101,'192.168.200.97',3306,1,1000,10,'test proxysql'); Query OK, 1 row affected (0.00 sec) admin@127.0.0.1 : (none) 12:45:15>insert into mysql_servers(hostgroup_id,hostname,port,weight,max_connections,max_replication_lag,comment) values(1001,'192.168.200.245',3306,9,1000,10,'test proxysql'); Query OK, 1 row affected (0.00 sec) admin@127.0.0.1 : (none) 12:45:24>insert into mysql_servers(hostgroup_id,hostname,port,weight,max_connections,max_replication_lag,comment) values(1001,'192.168.200.97',3306,1,1000,10,'test proxysql'); Query OK, 1 row affected (0.00 sec) admin@127.0.0.1 : (none) 12:45:36>insert into mysql_users(username,password,active,default_hostgroup,transaction_persistent) values('proxysql','proxysql',1,101,1); Query OK, 1 row affected (0.00 sec) admin@127.0.0.1 : (none) 12:46:55>INSERT INTO mysql_query_rules(active,schemaname,match_pattern,destination_hostgroup,apply) VALUES(1,'proxysql','^SELECT.*FOR UPDATE$',101,1); Query OK, 1 row affected (0.00 sec) admin@127.0.0.1 : (none) 12:56:47> admin@127.0.0.1 : (none) 12:56:47>INSERT INTO mysql_query_rules(active,schemaname,match_pattern,destination_hostgroup,apply) VALUES(1,'proxysql','^SELECT',1001,1); Query OK, 1 row affected (0.00 sec) -- 應用保存配置 admin@127.0.0.1 : (none) 12:56:55>load mysql servers to runtime; admin@127.0.0.1 : (none) 12:57:00>load mysql users to runtime;
admin@127.0.0.1 : (none) 12:57:04>load mysql query rules to runtime; admin@127.0.0.1 : (none) 12:57:11>save mysql servers to disk; admin@127.0.0.1 : (none) 12:57:17>save mysql users to disk; admin@127.0.0.1 : (none) 12:57:21>save mysql query rules to disk;
rules、servers、users信息：
admin@127.0.0.1 : (none) 03:28:11>select rule_id,active,username,schemaname,client_addr,destination_hostgroup,match_pattern,flagIN,flagOUT,apply from mysql_query_rules; +---------+--------+----------+------------+-------------+-----------------------+----------------------+--------+---------+-------+ | rule_id | active | username | schemaname | client_addr | destination_hostgroup | match_pattern | flagIN | flagOUT | apply | +---------+--------+----------+------------+-------------+-----------------------+----------------------+--------+---------+-------+ | 3 | 1 | NULL | NULL | NULL | 100 | ^SELECT.*FOR UPDATE$ | 0 | NULL | 1 | | 4 | 1 | NULL | NULL | NULL | 1000 | ^SELECT | 0 | NULL | 1 | | 5 | 1 | NULL | proxysql | NULL | 101 | ^SELECT.*FOR UPDATE$ | 0 | NULL | 1 | | 6 | 1 | NULL | proxysql | NULL | 1001 | ^SELECT | 0 | NULL | 1 | +---------+--------+----------+------------+-------------+-----------------------+----------------------+--------+---------+-------+ 4 rows in set (0.00 sec) admin@127.0.0.1 : (none) 03:29:10>select username,default_hostgroup from mysql_users; +----------+-------------------+ | username | default_hostgroup | +----------+-------------------+ | sbuser | 100 | | proxysql | 101 | +----------+-------------------+ 2 rows in set (0.00 sec) admin@127.0.0.1 : (none) 03:29:28>select hostgroup_id,hostname,port,status from mysql_servers; +--------------+-----------------+------+--------+ | hostgroup_id | hostname | port | status | +--------------+-----------------+------+--------+ | 1000 | 192.168.200.132 | 3306 | ONLINE | | 100 | 192.168.200.202 | 3306 | ONLINE | | 1000 | 192.168.200.202 | 3306 | ONLINE | | 101 | 192.168.200.97 | 3306 | ONLINE | | 1001 | 192.168.200.245 | 3306 | ONLINE | | 1001 | 192.168.200.97 | 3306 | ONLINE | +--------------+-----------------+------+--------+ 6 rows in set (0.00 sec)
模擬app鏈接：
/Users/jinyizhou [15:32:09] ~$ mysql -uproxysql -pproxysql -h192.168.200.24 -P6033 -A ... proxysql@192.168.200.24 : (none) 03:32:11>show databases; +--------------------+ | Database | +--------------------+ | information_schema | | proxysql | +--------------------+ 2 rows in set (0.00 sec) proxysql@192.168.200.24 : (none) 03:32:13>use proxysql Database changed proxysql@192.168.200.24 : proxysql 03:32:17>show tables; +--------------------+ | Tables_in_proxysql | +--------------------+ | xx | +--------------------+ 1 row in set (0.00 sec) proxysql@192.168.200.24 : proxysql 03:32:24>insert into xx values(999); Query OK, 1 row affected (0.00 sec) proxysql@192.168.200.24 : proxysql 03:35:49>select * from xx; ERROR 1044 (#4200): Access denied for user 'proxysql'@'192.168.200.24' to database 'proxysql'
只有select的時候沒有權限，其餘insert，update等都是有權限的，爲啥呢？緣由是這裏的路由關係，ProxySQL的讀寫分離核心就是路由，這裏由於select的路由錯了，到了HG爲1000的主從上了：
admin@127.0.0.1 : (none) 03:32:28>select hostgroup,schemaname,username,digest_text from stats_mysql_query_digest; +-----------+--------------------+----------+----------------------------------+ | hostgroup | schemaname | username | digest_text | +-----------+--------------------+----------+----------------------------------+ | 1000 | proxysql | proxysql | select * from xx | | 101 | proxysql | proxysql | show tables | | 101 | information_schema | proxysql | show databases | | 1000 | information_schema | proxysql | SELECT DATABASE() | | 101 | information_schema | proxysql | select USER() | | 101 | information_schema | proxysql | select @@version_comment limit ? | +-----------+--------------------+----------+----------------------------------+
mysql_query_rules是整個ProxySQL的核心，上篇文章已經對該表進行了說明，在這裏對這例子再次講解下：
rule_id是表的自增主鍵，路由規則處理是以 rule_id 的順序進行匹配，若沒有找到規則就直接去mysql_users.default_hostgroup字段裏找。上面信息中除了select以外的其餘操做都找不到規則就直接去users表裏取，因此這些操做不會報錯。而咱們執行的select被rule_id爲4的規則匹配上，由於rule_id=4的是匹配全部庫而且apply=1表示該正則匹配後，將再也不接受其餘匹配，直接轉發。這樣就轉發到了HG爲1000上面的主機上了，就報沒有權限的錯誤。若apply=0則繼續匹配下面，若沒有找到路由規則，則返回再看flagOUT是否爲NULL，是NULL則直接匹配，不然報錯。大體的流程以下：
flagIN, flagOUT, apply: 用來定義路由鏈 chains of rules 首先會檢查 flagIN=0 的規則，以rule_id的順序；若是都沒匹配上，則走這個用戶的default_hostgroup 當匹配一條規則後，會檢查 apply是否爲1，是1則直接轉發，不是1則繼續匹配，匹配到就轉發，不然看flagOUT, 若是不爲NULL，而且 flagIN != flagOUT ，則進入以flagIN爲上一個flagOUT值的新規則鏈 若是不爲NULL，而且 flagIN = flagOUT，則應用這條規則 若是爲NULL，則結束，應用這條規則
經過上面的說明，如何讀取到正確的HG呢？這裏能夠設置apply=0
admin@127.0.0.1 : (none) 04:18:45>update mysql_query_rules set apply=0 where rule_id in (3,4); Query OK, 2 rows affected (0.00 sec) admin@127.0.0.1 : (none) 04:18:56>load mysql query rules to runtime; Query OK, 0 rows affected (0.00 sec) admin@127.0.0.1 : (none) 04:18:59>save mysql query rules to disk; Query OK, 0 rows affected (0.00 sec) admin@127.0.0.1 : (none) 04:19:01>select rule_id,active,username,schemaname,client_addr,destination_hostgroup,match_pattern,flagIN,flagOUT,apply from mysql_query_rules; +---------+--------+----------+------------+-------------+-----------------------+----------------------+--------+---------+-------+ | rule_id | active | username | schemaname | client_addr | destination_hostgroup | match_pattern | flagIN | flagOUT | apply | +---------+--------+----------+------------+-------------+-----------------------+----------------------+--------+---------+-------+ | 3 | 1 | NULL | NULL | NULL | 100 | ^SELECT.*FOR UPDATE$ | 0 | NULL | 0 | | 4 | 1 | NULL | NULL | NULL | 1000 | ^SELECT | 0 | NULL | 0 | | 5 | 1 | NULL | proxysql | NULL | 101 | ^SELECT.*FOR UPDATE$ | 0 | NULL | 1 | | 6 | 1 | NULL | proxysql | NULL | 1001 | ^SELECT | 0 | NULL | 1 | +---------+--------+----------+------------+-------------+-----------------------+----------------------+--------+---------+-------+ 4 rows in set (0.00 sec)
和上面同樣模擬app鏈接，獲得的信息：發現所有走了正確的路由。
admin@127.0.0.1 : (none) 05:58:55>select hostgroup,schemaname,username,digest_text from stats_mysql_query_digest; +-----------+------------+----------+----------------------------------+ | hostgroup | schemaname | username | digest_text | +-----------+------------+----------+----------------------------------+ | 101 | proxysql | proxysql | insert into xx values(?) | | 1001 | proxysql | proxysql | select * from xx | | 1001 | proxysql | proxysql | SELECT DATABASE() | | 101 | proxysql | proxysql | select USER() | | 101 | proxysql | proxysql | select @@version_comment limit ? | +-----------+------------+----------+----------------------------------+ 5 rows in set (0.00 sec)
查看路由規則的命中狀況：
admin@127.0.0.1 : (none) 05:59:19>select * from stats_mysql_query_rules; +---------+------+ | rule_id | hits | +---------+------+ | 3 | 0 | | 4 | 4 | | 5 | 0 | | 6 | 4 | +---------+------+ 4 rows in set (0.00 sec)
從上面看到，apply=0 & falgOUT=null，會繼續往下找路由，找到了rule_id=6的，直接進行轉發。apply=1 直接轉發，flagOUT != null 直接轉發。
小結：經過上面的測試說明ProxySQL只要設置好路由規則，能夠有多個主庫。
② 根據用戶名路由
和多主路由同樣，區別是寫入到路由表的字段不同：
admin@127.0.0.1 : (none) 06:09:20>INSERT INTO mysql_query_rules(active,username,match_pattern,destination_hostgroup,apply) VALUES(1,'proxysql','^SELECT.*FOR UPDATE$',101,1); Query OK, 1 row affected (0.00 sec) admin@127.0.0.1 : (none) 06:10:09>INSERT INTO mysql_query_rules(active,username,match_pattern,destination_hostgroup,apply) VALUES(1,'proxysql','^SELECT',1001,1); Query OK, 1 row affected (0.00 sec) admin@127.0.0.1 : (none) 06:10:32>select rule_id,active,username,schemaname,client_addr,destination_hostgroup,match_pattern,flagIN,flagOUT,apply from mysql_query_rules; +---------+--------+----------+------------+-------------+-----------------------+----------------------+--------+---------+-------+ | rule_id | active | username | schemaname | client_addr | destination_hostgroup | match_pattern | flagIN | flagOUT | apply | +---------+--------+----------+------------+-------------+-----------------------+----------------------+--------+---------+-------+ | 3 | 1 | NULL | NULL | NULL | 100 | ^SELECT.*FOR UPDATE$ | 0 | NULL | 1 | | 4 | 1 | NULL | NULL | NULL | 1000 | ^SELECT | 0 | NULL | 1 | | 1405 | 1 | proxysql | NULL | NULL | 101 | ^SELECT.*FOR UPDATE$ | 0 | NULL | 1 | | 1406 | 1 | proxysql | NULL | NULL | 1001 | ^SELECT | 0 | NULL | 1 | +---------+--------+----------+------------+-------------+-----------------------+----------------------+--------+---------+-------+ 4 rows in set (0.00 sec)
3，flagIN/flahOUT規則鏈實現多實例（推薦）
和2中的條件同樣，先配置ProxySQL的servers，users：
admin@127.0.0.1 : (none) 01:09:52>insert into mysql_servers(hostgroup_id,hostname,port,weight,max_replication_lag,comment) -> values -> (100, '192.168.200.202', 3306, 1, 10, 'ReadWrite'), -> (1000, '192.168.200.202', 3306, 1, 10, 'ReadWrite'), -> (1000, '192.168.200.132', 3306, 9, 10, 'ReadOnly'); Query OK, 3 rows affected (0.00 sec) admin@127.0.0.1 : (none) 01:09:54>insert into mysql_servers(hostgroup_id,hostname,port,weight,max_replication_lag,comment) -> values -> (101, '192.168.200.97', 3306, 1, 10, 'ReadWrite'), -> (1001, '192.168.200.97', 3306, 1, 10, 'ReadWrite'), -> (1001, '192.168.200.245', 3306, 9, 10, 'ReadOnly'); Query OK, 3 rows affected (0.01 sec) admin@127.0.0.1 : (none) 01:11:01>insert into mysql_users(username, password,active,default_hostgroup,transaction_persistent) -> values -> ('sbuser', 'sbuser', 1, 100, 1), -> ('proxysql', 'proxysql', 1, 101, 1); Query OK, 2 rows affected (0.00 sec) admin@127.0.0.1 : (none) 01:19:44>set mysql-monitor_username='proxysql'; Query OK, 1 row affected (0.00 sec) admin@127.0.0.1 : (none) 01:19:44>set mysql-monitor_password='proxysql'; Query OK, 1 row affected (0.00 sec) admin@127.0.0.1 : (none) 01:56:09>select hostgroup_id,hostname,port,status from mysql_servers; +--------------+-----------------+------+--------+ | hostgroup_id | hostname | port | status | +--------------+-----------------+------+--------+ | 100 | 192.168.200.202 | 3306 | ONLINE | | 1000 | 192.168.200.202 | 3306 | ONLINE | | 1000 | 192.168.200.132 | 3306 | ONLINE | | 101 | 192.168.200.97 | 3306 | ONLINE | | 1001 | 192.168.200.97 | 3306 | ONLINE | | 1001 | 192.168.200.245 | 3306 | ONLINE | +--------------+-----------------+------+--------+ 6 rows in set (0.00 sec) admin@127.0.0.1 : (none) 01:58:18>select username,default_hostgroup from mysql_users; +----------+-------------------+ | username | default_hostgroup | +----------+-------------------+ | sbuser | 100 | | proxysql | 101 | +----------+-------------------+
再配置flagOUT/flagIN，flag20是讀，flag21是寫：
admin@127.0.0.1 : (none) 01:21:34>INSERT INTO mysql_query_rules(rule_id,active,match_pattern,apply,flagOUT) VALUES(49,1,'^SELECT.*FOR UPDATE$',0,21); Query OK, 1 row affected (0.00 sec) admin@127.0.0.1 : (none) 01:27:18>INSERT INTO mysql_query_rules(rule_id,active,match_pattern,apply,flagOUT) VALUES(50,1,'^SELECT',0,20); Query OK, 1 row affected (0.00 sec) admin@127.0.0.1 : (none) 01:32:11>insert into mysql_query_rules(active,schemaname,destination_hostgroup,apply,flagIN,flagOUT) values -> (1,'sbtest',100,1,21,21), -> (1,'proxysql',101,1,21,21); Query OK, 2 rows affected (0.00 sec) admin@127.0.0.1 : (none) 01:32:53>insert into mysql_query_rules(active,schemaname,destination_hostgroup,apply,flagIN,flagOUT) values -> (1,'sbtest',1000,1,20,20), -> (1,'proxysql',1001,1,20,20); Query OK, 2 rows affected (0.00 sec) admin@127.0.0.1 : (none) 01:58:28>select rule_id,active,username,schemaname,client_addr,destination_hostgroup,match_pattern,flagIN,flagOUT,apply from mysql_query_rules; +---------+--------+----------+------------+-------------+-----------------------+----------------------+--------+---------+-------+ | rule_id | active | username | schemaname | client_addr | destination_hostgroup | match_pattern | flagIN | flagOUT | apply | +---------+--------+----------+------------+-------------+-----------------------+----------------------+--------+---------+-------+ | 49 | 1 | NULL | NULL | NULL | NULL | ^SELECT.*FOR UPDATE$ | 0 | 21 | 0 | | 50 | 1 | NULL | NULL | NULL | NULL | ^SELECT | 0 | 20 | 0 | | 51 | 1 | NULL | sbtest | NULL | 100 | NULL | 21 | 21 | 1 | | 52 | 1 | NULL | proxysql | NULL | 101 | NULL | 21 | 21 | 1 | | 53 | 1 | NULL | sbtest | NULL | 1000 | NULL | 20 | 20 | 1 | | 54 | 1 | NULL | proxysql | NULL | 1001 | NULL | 20 | 20 | 1 | +---------+--------+----------+------------+-------------+-----------------------+----------------------+--------+---------+-------+
最後保存上線：
-- 應用 load mysql users to runtime; load mysql servers to runtime; load mysql variables to runtime; LOAD MYSQL QUERY RULES TO RUN; -- 保存到磁盤 save mysql users to disk; save mysql servers to disk; save mysql variables to disk; SAVE MYSQL QUERY RULES TO DISK; save mysql users to mem; -- 能夠屏蔽看到的明文密碼
app鏈接測試：
1）鏈接實例202
[zhoujy@localhost ~]$ mysql -usbuser -psbuser -h192.168.200.24 -P6033 ... sbuser@192.168.200.24 : (none) 02:19:41>show databases; +--------------------+ | Database | +--------------------+ | information_schema | | sbtest | +--------------------+ 2 rows in set (0.00 sec) sbuser@192.168.200.24 : (none) 02:19:44>use sbtest Database changed sbuser@192.168.200.24 : sbtest 02:19:48>show tables; ... sbuser@192.168.200.24 : sbtest 02:19:57>insert into x values(10000); ... sbuser@192.168.200.24 : sbtest 02:20:10>select * from x; ...
相關信息：路由的信息都是正確的
admin@127.0.0.1 : (none) 02:24:15>select hostgroup,schemaname,username,digest_text,count_star from stats_mysql_query_digest; +-----------+--------------------+----------+----------------------------------+------------+ | hostgroup | schemaname | username | digest_text | count_star | +-----------+--------------------+----------+----------------------------------+------------+ | 1000 | sbtest | sbuser | select * from x | 5 | | 100 | sbtest | sbuser | insert into x values(?) | 5 | | 100 | sbtest | sbuser | show tables | 2 | | 100 | sbtest | sbuser | show databases | 1 | | 100 | information_schema | sbuser | SELECT DATABASE() | 1 | | 100 | information_schema | sbuser | show databases | 1 | | 100 | information_schema | sbuser | select USER() | 1 | | 100 | information_schema | sbuser | select @@version_comment limit ? | 1 | +-----------+--------------------+----------+----------------------------------+------------+ 8 rows in set (0.00 sec)
--路由命中 admin@127.0.0.1 : (none) 02:25:13>admin@1* from stats_mysql_query_rules; +---------+------+ | rule_id | hits | +---------+------+ | 49 | 0 | | 50 | 6 | | 51 | 0 | | 52 | 0 | | 53 | 5 | | 54 | 0 | +---------+------+ 6 rows in set (0.00 sec)
結論：經過實例202的帳號訪問ProxySQL，首先會檢查flagIN=0，在其上面進行匹配（Proxysql入口都是flagIN =0，順序往下）， 匹配到以後檢查flagOUT，發現 flagOUT不爲NULL且flagIN !（0）= flagOUT （20），則進入以flagIN爲上一個flagOUT值的新規則鏈，即20。再去flagIN=20裏匹配，最終匹配到了rule_id=53的規則，最後轉發。
2）鏈接實例97
相關狀況和上面同樣，最終經過rule_id=54進行轉發。
建議：若要用ProxySQL來控制多主從實例的讀寫分離，推薦使用flagIN/flahOUT規則鏈實現多實例。
4，flagIN/flahOUT規則鏈實現分庫
目的：客戶端應用鏈接上 proxysql 的ip:port，鏈接時指定分庫db名，自動路由到對應的實例、對應的庫。
① ：環境
APP：192.168.200.25、192.168.200.64 M1： IP：192.168.200.202 Port：3306 DB：M一、M二、M3 S1： IP：192.168.200.132 Port：3306 DB：M一、M二、M3 M2： IP：192.168.200.97 Port：3306 DB：M四、M五、M6 S2： IP：192.168.200.245 Port：3306 DB：M四、M五、M6 ProxySQL：192.168.200.24
② 搭建
和以前同樣先在後端數據庫建立程序和檢測帳號：
--程序帳號 GRANT SELECT,INSERT,UPDATE,DELETE ON `mtest%`.* TO 'mtest'@'192.168.200.24' identified by 'mtest'; --健康檢測帳號 GRANT SUPER, REPLICATION CLIENT ON *.* TO 'proxysql'@'192.168.200.24' IDENTIFIED BY 'proxysql';
配置ProxySQL：
--插入後端用戶信息 insert into mysql_users(username, password,active,transaction_persistent) values('mtest','mtest',1,1); --插入後端數據庫信息 insert into mysql_servers(hostgroup_id,hostname,port,weight,max_replication_lag,comment) values(100,'192.168.200.202',3306,1,10,'test proxysql'); insert into mysql_servers(hostgroup_id,hostname,port,weight,max_replication_lag,comment) values(1000,'192.168.200.132',3306,9,10,'test proxysql'); insert into mysql_servers(hostgroup_id,hostname,port,weight,max_replication_lag,comment) values(1000,'192.168.200.202',3306,1,10,'test proxysql'); insert into mysql_servers(hostgroup_id,hostname,port,weight,max_replication_lag,comment) values(101,'192.168.200.97',3306,1,10,'test proxysql'); insert into mysql_servers(hostgroup_id,hostname,port,weight,max_replication_lag,comment) values(1001,'192.168.200.245',3306,9,10,'test proxysql'); insert into mysql_servers(hostgroup_id,hostname,port,weight,max_replication_lag,comment) values(1001,'192.168.200.97',3306,1,10,'test proxysql'); --配置健康檢測信息 set mysql-monitor_username='proxysql'; set mysql-monitor_password='proxysql';
應用保存配置：
-- 應用 load mysql users to runtime; load mysql servers to runtime; load mysql variables to runtime; -- 保存到磁盤 save mysql users to disk; save mysql servers to disk; save mysql variables to disk; save mysql users to mem; -- 能夠屏蔽看到的明文密碼
配置路由規則：
----添加讀寫分離的路由 --寫:寫的入口 flagIN=0 INSERT INTO mysql_query_rules(rule_id,active,match_pattern,apply,flagOUT) VALUES(49,1,'^SELECT.*FOR UPDATE$',0,21); --讀：讀的入口 flagIN=0 INSERT INTO mysql_query_rules(rule_id,active,match_pattern,apply,flagOUT) VALUES(50,1,'^SELECT',0,20); --反向匹配，至關於對 match_digest/match_pattern 的匹配取反，非select，即寫 INSERT INTO mysql_query_rules(rule_id, active,match_pattern,negate_match_pattern,apply,flagOUT) values(60, 1,'^SELECT',1,0,21); ----爲後端服務器配置路由 --讀 insert into mysql_query_rules(active,schemaname,destination_hostgroup,apply,flagIN,flagOUT) values(1,'M1',1000,1,20,20),(1,'M2',1000,1,20,20),(1,'M3',1000,1,20,20); insert into mysql_query_rules(active,schemaname,destination_hostgroup,apply,flagIN,flagOUT) values(1,'M4',1001,1,20,20),(1,'M5',1001,1,20,20),(1,'M6',1001,1,20,20); --寫 insert into mysql_query_rules(active,schemaname,destination_hostgroup,apply,flagIN,flagOUT) values(1,'M1',100,1,21,21),(1,'M2',100,1,21,21),(1,'M3',100,1,21,21); insert into mysql_query_rules(active,schemaname,destination_hostgroup,apply,flagIN,flagOUT) values(1,'M4',101,1,21,21),(1,'M5',101,1,21,21),(1,'M6',101,1,21,21); --鏈接時，若沒有指定數據庫，則進行show databases/tables、use 等會超時出錯，鏈接時，默認的數據庫是在information_schema，因此寫一條根據information_schema庫的路由,直接返回錯誤信息。 insert into mysql_query_rules(rule_id,active,schemaname,apply,flagOUT) values(20,1,'information_schema',0,302); insert into mysql_query_rules(rule_id,active,apply, flagIN,flagOUT,error_msg) values(9999,1,1, 302,302,'No query rules matched (by ProxySQL)'); --鏈接時，若沒有指定數據庫，則可使用 schemaname.tablename 的形式匹配數據。 insert into mysql_query_rules(rule_id,active,match_pattern,destination_hostgroup,apply,flagIN,flagOUT) values(1000,1,'([\s\`])M(1|2|3)([\.\`])',100,1,302,302); insert into mysql_query_rules(rule_id,active,match_pattern,destination_hostgroup,apply,flagIN,flagOUT) values(1001,1,'([\s\`])M(4|5|6)([\.\`])',101,1,302,302);
應用規則：
LOAD MYSQL QUERY RULES TO RUN; SAVE MYSQL QUERY RULES TO DISK;
最終的路由規則以下：
select rule_id,schemaname,destination_hostgroup,match_pattern,negate_match_pattern,flagIN,flagOUT,apply,error_msg from mysql_query_rules;
app鏈接測試：
~$ mysql -umtest -pmtest -h192.168.200.24 -P6033 -A ... mtest@192.168.200.24 : (none) 11:27:29>show databases; --觸發了定義的路由 ERROR 1148 (42000): No query rules matched (by ProxySQL) mtest@192.168.200.24 : (none) 11:27:34>select * from M5.mtest5; --能夠直接用schema.tables 訪問 +------+ | id | +------+ | 5 | | 55 | +------+ 2 rows in set (0.00 sec) mtest@192.168.200.24 : (none) 11:27:47>use M1 --切換數據庫 Database changed mtest@192.168.200.24 : M1 11:27:52>show tables; --能夠show了 +--------------+ | Tables_in_M1 | +--------------+ | mtest1 | +--------------+ 1 row in set (0.00 sec) mtest@192.168.200.24 : M1 11:27:56>select * from mtest1; +------+ | id | +------+ | 1 | | 11 | | 111 | | 1111 | +------+ 4 rows in set (0.00 sec) mtest@192.168.200.24 : M1 11:28:02>insert into mtest1 values(11111); Query OK, 1 row affected (0.00 sec) mtest@192.168.200.24 : M1 11:28:11>select * from mtest1; +-------+ | id | +-------+ | 1 | | 11 | | 111 | | 1111 | | 11111 | +-------+ 5 rows in set (0.00 sec) mtest@192.168.200.24 : M1 11:28:12>show databases; --能夠show了 +--------------------+ | Database | +--------------------+ | information_schema | | M1 | | M2 | | M3 | +--------------------+ 4 rows in set (0.00 sec) mtest@192.168.200.24 : M1 11:28:20>use M5 --切換到另外一個實例的db Database changed mtest@192.168.200.24 : M5 11:28:52>show databases; +--------------------+ | Database | +--------------------+ | information_schema | | M4 | | M5 | | M6 | +--------------------+ 4 rows in set (0.00 sec) mtest@192.168.200.24 : M5 11:28:55>select * from mtest5; +------+ | id | +------+ | 5 | | 55 | +------+ 2 rows in set (0.00 sec) mtest@192.168.200.24 : M5 11:29:03>insert into mtest5 values(555); Query OK, 1 row affected (0.01 sec) mtest@192.168.200.24 : M5 11:29:12>select * from mtest5; +------+ | id | +------+ | 5 | | 55 | | 555 | +------+ 3 rows in set (0.00 sec)
查看路由命中率：
select active,hits, mysql_query_rules.rule_id, schemaname,match_pattern,destination_hostgroup hostgroup,flagIn,flagOUT FROM mysql_query_rules NATURAL JOIN stats.stats_mysql_query_rules ORDER BY mysql_query_rules.rule_id;
查看SQL統計信息：
admin@127.0.0.1 : (none) 11:36:46>select hostgroup,schemaname,username,substr(digest_text,120,-120),count_star from stats_mysql_query_digest; +-----------+--------------------+----------+----------------------------------+------------+ | hostgroup | schemaname | username | substr(digest_text,120,-120) | count_star | +-----------+--------------------+----------+----------------------------------+------------+ | 101 | M5 | mtest | show databases | 1 | | 1000 | M1 | mtest | SELECT DATABASE() | 1 | | 101 | M5 | mtest | insert into mtest5 values(?) | 1 | | 100 | M1 | mtest | show databases | 1 | | 100 | M1 | mtest | insert into mtest1 values(?) | 1 | | 1000 | M1 | mtest | select * from mtest1 | 2 | | 1001 | M5 | mtest | select * from mtest5 | 2 | | 100 | M1 | mtest | show tables | 1 | | 101 | information_schema | mtest | select * from M5.mtest5 | 1 | | 0 | information_schema | mtest | show databases | 1 | | 0 | information_schema | mtest | SELECT DATABASE() | 1 | | 0 | information_schema | mtest | select USER() | 1 | | 0 | information_schema | mtest | select @@version_comment limit ? | 1 | +-----------+--------------------+----------+----------------------------------+------------+
具體的說明能夠看ProxySQL之讀寫分離與分庫路由演示，到此讀寫分離的測試介紹完畢，
5，查詢重寫
查詢重寫這種對於線上環境SQL問題引發的緊急故障處理仍是頗有用處的。若是定位到了問題所在，必須修改SQL，時間緊急，這時查詢重寫這個東西就很是有用了。相似於MySQL5.7的查詢重寫插件。這裏作下相關的說明：
ProxySQL的核心就是路由，查詢重寫也只是添加一條路由而已，在4的基礎上進行測試：
select * from mtest1 order by id 重寫成 select * from mtest1
添加路由：
--查詢的路由，flagIN=0，當匹配上規則後進行重寫，而且不該用，而經過flagOUT下去繼續查詢 INSERT INTO mysql_query_rules (rule_id,active,match_pattern,replace_pattern,apply,flagOUT) VALUES (48,1,'(.*)order by id','\1',0,20);
其實查詢重寫的實如今proxysql中也實現爲正則匹配替換，表示當proxysql匹配到<若干字符>order by id這個模式後，就將這個模式的order by id去掉。那麼\1是什麼意思呢，就是sed的向前引用。
加載load和save完rules以後，查看是否重寫成功：
--初始 #查詢路由命中信息 admin@127.0.0.1 : (none) 02:44:52>select * from stats_mysql_query_rules; +---------+------+ | rule_id | hits | +---------+------+ | 20 | 0 | | 48 | 0 | | 49 | 0 | | 50 | 0 | | 60 | 0 | | 61 | 0 | | 62 | 0 | | 63 | 0 | | 64 | 0 | | 65 | 0 | | 66 | 0 | | 67 | 0 | | 68 | 0 | | 69 | 0 | | 70 | 0 | | 71 | 0 | | 72 | 0 | | 1000 | 0 | | 1001 | 0 | | 9999 | 0 | +---------+------+ 20 rows in set (0.00 sec) #查詢統計信息 admin@127.0.0.1 : (none) 02:45:09>select * from stats_mysql_query_digest; Empty set (0.00 sec) --操做 ~$ mysql -umtest -pmtest -h192.168.200.24 -P6033 -A ... mtest@192.168.200.24 : (none) 02:45:27>use M1 Database changed mtest@192.168.200.24 : M1 02:45:31>show tables; +--------------+ | Tables_in_M1 | +--------------+ | mtest1 | +--------------+ 1 row in set (0.00 sec) mtest@192.168.200.24 : M1 02:45:33>select * from mtest1; +-------+ | id | +-------+ | 1 | | 11 | | 111 | | 1111 | | 11111 | +-------+ 5 rows in set (0.00 sec) mtest@192.168.200.24 : M1 02:45:37>select * from mtest1 order by id; +-------+ | id | +-------+ | 1 | | 11 | | 111 | | 1111 | | 11111 | +-------+ 5 rows in set (0.00 sec) mtest@192.168.200.24 : M1 02:45:46>select * from mtest1 order by id; +-------+ | id | +-------+ | 1 | | 11 | | 111 | | 1111 | | 11111 | +-------+ 5 rows in set (0.01 sec) ----以上執行了2次order by id和1此沒有order by id的查詢，去查詢統計應該獲得的值是3次沒有order by id的查詢。 --驗證 #查詢統計信息，查看沒有order by id的SQL出現了3次，沒有出現有order by id的SQL admin@127.0.0.1 : (none) 02:49:49>select hostgroup,schemaname,digest_text,count_star from stats_mysql_query_digest; +-----------+--------------------+----------------------------------+------------+ | hostgroup | schemaname | digest_text | count_star | +-----------+--------------------+----------------------------------+------------+ | 1000 | M1 | select * from mtest1 | 3 | | 100 | M1 | show tables | 1 | | 0 | information_schema | SELECT DATABASE() | 1 | | 0 | information_schema | select USER() | 1 | | 0 | information_schema | select @@version_comment limit ? | 1 | +-----------+--------------------+----------------------------------+------------+ 5 rows in set (0.00 sec) #重寫查詢的路由命中了2次 admin@127.0.0.1 : (none) 02:50:12>select * from stats_mysql_query_rules; +---------+------+ | rule_id | hits | +---------+------+ | 20 | 1 | | 48 | 2 | | 49 | 0 | | 50 | 1 | | 60 | 1 | | 61 | 3 | | 62 | 0 | | 63 | 0 | | 64 | 0 | | 65 | 0 | | 66 | 0 | | 67 | 1 | | 68 | 0 | | 69 | 0 | | 70 | 0 | | 71 | 0 | | 72 | 0 | | 1000 | 0 | | 1001 | 0 | | 9999 | 1 | +---------+------+ 20 rows in set (0.00 sec)
從上面的結果看，查詢重寫已經測試經過。到此，關於ProxySQL的相關測試已經結束，下面分析下和DBProxy的特性差異和性能差別。
性能測試
環境：
ProxySQL：192.168.200.24 DBProxy ：192.168.200.24 M： IP：192.168.200.202 Port：3306 DB：sbtest S： IP：192.168.200.132 Port：3306 DB：sbtest
讀寫混合(oltp_read_write.lua)測試對比：
直連數據庫：
./bin/sysbench --test=./share/sysbench/oltp_read_write.lua --mysql-host=192.168.200.202 --mysql-port=3306 --mysql-user=sbuser --mysql-password=sbuser --mysql-db=sbtest --report-interval=10 --max-requests=0 --time=120 --threads=1 --tables=3 --table-size=1000000 prepare/run/cleanup
ProxySQL鏈接數據庫：
./bin/sysbench --test=./share/sysbench/oltp_read_write.lua --mysql-host=192.168.200.24 --mysql-port=6033 --mysql-user=sbuser --mysql-password=sbuser --mysql-db=sbtest --report-interval=10 --max-requests=0 --time=120 --threads=1 --tables=3 --table-size=1000000 --skip-trx=on --db-ps-mode=disable --mysql-ignore-errors=1062 prepare/run/cleanup
DBProxy鏈接數據庫
./bin/sysbench --test=./share/sysbench/oltp_read_write.lua --mysql-host=192.168.200.24 --mysql-port=3308 --mysql-user=sbuser --mysql-password=sbuser --mysql-db=sbtest --report-interval=10 --max-requests=0 --time=120 --threads=1 --tables=3 --table-size=1000000 --skip-trx=on --db-ps-mode=disable --mysql-ignore-errors=1062 prepare/run/cleanup
測試的線程：一、二、四、八、1六、3二、6四、128
把上面數據以曲線圖的形式表現：
TPS：
QPS：
測試小結：
在讀寫混合的模式下：線程越少差距越大，測試結果和美團點評DBProxy讀寫分離使用說明裏的測試報告基本吻合，這裏主要對比ProxySQL和DBProxy的性能狀況，從上圖看到兩者性能差很少，不過DBProxy的CPU消耗是ProxySQL的1到1.5倍。
總結：
經過上面的一些基本介紹，大體瞭解了ProxySQL讀寫分離功能的使用，關於ProxySQL的其餘功能內容在手冊裏有了詳盡的介紹，具體的狀況請參考手冊說明。如今大體整理下ProxySQL和DBproxy的差異:
①：鏈接池，是 multiplexing。
②：強大的正則路由，能夠本身干預讀寫路由算法。
③：從庫不可用自動下線，不須要人爲干預，支持多主庫。
④：支持重寫SQL。
⑤：足夠輕量，配置簡單。
可是在安全配置上面，DBProxy比ProxySQL要強，ProxySQL先後端帳號未分離，能夠經過mysql_users查看，先後端公用一個帳號，可是在runting_mysql_users裏面先後端帳號是分離的（backend、frontend）。其餘的相關安全能夠參考美團點評DBProxy讀寫分離使用說明。最後根據狀況選擇到底使用哪一個proxy，要是使用的是MySQL Server 5.7，因DBProxy沒有對5.7進行測試，因此推薦使用ProxySQL。至於如何防止ProxySQL的單點問題，也能夠用lvs來解決，具體的說能夠看LVS+Keepalived實現DBProxy的高可用。
參考文檔
https://github.com/sysown/proxysql/wiki
http://proxysql.blogspot.jp/2015/09/proxysql-tutorial-setup-in-mysql.html
https://severalnines.com/blog/how-set-read-write-split-galera-cluster-using-proxysql
相關文章
- 1. MySQL ProxySQL讀寫分離實踐
- 2. ProxySQL實現讀寫分離
- 3. ProxySQL讀寫分離
- 4. Mysql使用ProxySQL實現讀寫分離
- 5. ProxySQL實現MySQL讀寫分離
- 6. ProxySQL（讀寫分離）部署
- 7. Proxysql讀寫分離配置
- 8. mysql中間件proxysql實現mysql讀寫分離
- 9. Mysql中間件Proxysql實現mysql主從架構讀寫分離
- 10. ProxySQL+Mysql實現數據庫讀寫分離實戰
- 更多相關文章...
- • Thymeleaf項目實踐 - Thymeleaf 教程
- • SQLite 分離數據庫 - SQLite教程
- • 漫談MySQL的鎖機制
- • Docker容器實戰(六) - 容器的隔離與限制
相關標籤/搜索
每日一句
-
每一个你不满意的现在，都有一个你没有努力的曾经。
最新文章
- 1. Duang!超快Wi-Fi來襲
- 2. 機器學習-補充03 神經網絡之**函數(Activation Function)
- 3. git上開源maven項目部署 多module maven項目（多module maven+redis+tomcat+mysql）後臺部署流程學習記錄
- 4. ecliple-tomcat部署maven項目方式之一
- 5. eclipse新導入的項目經常可以看到「XX cannot be resolved to a type」的報錯信息
- 6. Spark RDD的依賴於DAG的工作原理
- 7. VMware安裝CentOS-8教程詳解
- 8. YDOOK：Java 項目 Spring 項目導入基本四大 jar 包 導入依賴，怎樣在 IDEA 的項目結構中導入 jar 包 導入依賴
- 9. 簡單方法使得putty（windows10上）可以免密登錄樹莓派
- 10. idea怎麼用本地maven
歡迎關注本站公眾號,獲取更多信息
