ProxySQL官檔翻譯__20_Mirroring
20_Mirroring
備註：文章編寫時間201904-201905期間，後續官方在github的更新沒有被寫入mysql
~
~
鏡像[Mirroring]
注意：
1）它包含的規則能夠隨時改變；
2）它不支持預備(prepare)語句；git
1、mysql_query_rules的擴展[Extensions to mysql_query_rules]
修改了 mysql_query_rules 表，添加了2個列：
1） mirror_flagOUT
2） mirror_hostgroupgithub
所以，mysql_query_rules 表的新的定義變爲：正則表達式
Admin> show create table mysql_query_rules\G
*************************** 1. row ***************************
table: mysql_query_rules
Create Table: CREATE TABLE mysql_query_rules (
rule_id INTEGER PRIMARY KEY AUTOINCREMENT NOT NULL,
active INT CHECK (active IN (0,1)) NOT NULL DEFAULT 0,
username VARCHAR,
schemaname VARCHAR,
flagIN INT CHECK (flagIN >= 0) NOT NULL DEFAULT 0,
client_addr VARCHAR,
proxy_addr VARCHAR,
proxy_port INT CHECK (proxy_port >= 0 AND proxy_port <= 65535), digest VARCHAR,
match_digest VARCHAR,
match_pattern VARCHAR,
negate_match_pattern INT CHECK (negate_match_pattern IN (0,1)) NOT NULL DEFAULT 0,
re_modifiers VARCHAR DEFAULT 'CASELESS',
flagOUT INT CHECK (flagOUT >= 0),
replace_pattern VARCHAR CHECK(CASE WHEN replace_pattern IS NULL THEN 1 WHEN replace_pattern IS NOT NULL AND match_pattern IS NOT NULL THEN 1 ELSE 0 END),
destination_hostgroup INT DEFAULT NULL,
cache_ttl INT CHECK(cache_ttl > 0),
cache_empty_result INT CHECK (cache_empty_result IN (0,1)) DEFAULT NULL,
cache_timeout INT CHECK(cache_timeout >= 0),
reconnect INT CHECK (reconnect IN (0,1)) DEFAULT NULL,
timeout INT UNSIGNED CHECK (timeout >= 0),
retries INT CHECK (retries>=0 AND retries <=1000),
delay INT UNSIGNED CHECK (delay >=0),
next_query_flagIN INT UNSIGNED,
mirror_flagOUT INT UNSIGNED,
mirror_hostgroup INT UNSIGNED,
error_msg VARCHAR,
OK_msg VARCHAR,
sticky_conn INT CHECK (sticky_conn IN (0,1)),
multiplex INT CHECK (multiplex IN (0,1,2)),
gtid_from_hostgroup INT UNSIGNED,
log INT CHECK (log IN (0,1)),
apply INT CHECK(apply IN (0,1)) NOT NULL DEFAULT 0,
comment VARCHAR)
1 row in set (0.00 sec)
2、功能實現概述[Implementation overview]
當爲匹配查詢的規則設置了 mirror_flagOUT 或 mirror_hostgroup 時，將自動啓用鏡像實時查詢功能。sql
請注意：
若是在規則中設置了 mirror_flagOUT 或 mirror_hostgroup 時又設置了replace_pattern：也就是若是原始查詢的文本內容(SQL語句)已被重寫(即查詢規則設置了replace_pattern對原始SQL進行重寫/替換)，則會爲最終執行的查詢(被修改後的語句)啓用鏡像查詢功能，而會爲原始SQL啓用鏡像查詢：即若是查詢(SQL語句)在根據digest、match_digest或match_pattern匹配到了查詢規則，卻發現該查詢規則設置了replace_pattern對匹配到的查詢SQL進行重寫，那麼，這時鏡像邏輯將應用於被重寫後的查詢(SQL)。雖然被鏡像的查詢(即原始SQL語句)能夠被從新編寫或修改，但只要設置了mirror_flagOUT 或 mirror_hostgroup就會啓用鏡像查詢功能。詳情在後面。服務器
若是源查詢(SQL)與多個查詢規則匹配，則可能會屢次更改 mirror_flagOUT 或 mirror_hostgroup 。app
鏡像邏輯以下：
1）若是在處理源查詢時設置了 mirror_flagOUT 或 mirror_hostgroup (查詢規則設置的)，則會建立一個新的mysql會話。
2）新的mysql會話將得到原始mysql會話的全部相同屬性：相同的憑據，庫名，默認主機組等（注意：charset當前未被複制）
3）若是在原始會話中設置了 mirror_hostgroup ，則新會話將其默認主機組更改成 mirror_hostgroup 。
4）若是未設置 mirror_flagOUT ，則新會話將針對定義的 mirror_hostgroup 執行原始查詢。
5）若是在原始會話中設置了 mirror_flagOUT ，則新的mysql會話將嘗試根據原始會話的mirror_flagOUT的值在 mysql_query_rules 中查找FlagIN值與之相等的查詢規則(即查找FlagIN=mirror_flagOUT的規則)； 而後將鏡像查詢請求發送到這個規則中進行處理：這樣就能夠修改查詢，如重寫查詢，或再次更改 hostgroup 。
(參考後面的<7、高級示例：使用鏡像測試查詢重寫>內容)dom
3、案例一：原始查詢與鏡像查詢選擇同一主機組[Mirror selects to same hostgroup]
在這個很是簡單的示例中，咱們將把全部SELECT語句發送到hostgroup10，包括原始語句和鏡像語句。ide
一、設置查詢規則
1）設置查詢規則
將原始查詢與鏡像查詢指向同一主機組：性能
Admin> SELECT rule_id,active,match_pattern,destination_hostgroup,mirror_hostgroup,apply FROM mysql_query_rules ; +---------+--------+---------------+-----------------------+------------------+-------+ | rule_id | active | match_pattern | destination_hostgroup | mirror_hostgroup | apply | +---------+--------+---------------+-----------------------+------------------+-------+ | 5 | 1 | NULL | NULL | NULL | 1 | +---------+--------+---------------+-----------------------+------------------+-------+ 1 row in set (0.00 sec) Admin> INSERT INTO mysql_query_rules (rule_id,active,match_pattern,destination_hostgroup,mirror_hostgroup,apply) VALUES (6,1,'^SELECT',10,10,1); Query OK, 1 row affected (0.00 sec) Admin> SELECT rule_id,active,match_pattern,destination_hostgroup,mirror_hostgroup,apply FROM mysql_query_rules ; +---------+--------+---------------+-----------------------+------------------+-------+ | rule_id | active | match_pattern | destination_hostgroup | mirror_hostgroup | apply | +---------+--------+---------------+-----------------------+------------------+-------+ | 5 | 1 | NULL | NULL | NULL | 1 | | 6 | 1 | ^SELECT | 10 | 10 | 1 | +---------+--------+---------------+-----------------------+------------------+-------+ 2 rows in set (0.00 sec)
2）加載配置到RUNTIME層
Admin> LOAD MYSQL QUERY RULES TO RUNTIME; Query OK, 0 rows affected (0.00 sec) Admin> SELECT rule_id,active,match_pattern,destination_hostgroup,mirror_hostgroup,apply FROM runtime_mysql_query_rules ; +---------+--------+---------------+-----------------------+------------------+-------+ | rule_id | active | match_pattern | destination_hostgroup | mirror_hostgroup | apply | +---------+--------+---------------+-----------------------+------------------+-------+ | 5 | 1 | NULL | NULL | NULL | 1 | | 6 | 1 | ^SELECT | 10 | 10 | 1 | +---------+--------+---------------+-----------------------+------------------+-------+ 2 rows in set (0.00 sec)
二、鏈接ProxySQL執行查詢
從mysql會話中咱們將運行一些查詢：
1）使用業務帳號連入ProxySQL
# mysql -h 188.188.0.71 -P 6033 -umsandbox -p mysql> use sbtest; mysql> show tables; +------------------+ | Tables_in_sbtest | +------------------+ | sbtest1 | | sbtest2 | | sbtest3 | | sbtest4 | | sbtest5 | +------------------+ 5 rows in set (0.00 sec)
2）ProxySQL管理端清空並查看當前以執行SQL的查詢概要統計：
Admin> SELECT hostgroup,count_star,schemaname,digest_text FROM stats_mysql_query_digest_reset WHERE schemaname='sbtest' ORDER BY digest; +-----------+------------+------------+--------------------------------+ | hostgroup | count_star | schemaname | digest_text | +-----------+------------+------------+--------------------------------+ | 10 | 1 | sbtest | show databases | | 10 | 1 | sbtest | show tables | +-----------+------------+------------+--------------------------------+ 2 rows in set (0.00 sec)
3）業務帳號執行查詢
mysql> SELECT id FROM sbtest1 LIMIT 3; +------+ | id | +------+ | 1891 | | 1511 | | 8032 | +------+ 3 rows in set (0.00 sec)
4）ProxySQL管理端，查看當前以執行SQL的查詢概要統計：
Admin> SELECT hostgroup,count_star,schemaname,digest_text FROM stats_mysql_query_digest WHERE schemaname='sbtest' ORDER BY digest; +-----------+------------+------------+--------------------------------+ | hostgroup | count_star | schemaname | digest_text | +-----------+------------+------------+--------------------------------+ | 10 | 2 | sbtest | SELECT id FROM sbtest1 LIMIT ? | +-----------+------------+------------+--------------------------------+ 1 row in set (0.00 sec)
咱們能夠看到 SELECT 語句被執行了兩次！！
5）做爲附加測試，咱們從新運行相同的查詢：
mysql> SELECT id FROM sbtest1 LIMIT 3; +------+ | id | +------+ | 1891 | | 1511 | | 8032 | +------+ 3 rows in set (0.00 sec
6）ProxySQL管理端，查看當前以執行SQL的查詢概要統計：
Admin> SELECT hostgroup,count_star,schemaname,digest_text FROM stats_mysql_query_digest WHERE schemaname='sbtest' ORDER BY digest; +-----------+------------+------------+--------------------------------+ | hostgroup | count_star | schemaname | digest_text | +-----------+------------+------------+--------------------------------+ | 10 | 4 | sbtest | SELECT id FROM sbtest1 LIMIT ? | +-----------+------------+------------+--------------------------------+ 1 row in set (0.01 sec)
count_star是咱們執行查詢次數的兩倍，由於它是鏡像的。值得注意的是，ProxySQL會收集原始查詢和鏡像查詢的指標。
4、案例二：原始查詢與鏡像查詢選擇到不一樣的主機組[Mirror SELECT to different hostgroup]
在此示例中，咱們將從新配置proxysql以將全部SELECT語句發送到hostgroup10，但要在hostgroup20上執行鏡像查詢：
一、設置查詢規則
1）設置查詢規則
Admin> DELETE FROM mysql_query_rules; Query OK, 2 rows affected (0.00 sec) Admin> INSERT INTO mysql_query_rules (rule_id,active,match_pattern,destination_hostgroup,mirror_hostgroup,apply) VALUES (5,1,'^SELECT',10,20,1); Query OK, 1 row affected (0.00 sec) Admin> SELECT rule_id,active,match_pattern,destination_hostgroup,mirror_hostgroup,apply FROM mysql_query_rules ; +---------+--------+---------------+-----------------------+------------------+-------+ | rule_id | active | match_pattern | destination_hostgroup | mirror_hostgroup | apply | +---------+--------+---------------+-----------------------+------------------+-------+ | 5 | 1 | ^SELECT | 10 | 20 | 1 | +---------+--------+---------------+-----------------------+------------------+-------+ 1 row in set (0.00 sec)
2）加載配置到RUNTIME層
Admin> LOAD MYSQL QUERY RULES TO RUNTIME; Query OK, 0 rows affected (0.00 sec)
二、鏈接ProxySQL執行查詢
1）使用業務帳號連入ProxySQL
# mysql -h 188.188.0.71 -P 6033 -umsandbox -p mysql> show databases; +--------------------+ | Database | +--------------------+ | information_schema | | sbtest | +--------------------+ 2 rows in set (0.00 sec) mysql> use sbtest; mysql> show tables; +------------------+ | Tables_in_sbtest | +------------------+ | sbtest1 | | sbtest2 | | sbtest3 | | sbtest4 | | sbtest5 | +------------------+ 5 rows in set (0.00 sec)
2）ProxySQL管理端清空stats_mysql_query_digest以得到新的統計信息：
Admin> SELECT hostgroup,count_star,schemaname,digest_text FROM stats_mysql_query_digest_reset WHERE schemaname='sbtest' ORDER BY digest; +-----------+------------+------------+----------------+ | hostgroup | count_star | schemaname | digest_text | +-----------+------------+------------+----------------+ | 10 | 1 | sbtest | show databases | | 10 | 1 | sbtest | show tabels | | 10 | 2 | sbtest | show tables | +-----------+------------+------------+----------------+ 3 rows in set (0.00 sec) Admin> SELECT hostgroup,count_star,schemaname,digest_text FROM stats_mysql_query_digest WHERE schemaname='sbtest' ORDER BY digest; Empty set (0.00 sec)
3）業務帳號執行查詢
從mysql客戶端咱們如今能夠運行一些查詢（爲簡單起見，咱們運行相同）：
mysql> SELECT id FROM sbtest1 LIMIT 3; +------+ | id | +------+ | 1891 | | 1511 | | 8032 | +------+ 3 rows in set (0.00 sec)
4）ProxySQL管理端，查看當前以執行SQL的查詢概要統計：
Admin> SELECT hostgroup,count_star,schemaname,digest_text FROM stats_mysql_query_digest WHERE schemaname='sbtest' ORDER BY digest; +-----------+------------+------------+--------------------------------+ | hostgroup | count_star | schemaname | digest_text | +-----------+------------+------------+--------------------------------+ | 20 | 1 | sbtest | SELECT id FROM sbtest1 LIMIT ? | | 10 | 1 | sbtest | SELECT id FROM sbtest1 LIMIT ? | +-----------+------------+------------+--------------------------------+ 2 rows in set (0.00 sec)
能夠看到ProxySQL向hostgroup10和hostgroup20發送了相同的相同查詢！
5、案例三：重寫源查詢和鏡像[rewrite both source query and mirror]
在這個例子中，咱們將重寫原始查詢，而後鏡像它：爲簡單起見，咱們將對錶 sbtest[0-9]+ 的操做重寫爲對錶 sbtest3 的操做：
一、設置查詢規則
1）設置查詢規則
Admin> DELETE FROM mysql_query_rules; Query OK, 1 row affected (0.00 sec) Admin> INSERT INTO mysql_query_rules (rule_id,active,match_pattern,destination_hostgroup,replace_pattern,mirror_hostgroup,apply) VALUES (5,1,'sbtest[0-9]+',10,'sbtest3',20,1); Query OK, 1 row affected (0.00 sec) Admin> SELECT rule_id,active,match_pattern,destination_hostgroup,replace_pattern,mirror_hostgroup,apply FROM mysql_query_rules ; +---------+--------+---------------+-----------------------+-----------------+------------------+-------+ | rule_id | active | match_pattern | destination_hostgroup | replace_pattern | mirror_hostgroup | apply | +---------+--------+---------------+-----------------------+-----------------+------------------+-------+ | 5 | 1 | sbtest[0-9]+ | 10 | sbtest3 | 20 | 1 | +---------+--------+---------------+-----------------------+-----------------+------------------+-------+ 1 row in set (0.00 sec)
2）加載配置到RUNTIME層
Admin> LOAD MYSQL QUERY RULES TO RUNTIME; Query OK, 0 rows affected (0.00 sec)
二、鏈接ProxySQL執行查詢
1）使用業務帳號連入ProxySQL
# mysql -h 188.188.0.71 -P 6033 -umsandbox -p mysql> show databases; +--------------------+ | Database | +--------------------+ | information_schema | | sbtest | +--------------------+ 2 rows in set (0.00 sec) mysql> use sbtest; mysql> show tables; +------------------+ | Tables_in_sbtest | +------------------+ | sbtest1 | | sbtest2 | | sbtest3 | | sbtest4 | | sbtest5 | +------------------+ 5 rows in set (0.00 sec)
2）ProxySQL管理端清空stats_mysql_query_digest以得到新的統計信息：
Admin> SELECT hostgroup,count_star,schemaname,digest_text FROM stats_mysql_query_digest_reset WHERE schemaname='sbtest' ORDER BY digest; +-----------+------------+------------+----------------+ | hostgroup | count_star | schemaname | digest_text | +-----------+------------+------------+----------------+ | 10 | 1 | sbtest | show databases | | 10 | 2 | sbtest | show tables | +-----------+------------+------------+----------------+ 2 rows in set (0.00 sec) Admin> SELECT hostgroup,count_star,schemaname,digest_text FROM stats_mysql_query_digest WHERE schemaname='sbtest' ORDER BY digest; Empty set (0.01 sec)
3）業務帳號執行查詢
從mysql客戶端咱們能夠運行一般的查詢：
mysql> SELECT id FROM sbtest1 LIMIT 3; +------+ | id | +------+ | 1149 | | 9825 | | 5704 | +------+ 3 rows in set (0.00 sec)
正如預期的那樣，此次查詢輸出的結果與前一個不一樣，由於如今重寫了原始查詢。表名是是查詢了sbtest1其實被改寫到了sbtest3;經過下面內容能夠驗證。
4）ProxySQL管理端，查看當前以執行SQL的查詢概要統計：
Admin> SELECT hostgroup,count_star,schemaname,digest_text FROM stats_mysql_query_digest WHERE schemaname='sbtest' ORDER BY digest; +-----------+------------+------------+--------------------------------+ | hostgroup | count_star | schemaname | digest_text | +-----------+------------+------------+--------------------------------+ | 20 | 1 | sbtest | SELECT id FROM sbtest3 LIMIT ? | | 10 | 1 | sbtest | SELECT id FROM sbtest3 LIMIT ? | +-----------+------------+------------+--------------------------------+ 2 rows in set (0.01 sec)
正如所料，修改後的查詢在兩個主機組上執行，並且原始SQL中的表名sbtest1被重寫爲了sbtest3。
6、僅重寫鏡像查詢[rewrite mirror query only]
在此示例中，咱們將只重寫鏡像查詢。
這很是有用，例如，咱們想要了解重寫查詢的性能，或者新索引是否會提升性能。
在此示例中，咱們將以使用和不使用索引來比較相同查詢的性能。固然，咱們也會將查詢發送到相同的主機組。
一、設置查詢規則
1）設置查詢規則
建立如下規則（rule_id = 5）：
1）匹配 FROM sbtest1 ;
2）設置destination_hostgroup = 20 ;
3）設置mirror_flagOUT=100 ;
4）不設置mirror_hostgroup ;
Admin> DELETE FROM mysql_query_rules; Query OK, 1 row affected (0.00 sec) Admin> INSERT INTO mysql_query_rules (rule_id,active,match_pattern,destination_hostgroup,mirror_flagOUT,apply) VALUES (5,1,'FROM sbtest1 ',20,100,1); Query OK, 1 row affected (0.00 sec) Admin> SELECT rule_id,active,match_pattern,destination_hostgroup,mirror_flagOUT,mirror_hostgroup,apply FROM mysql_query_rules ; +---------+--------+---------------+-----------------------+----------------+------------------+-------+ | rule_id | active | match_pattern | destination_hostgroup | mirror_flagOUT | mirror_hostgroup | apply | +---------+--------+---------------+-----------------------+----------------+------------------+-------+ | 5 | 1 | FROM sbtest1 | 20 | 100 | NULL | 1 | +---------+--------+---------------+-----------------------+----------------+------------------+-------+ 1 row in set (0.00 sec)
因爲設置了 mirror_flagOUT ，所以將建立一個新會話來運行相同的查詢。可是，由於未設置 mirror_hostgroup ，因此會根據當前用戶在 mysql_users 中設置的默認主機組，將查詢將被髮送到其默認主機組上。相反的，若是咱們但願將鏡像查詢發送到與原始主機組相同的主機組。咱們能夠在rule_id = 5的規則中設置mirror_hostgroup，或者建立
一個新規則。這裏，選擇後者，咱們將建立一個新規則來匹配重寫查詢：
Admin> INSERT INTO mysql_query_rules (rule_id,active,flagIN,match_pattern,destination_hostgroup,replace_pattern,apply) VALUES (10,1,100,'FROM sbtest1 ',20,'FROM sbtest1 IGNORE INDEX(k_1) ',1); Query OK, 1 row affected (0.00 sec) Admin> SELECT rule_id,active,flagIN,match_pattern,replace_pattern,destination_hostgroup,mirror_flagOUT,mirror_hostgroup,apply FROM mysql_query_rules ; +---------+--------+--------+---------------+---------------------------------+-----------------------+----------------+------------------+-------+ | rule_id | active | flagIN | match_pattern | replace_pattern | destination_hostgroup | mirror_flagOUT | mirror_hostgroup | apply | +---------+--------+--------+---------------+---------------------------------+-----------------------+----------------+------------------+-------+ | 5 | 1 | 0 | FROM sbtest1 | NULL | 20 | 100 | NULL | 1 | | 10 | 1 | 100 | FROM sbtest1 | FROM sbtest1 IGNORE INDEX(k_1) | 20 | NULL | NULL | 1 | +---------+--------+--------+---------------+---------------------------------+-----------------------+----------------+------------------+-------+ 2 rows in set (0.00 sec)
或查看規則的所有信息：
Admin> SELECT * FROM mysql_query_rules \G
*************************** 1. row ***************************
rule_id: 5
active: 1
username: NULL
schemaname: NULL
flagIN: 0
client_addr: NULL
proxy_addr: NULL
proxy_port: NULL
digest: NULL
match_digest: NULL
match_pattern: FROM sbtest1
negate_match_pattern: 0
re_modifiers: CASELESS
flagOUT: NULL
replace_pattern: NULL
destination_hostgroup: 20
cache_ttl: NULL
cache_empty_result: NULL
cache_timeout: NULL
reconnect: NULL
timeout: NULL
retries: NULL
delay: NULL
next_query_flagIN: NULL
mirror_flagOUT: 100
mirror_hostgroup: NULL
error_msg: NULL
OK_msg: NULL
sticky_conn: NULL
multiplex: NULL
gtid_from_hostgroup: NULL
log: NULL
apply: 1
comment: NULL
*************************** 2. row ***************************
rule_id: 10
active: 1
username: NULL
schemaname: NULL
flagIN: 100
client_addr: NULL
proxy_addr: NULL
proxy_port: NULL
digest: NULL
match_digest: NULL
match_pattern: FROM sbtest1
negate_match_pattern: 0
re_modifiers: CASELESS
flagOUT: NULL
replace_pattern: FROM sbtest1 IGNORE INDEX(k_1)
destination_hostgroup: 20
cache_ttl: NULL
cache_empty_result: NULL
cache_timeout: NULL
reconnect: NULL
timeout: NULL
retries: NULL
delay: NULL
next_query_flagIN: NULL
mirror_flagOUT: NULL
mirror_hostgroup: NULL
error_msg: NULL
OK_msg: NULL
sticky_conn: NULL
multiplex: NULL
gtid_from_hostgroup: NULL
log: NULL
apply: 1
comment: NULL
2 rows in set (0.00 sec)
須要注意的是，在rule_id = 10的規則中，鏡像查詢將匹配該規則[由於mirror_flagOUT(5號規則)=flagIN(100=10號規則)]，此時咱們須要設置 destination_hostgroup 而不是 mirror_hostgroup ：只應該爲原始查詢設置mirror_hostgroup ，以便當即能夠將鏡像查詢請求發送到指定位置，而無需 mysql_query_rules 中的其餘額外規則介入。
2）加載配置到RUNTIME層
Admin> LOAD MYSQL QUERY RULES TO RUNTIME; Query OK, 0 rows affected (0.00 sec) Admin> SELECT rule_id,active,flagIN,match_pattern,replace_pattern,destination_hostgroup,mirror_flagOUT,mirror_hostgroup,apply FROM runtime_mysql_query_rules ; +---------+--------+--------+---------------+---------------------------------+-----------------------+----------------+------------------+-------+ | rule_id | active | flagIN | match_pattern | replace_pattern | destination_hostgroup | mirror_flagOUT | mirror_hostgroup | apply | +---------+--------+--------+---------------+---------------------------------+-----------------------+----------------+------------------+-------+ | 5 | 1 | 0 | FROM sbtest1 | NULL | 20 | 100 | NULL | 1 | | 10 | 1 | 100 | FROM sbtest1 | FROM sbtest1 IGNORE INDEX(k_1) | 20 | NULL | NULL | 1 | +---------+--------+--------+---------------+---------------------------------+-----------------------+----------------+------------------+-------+ 2 rows in set (0.00 sec)
配置已生效！
二、業務帳號執行查詢
1）使用業務帳號連入ProxySQL
# mysql -h 188.188.0.71 -P 6033 -umsandbox -p mysql> show databases; +--------------------+ | Database | +--------------------+ | information_schema | | sbtest | +--------------------+ 2 rows in set (0.00 sec) mysql> use sbtest; mysql> show tables; +------------------+ | Tables_in_sbtest | +------------------+ | sbtest1 | | sbtest2 | | sbtest3 | | sbtest4 | | sbtest5 | +------------------+ 5 rows in set (0.00 sec)
2）ProxySQL管理端清空stats_mysql_query_digest以得到新的統計信息：
Admin> SELECT COUNT(*) FROM stats_mysql_query_digest_reset; +----------+ | COUNT(*) | +----------+ | 0 | +----------+ 1 row in set (0.00 sec)
3）業務帳號執行查詢
從mysql客戶端咱們能夠運行一般的查詢：
mysql> SELECT id FROM sbtest1 ORDER BY k DESC LIMIT 3; +------+ | id | +------+ | 5076 | | 5121 | | 8573 | +------+ 3 rows in set (0.00 sec) mysql> SELECT id,k FROM sbtest1 ORDER BY k DESC LIMIT 3; +------+------+ | id | k | +------+------+ | 5076 | 7672 | | 5121 | 7655 | | 8573 | 7565 | +------+------+ 3 rows in set (0.01 sec)
4）ProxySQL管理端，查看當前以執行SQL的查詢概要統計：
Admin> SELECT hostgroup,count_star,sum_time,digest_text FROM stats_mysql_query_digest ORDER BY sum_time DESC; +-----------+------------+----------+--------------------------------------------------------------------+ | hostgroup | count_star | sum_time | digest_text | +-----------+------------+----------+--------------------------------------------------------------------+ | 20 | 1 | 3997 | SELECT id,k FROM sbtest1 IGNORE INDEX(k_1) ORDER BY k DESC LIMIT ? | | 20 | 1 | 3997 | SELECT id FROM sbtest1 IGNORE INDEX(k_1) ORDER BY k DESC LIMIT ? | | 20 | 1 | 615 | SELECT id FROM sbtest1 ORDER BY k DESC LIMIT ? | | 20 | 1 | 342 | SELECT id,k FROM sbtest1 ORDER BY k DESC LIMIT ? | +-----------+------------+----------+--------------------------------------------------------------------+ 4 rows in set (0.01 sec)
表stats_mysql_query_digest 的結果代表：
1）原始查詢被鏡像了；
2）原始查詢未被重寫(而是鏡像)；
3）鏡像查詢被重寫了(帶了規則裏的IGNORE語句)；
4）鏡像查詢速度要慢得多，由於忽略了索引；
7、高級示例：使用鏡像測試查詢重寫[Advanced example: use mirroring to test query rewrite]
在處理鏡像時，我被問到一個徹底不一樣的問題，與查詢重寫有關：如何知道給定的正則表達式是否與給定的查詢匹配，並驗證重寫模式是否正確？
更具體地說，問題是要了解重寫是否正確而不影響實時流量。雖然鏡像最初並非爲此設計的，但它能夠回答這個問題。
在這個例子中，咱們將編寫一個規則來匹配全部SELECT，"鏡像"它們，並嘗試重寫它們。
一、設置查詢規則
1）設置查詢規則
Admin> DELETE FROM mysql_query_rules;
Query OK, 2 rows affected (0.00 sec)
Admin> INSERT INTO mysql_query_rules (rule_id,active,match_pattern,destination_hostgroup,mirror_flagOUT,apply) VALUES (5,1,'^SELECT ',20,100,1);
Query OK, 1 row affected (0.00 sec)
Admin> INSERT INTO mysql_query_rules (rule_id,active,flagIN,match_pattern,destination_hostgroup,replace_pattern,apply) VALUES
(10,1,100,'^SELECT DISTINCT c FROM sbtest([0-9]{1,2}) WHERE id BETWEEN ([0-9]+) AND ([0-9]+)\+([0-9]+) ORDER BY c$',20,'SELECT DISTINCT c FROM sbtest\1 WHERE id = \3 \+ \4 ORDER BY c',1);
Query OK, 1 row affected (0.00 sec)
查看設置結果：
Admin> SELECT rule_id,active,flagIN,match_pattern,destination_hostgroup,replace_pattern,mirror_flagOUT,apply FROM mysql_query_rules \G
*************************** 1. row ***************************
rule_id: 5
active: 1
flagIN: 0
match_pattern: ^SELECT
destination_hostgroup: 20
replace_pattern: NULL
mirror_flagOUT: 100
apply: 1
*************************** 2. row ***************************
rule_id: 10
active: 1
flagIN: 100
match_pattern: ^SELECT DISTINCT c FROM sbtest([0-9]{1,2}) WHERE id BETWEEN ([0-9]+) AND ([0-9]+)\+([0-9]+) ORDER BY c$
destination_hostgroup: 20
replace_pattern: SELECT DISTINCT c FROM sbtest\1 WHERE id = \3 \+ \4 ORDER BY c
mirror_flagOUT: NULL
apply: 1
2 rows in set (0.00 sec)
2）加載配置到RUNTIME層
Admin> LOAD MYSQL QUERY RULES TO RUNTIME ;
Query OK, 0 rows affected (0.00 sec)
Admin> SELECT rule_id,active,flagIN,match_pattern,destination_hostgroup,replace_pattern,mirror_flagOUT,apply FROM runtime_mysql_query_rules \G
*************************** 1. row ***************************
rule_id: 5
active: 1
flagIN: 0
match_pattern: ^SELECT
destination_hostgroup: 20
replace_pattern: NULL
mirror_flagOUT: 100
apply: 1
*************************** 2. row ***************************
rule_id: 10
active: 1
flagIN: 100
match_pattern: ^SELECT DISTINCT c FROM sbtest([0-9]{1,2}) WHERE id BETWEEN ([0-9]+) AND ([0-9]+)\+([0-9]+) ORDER BY c$
destination_hostgroup: 20
replace_pattern: SELECT DISTINCT c FROM sbtest\1 WHERE id = \3 \+ \4 ORDER BY c
mirror_flagOUT: NULL
apply: 1
2 rows in set (0.00 sec)
上面的正則表達式很是複雜，這就是爲何鏡像查詢功能有用之處，它不是直接重寫實時流量。
二、業務帳號執行查詢
1）使用業務帳號連入ProxySQL
# mysql -h 188.188.0.71 -P 6033 -umsandbox -p mysql> show databases; +--------------------+ | Database | +--------------------+ | information_schema | | sbtest | +--------------------+ 2 rows in set (0.00 sec) mysql> use sbtest; mysql> show tables; +------------------+ | Tables_in_sbtest | +------------------+ | sbtest1 | | sbtest2 | | sbtest3 | | sbtest4 | | sbtest5 | +------------------+ 5 rows in set (0.00 sec)
2）ProxySQL管理端清空stats_mysql_query_digest以得到新的統計信息：
Admin> SELECT COUNT(*) FROM stats_mysql_query_digest_reset; +----------+ | COUNT(*) | +----------+ | 10 | +----------+ 1 row in set (0.00 sec)
3）業務帳號執行查詢
從mysql客戶端咱們能夠運行一般的查詢：
mysql> SELECT DISTINCT c FROM sbtest1 WHERE id BETWEEN 10 AND 10+2 ORDER BY c; +-------------------------------------------------------------------------------------------------------------------------+ | c | +-------------------------------------------------------------------------------------------------------------------------+ | 06208928544-69213163800-95083408911-83949560459-26629535077-58798231143-58688386449-59141897529-07315042085-86003451120 | | 68305043604-07392484646-78480928447-88155597080-08908465928-35357008626-44894171482-13904841657-13998032237-49278517178 | | 84225420767-95119807827-48689909948-04145663437-29723649568-88238910120-61256632514-12324871715-71270848294-09484980067 | +-------------------------------------------------------------------------------------------------------------------------+ 3 rows in set (0.00 sec)
查詢已成功運行。 如上所述，咱們沒有修改原始流量。
那 stats_mysql_query_digest 中是又怎麼樣？
4）ProxySQL管理端，查看當前以執行SQL的查詢概要統計：
Admin> select hostgroup,count_star,sum_time,digest_text from stats_mysql_query_digest ORDER BY digest_text; +-----------+------------+----------+----------------------------------------------------------------------+ | hostgroup | count_star | sum_time | digest_text | +-----------+------------+----------+----------------------------------------------------------------------+ | 20 | 2 | 1493 | SELECT DISTINCT c FROM sbtest1 WHERE id BETWEEN ? AND ?+? ORDER BY c | +-----------+------------+----------+----------------------------------------------------------------------+ 1 row in set (0.00 sec)
能夠看到，原始查詢執行了兩次，所以某些內容沒法正常運行。咱們能夠注意到兩個查詢都被髮送到hostgroup20：咱們應該相信rule_id = 10是匹配的，可是沒有重寫查詢。
讓咱們驗證它是匹配的：
Admin> SELECT * from stats_mysql_query_rules; +---------+------+ | rule_id | hits | +---------+------+ | 5 | 2 | -->其中有一次爲show tables觸發的。 | 10 | 1 | +---------+------+ 2 rows in set (0.00 sec)
根據 stats_mysql_query_rules 中的規則命中信息能夠看到，rule_id = 10的規則是匹配的。
那爲何不重寫查詢？
5）問題修復
在從新審查一次rule_id=10的replace_pattern內容：
SELECT DISTINCT c FROM sbtest\1 WHERE id = \3 \+ \4 ORDER BY c
發現，+號前面不應有轉義的。更新查詢規則10：
Admin> UPDATE mysql_query_rules SET replace_pattern='SELECT DISTINCT c FROM sbtest\1 WHERE id = \3 + \4 ORDER BY c' WHERE rule_id=10;
Query OK, 1 row affected (0.00 sec)
Admin> LOAD MYSQL QUERY RULES TO RUNTIME;
Query OK, 0 rows affected (0.00 sec)
Admin> SELECT rule_id,active,flagIN,match_pattern,destination_hostgroup,replace_pattern,mirror_flagOUT,apply FROM runtime_mysql_query_rules WHERE rule_id=10 \G
*************************** 1. row ***************************
rule_id: 10
active: 1
flagIN: 100
match_pattern: ^SELECT DISTINCT c FROM sbtest([0-9]{1,2}) WHERE id BETWEEN ([0-9]+) AND ([0-9]+)\+([0-9]+) ORDER BY c$
destination_hostgroup: 20
replace_pattern: SELECT DISTINCT c FROM sbtest\1 WHERE id = \3 + \4 ORDER BY c
mirror_flagOUT: NULL
apply: 1
1 row in set (0.00 sec)
修改已經生效！！
6）驗證結果
ProxySQL管理端清空stats_mysql_query_digest以得到新的統計信息：
Admin> SELECT COUNT(*) FROM stats_mysql_query_digest_reset; +----------+ | COUNT(*) | +----------+ | 5 | +----------+ 1 row in set (0.00 sec)
在客戶端再次執行原SQL：
mysql> SELECT DISTINCT c FROM sbtest1 WHERE id BETWEEN 10 AND 10+2 ORDER BY c; +-------------------------------------------------------------------------------------------------------------------------+ | c | +-------------------------------------------------------------------------------------------------------------------------+ | 06208928544-69213163800-95083408911-83949560459-26629535077-58798231143-58688386449-59141897529-07315042085-86003451120 | | 68305043604-07392484646-78480928447-88155597080-08908465928-35357008626-44894171482-13904841657-13998032237-49278517178 | | 84225420767-95119807827-48689909948-04145663437-29723649568-88238910120-61256632514-12324871715-71270848294-09484980067 | +-------------------------------------------------------------------------------------------------------------------------+ 3 rows in set (0.00 sec)
如今讓咱們驗證查詢是否被正確重寫：
Admin> select hostgroup,count_star,sum_time,digest_text from stats_mysql_query_digest ORDER BY digest_text; +-----------+------------+----------+----------------------------------------------------------------------+ | hostgroup | count_star | sum_time | digest_text | +-----------+------------+----------+----------------------------------------------------------------------+ | 20 | 1 | 414 | SELECT DISTINCT c FROM sbtest1 WHERE id = ? + ? ORDER BY c | | 20 | 1 | 661 | SELECT DISTINCT c FROM sbtest1 WHERE id BETWEEN ? AND ?+? ORDER BY c | +-----------+------------+----------+----------------------------------------------------------------------+ 2 rows in set (0.00 sec)
到此能夠看到，查詢被正確重寫，而且執行了！
8、高級示例：使用鏡像和防火牆來測試查詢重寫[Advanced example: use mirroring and firewall to test query rewrite]
前面的示例/練習有點超前：那麼，是否能夠在不執行查詢的狀況下重寫查詢？答案是：能夠的！
爲此，咱們將爲鏡像查詢設置 error_msg ：這樣ProxySQL將處理鏡像查詢，但會過濾它而不將其發送到任何mysql服務器。
如最初所講的那樣，能夠修改鏡像查詢，而且防火牆是修改鏡像查詢的示例。
例如：
接着前面的案例，繼續操做。
1）修改查詢規則
爲規則10添加error_msg信息。
Admin> UPDATE mysql_query_rules SET error_msg="random error, blah blah" WHERE rule_id=10; Query OK, 1 row affected (0.00 sec)
2）加載配置到RUNTIME層
Admin> LOAD MYSQL QUERY RULES TO RUNTIME; Query OK, 0 rows affected (0.01 sec)
查看修改結果：
Admin> SELECT rule_id,active,flagIN,match_pattern,destination_hostgroup,replace_pattern,mirror_flagOUT,apply,error_msg FROM runtime_mysql_query_rules WHERE rule_id=10 \G
*************************** 1. row ***************************
rule_id: 10
active: 1
flagIN: 100
match_pattern: ^SELECT DISTINCT c FROM sbtest([0-9]{1,2}) WHERE id BETWEEN ([0-9]+) AND ([0-9]+)\+([0-9]+) ORDER BY c$
destination_hostgroup: 20
replace_pattern: SELECT DISTINCT c FROM sbtest\1 WHERE id = \3 + \4 ORDER BY c
mirror_flagOUT: NULL
apply: 1
error_msg: random error, blah blah
1 row in set (0.00 sec)
能夠看到，修改已生效！！
3）ProxySQL管理端清空並查看當前以執行SQL的查詢概要統計：
Admin> SELECT COUNT(*) FROM stats_mysql_query_digest_reset; +----------+ | COUNT(*) | +----------+ | 0 | +----------+ 1 row in set (0.00 sec)
4）業務帳號執行查詢
在mysql客戶端從新運行查詢：
mysql> SELECT DISTINCT c FROM sbtest1 WHERE id BETWEEN 10 AND 10+2 ORDER BY c; +-------------------------------------------------------------------------------------------------------------------------+ | c | +-------------------------------------------------------------------------------------------------------------------------+ | 06208928544-69213163800-95083408911-83949560459-26629535077-58798231143-58688386449-59141897529-07315042085-86003451120 | | 68305043604-07392484646-78480928447-88155597080-08908465928-35357008626-44894171482-13904841657-13998032237-49278517178 | | 84225420767-95119807827-48689909948-04145663437-29723649568-88238910120-61256632514-12324871715-71270848294-09484980067 | +-------------------------------------------------------------------------------------------------------------------------+ 3 rows in set (0.00 sec)
5）ProxySQL管理端，查看當前以執行SQL的查詢概要統計：
Admin> select hostgroup,count_star,sum_time,digest_text from stats_mysql_query_digest ORDER BY digest_text; +-----------+------------+----------+----------------------------------------------------------------------+ | hostgroup | count_star | sum_time | digest_text | +-----------+------------+----------+----------------------------------------------------------------------+ | 10 | 1 | 0 | SELECT DISTINCT c FROM sbtest1 WHERE id = ? + ? ORDER BY c | | 20 | 1 | 730 | SELECT DISTINCT c FROM sbtest1 WHERE id BETWEEN ? AND ?+? ORDER BY c | +-----------+------------+----------+----------------------------------------------------------------------+ 2 rows in set (0.00 sec) Admin> SELECT * from stats_mysql_query_rules; +---------+------+ | rule_id | hits | +---------+------+ | 5 | 1 | | 10 | 1 | +---------+------+ 2 rows in set (0.01 sec)
Great!!咱們已看到查詢已被重寫，但實際上並未在任何地方發送：
1）sum_time = 0 ，由於響應是當即的;
2）hostgroup=10 ，在規則10中，設置了destination_hostgroup=20，但這裏卻顯示了業務帳號默認的主機組10；若是業務帳號未設置默認主機組，則顯示爲0；這種顯示和設置不一致也表面沒有發送到任何地方(不信能夠打開默認主機組MySQL的general_log去驗證).
完畢！
相關文章
- 1. ProxySQL官檔翻譯__24_Packaging
- 2. ProxySQL官檔翻譯__19_Monitor_Module
- 3. ProxySQL官檔翻譯__23_Error_codes
- 4. ProxySQL官檔翻譯__17_Multiplexing
- 5. ProxySQL官檔翻譯__18_Query_Cache
- 6. ProxySQL官檔翻譯__02_ProxySQL配置概述
- 7. ProxySQL官檔翻譯__22_Bug反饋
- 8. ProxySQL官檔翻譯__01_ProxySQL安裝
- 9. ProxySQL官檔翻譯__21_Memory_leak_detection(內存泄漏檢測)
- 10. ProxySQL官檔翻譯__09_ProxySQL配置之系統庫_00_Admin_Schemas
- 更多相關文章...
- • WSDL 文檔 - WSDL 教程
- • XSL-FO 文檔 - XSL-FO 教程
- • 使用阿里雲OSS+CDN部署前端頁面與加速靜態資源
- • ☆基於Java Instrument的Agent實現
相關標籤/搜索
每日一句
-
每一个你不满意的现在，都有一个你没有努力的曾经。
歡迎關注本站公眾號,獲取更多信息
相關文章
- 1. ProxySQL官檔翻譯__24_Packaging
- 2. ProxySQL官檔翻譯__19_Monitor_Module
- 3. ProxySQL官檔翻譯__23_Error_codes
- 4. ProxySQL官檔翻譯__17_Multiplexing
- 5. ProxySQL官檔翻譯__18_Query_Cache
- 6. ProxySQL官檔翻譯__02_ProxySQL配置概述
- 7. ProxySQL官檔翻譯__22_Bug反饋
- 8. ProxySQL官檔翻譯__01_ProxySQL安裝
- 9. ProxySQL官檔翻譯__21_Memory_leak_detection(內存泄漏檢測)
- 10. ProxySQL官檔翻譯__09_ProxySQL配置之系統庫_00_Admin_Schemas