Mycat 分片策略
- @(學習)[mycat, mysql]
1. 分片規則配置文件
分片規則配置文件rule.xml位於$MYCAT_HOME/conf目錄，它定義了全部拆分表的規則。在使用過程當中能夠靈活使用不一樣的分片算法，或者對同一個分片算法使用不一樣的參數，它讓分片過程可配置化，只須要簡單的幾步就可讓運維人員及數據庫管理員輕鬆將數據拆分到不一樣的物理庫中。該文件包含兩個重要的標籤，分別是Funcation和tableRule。node
1.1 Funcation標籤
<function name="rang-long" class="io.mycat.route.function.AutoPartitionByLong"> <property name="mapFile">autopartition-long.txt</property> </function>
<function name="rang-long" class="io.mycat.route.function.AutoPartitionByLong"> <property name="mapFile">autopartition-long.txt</property> </function>mysql
name屬性指定算法的名稱，在該文件中惟一。class屬性對應具體的分片算法，須要指定算法的具體類。property屬性根據算法的要求指定。
1.2 tableRule標籤
<tableRule name="auto-sharding-long"> <rule> <columns>id</columns> <algorithm>rang-long</algorithm> </rule> </tableRule>
name屬性指定分片規則的名稱，在該文件中惟一。rule屬性指定分片算法的具體內容，包含columns和algorithm兩個屬性。columns屬性指定對應的表中用於分片的列名。algorithm屬性對應function中指定的算法的名稱。
2. 連續分片和離散分片
分片優缺點算法
| 分片 | 連續分片 | 離散分片 |
|---|---|---|
| 優勢 | 擴容無需遷移數據 範圍條件查詢消耗資源少 |
併發訪問能力加強 範圍條件查詢性能提高 |
| 缺點 | 存在數據熱點的可能性 併發訪問能力受限於單一或少許DataNode |
數據擴容比較困難，涉及到數據遷移問題 數據庫鏈接消耗比較多 |
特殊分片 sql
2.1 連續分片
連續分片規則
* 自定義數字範圍分片
* 按日期（天）分片
* 按單月小時分片
* 天然月分片數據庫
2.1.1 自定義數字範圍分片
適用於明確知道分片字段的某個範圍屬於哪一個分片，其字段爲數字類型。vim
vim conf/schema.xml數組
<?xml version="1.0"?> <!DOCTYPE mycat:schema SYSTEM "schema.dtd"> <mycat:schema xmlns:mycat="http://io.mycat/"> <schema name="test" checkSQLschema="false" sqlMaxLimit="100"> <table name="auto_sharding_long" primaryKey="id" dataNode="dn0,dn1,dn2" rule="auto-sharding-long" /> </schema> <dataNode name="dn0" dataHost="dh-1" database="db0"/> <dataNode name="dn1" dataHost="dh-1" database="db1"/> <dataNode name="dn2" dataHost="dh-1" database="db2"/> <dataHost name="dh-1" maxCon="1000" minCon="10" balance="0" writeType="0" dbType="mysql" dbDriver="native" switchType="1" slaveThreshold="100"> <heartbeat>select user()</heartbeat> <writeHost host="hostM1" url="localhost:3306" user="root" password="123456"> </writeHost> </dataHost> </mycat:schema>
vim conf/rule.xml併發
<?xml version="1.0" encoding="UTF-8"?> <!DOCTYPE mycat:rule SYSTEM "rule.dtd"> <mycat:rule xmlns:mycat="http://io.mycat/"> <tableRule name="auto-sharding-long"> <rule> <columns>age</columns> <algorithm>rang-long</algorithm> </rule> </tableRule> <function name="rang-long" class="io.mycat.route.function.AutoPartitionByLong"> <property name="mapFile">autopartition-long.txt</property> <property name="defaultNode">0</property> </function> </mycat:rule>
配置說明：
* columns指定分片的表列名。
* algorithm指定分片函數與function對應。
* rang-long函數中的mapFile表明規則配置文件的路徑，可在$MYCAT_HOME/conf下新建目錄做好規範，好比將autopartition-long.txt放在$MYCAT_HOME/conf/func中，則用<property name="mapFile">func/autopartition-long.txt</property>來引用。
* defaultNode爲超過範圍後的默認節點。app
vim conf/autopartition-long.txt運維
# range start-end ,data node index # K=1000,M=10000. 0-2M=0 2M-3M=1 3M-6M=2
測試sql：
CREATE TABLE auto_sharding_long (`age` int NOT NULL ,`db_nm` varchar(20) NULL); INSERT INTO auto_sharding_long (age,db_nm) VALUES (20000, database()); INSERT INTO auto_sharding_long (age,db_nm) VALUES (25000, database()); INSERT INTO auto_sharding_long (age,db_nm) VALUES (35000, database());
- 1
- 2
- 3
- 4
測試結果：
[root@testA conf]# mysql -uroot -p123456 -h 127.0.0.1 -P8066 test mysql: [Warning] Using a password on the command line interface can be insecure. Welcome to the MySQL monitor. Commands end with ; or \g. Your MySQL connection id is 2 Server version: 5.6.29-mycat-1.6-RELEASE-20161028204710 MyCat Server (OpenCloundDB) Copyright (c) 2000, 2017, Oracle and/or its affiliates. All rights reserved. Oracle is a registered trademark of Oracle Corporation and/or its affiliates. Other names may be trademarks of their respective owners. Type 'help;' or '\h' for help. Type '\c' to clear the current input statement. mysql> CREATE TABLE auto_sharding_long (`age` int NOT NULL ,`db_nm` varchar(20) NULL); Query OK, 0 rows affected (0.10 sec) mysql> INSERT INTO auto_sharding_long (age,db_nm) VALUES (20000, database()); Query OK, 1 row affected (0.01 sec) mysql> INSERT INTO auto_sharding_long (age,db_nm) VALUES (25000, database()); Query OK, 1 row affected (0.00 sec) mysql> INSERT INTO auto_sharding_long (age,db_nm) VALUES (35000, database()); Query OK, 1 row affected (0.00 sec) mysql> select * from auto_sharding_long; +-------+-------+ | age | db_nm | +-------+-------+ | 20000 | db0 | | 35000 | db2 | | 25000 | db1 | +-------+-------+ 3 rows in set (0.00 sec) mysql>
注意
schema裏的table的dataNode節點個數必須 大於等於autopartition-long.txt裏的配置個數
dataNode節點從0開始
2.1.2 按日期（天）分片
根據時間類型字段，按天分片。
vim conf/schema.xml
<?xml version="1.0"?> <!DOCTYPE mycat:schema SYSTEM "schema.dtd"> <mycat:schema xmlns:mycat="http://io.mycat/"> <schema name="test" checkSQLschema="false" sqlMaxLimit="100"> <table name="sharding_by_day" primaryKey="id" dataNode="dn$0-2" rule="sharding-by-day" /> </schema> <dataNode name="dn0" dataHost="dh-1" database="db0"/> <dataNode name="dn1" dataHost="dh-1" database="db1"/> <dataNode name="dn2" dataHost="dh-1" database="db2"/> <dataHost name="dh-1" maxCon="1000" minCon="10" balance="0" writeType="0" dbType="mysql" dbDriver="native" switchType="1" slaveThreshold="100"> <heartbeat>select user()</heartbeat> <writeHost host="hostM1" url="localhost:3306" user="root" password="123456"> </writeHost> </dataHost> </mycat:schema>
vim conf/rule.xml
<?xml version="1.0" encoding="UTF-8"?> <!DOCTYPE mycat:rule SYSTEM "rule.dtd"> <mycat:rule xmlns:mycat="http://io.mycat/"> <tableRule name="sharding-by-day"> <rule> <columns>create_time</columns> <algorithm>part-by-day</algorithm> </rule> </tableRule> <function name="part-by-day" class="io.mycat.route.function.PartitionByDate"> <property name="dateFormat">yyyy-MM-dd</property> <property name="sBeginDate">2017-10-01</property> <property name="sPartionDay">10</property> </function> </mycat:rule>
配置說明：
* columns標識將要分片的表字段。
* algorithm爲分片函數。
* dateFormat爲日期格式。
* sBeginDate爲開始日期。
* sEndDate爲結束日期，若是配置了這個屬性，則表明數據達到了這個日期的分片後會重複從開始分片插入。
* sPartionDay爲分區天數，默認從開始日期算起，每隔10天一個分區。
測試SQL：
CREATE TABLE sharding_by_day (create_time timestamp NULL ON UPDATE CURRENT_TIMESTAMP ,`db_nm` varchar(20) NULL); INSERT INTO sharding_by_day (create_time,db_nm) VALUES ('2017-10-01', database()); INSERT INTO sharding_by_day (create_time,db_nm) VALUES ('2017-10-10', database()); INSERT INTO sharding_by_day (create_time,db_nm) VALUES ('2017-10-11', database()); INSERT INTO sharding_by_day (create_time,db_nm) VALUES ('2017-10-21', database()); INSERT INTO sharding_by_day (create_time,db_nm) VALUES ('2017-10-31', database()); select * from sharding_by_day;
測試結果：
[root@testA conf]# mysql -uroot -p123456 -h 127.0.0.1 -P8066 test mysql: [Warning] Using a password on the command line interface can be insecure. Welcome to the MySQL monitor. Commands end with ; or \g. Your MySQL connection id is 1 Server version: 5.6.29-mycat-1.6-RELEASE-20161028204710 MyCat Server (OpenCloundDB) Copyright (c) 2000, 2017, Oracle and/or its affiliates. All rights reserved. Oracle is a registered trademark of Oracle Corporation and/or its affiliates. Other names may be trademarks of their respective owners. Type 'help;' or '\h' for help. Type '\c' to clear the current input statement. mysql> CREATE TABLE sharding_by_day (create_time timestamp NULL ON UPDATE CURRENT_TIMESTAMP ,`db_nm` varchar(20) NULL); INSERT INTO sharding_by_day (create_time,db_nm) VALUES ('2017-10-01', database()); INSERT INTO sharding_by_day (create_time,db_nm) VALUES ('2017-10-10', database()); INSERT INTO sharding_by_day (create_time,db_nm) VALUES ('2017-10-11', database()); INSERT INTO sharding_by_day (create_time,db_nm) VALUES ('2017-10-21', database()); INSERT INTO sharding_by_day (create_time,db_nm) VALUES ('2017-10-31', database()); select * from sharding_by_day; Query OK, 0 rows affected (0.24 sec) mysql> INSERT INTO sharding_by_day (create_time,db_nm) VALUES ('2017-10-01', database()); Query OK, 1 row affected (0.10 sec) mysql> INSERT INTO sharding_by_day (create_time,db_nm) VALUES ('2017-10-10', database()); Query OK, 1 row affected (0.00 sec) mysql> INSERT INTO sharding_by_day (create_time,db_nm) VALUES ('2017-10-11', database()); Query OK, 1 row affected (0.01 sec) mysql> INSERT INTO sharding_by_day (create_time,db_nm) VALUES ('2017-10-21', database()); Query OK, 1 row affected (0.00 sec) mysql> INSERT INTO sharding_by_day (create_time,db_nm) VALUES ('2017-10-31', database()); ERROR 1064 (HY000): Can't find a valid data node for specified node index :SHARDING_BY_DAY -> CREATE_TIME -> 2017-10-31 -> Index : 3 mysql> select * from sharding_by_day; +---------------------+-------+ | create_time | db_nm | +---------------------+-------+ | 2017-10-01 00:00:00 | db0 | | 2017-10-10 00:00:00 | db0 | | 2017-10-11 00:00:00 | db1 | | 2017-10-21 00:00:00 | db2 | +---------------------+-------+ 4 rows in set (0.05 sec) mysql>
注意
以上測試結果中有個錯誤，由於配置的dataNode節點數爲3，而2017-10-31爲2017-10-01後的第四個10天中的第一天，所以須要至少4個dataNode，節點不夠就報如上錯誤了。
2.1.3 按單月小時分片
單月內按照小時拆分，最小粒度是小時，一天最多能夠有24個分片，最少1個分片，下個月從頭開始循環，每月末須要手工清理數據。
字段爲字符串類型，yyyymmddHH 10位。
vim conf/schema.xml
<?xml version="1.0"?> <!DOCTYPE mycat:schema SYSTEM "schema.dtd"> <mycat:schema xmlns:mycat="http://io.mycat/"> <schema name="test" checkSQLschema="false" sqlMaxLimit="100"> <table name="sharding_by_hour" primaryKey="id" dataNode="dn$0-2" rule="sharding-by-hour" /> </schema> <dataNode name="dn0" dataHost="dh-1" database="db0"/> <dataNode name="dn1" dataHost="dh-1" database="db1"/> <dataNode name="dn2" dataHost="dh-1" database="db2"/> <dataHost name="dh-1" maxCon="1000" minCon="10" balance="0" writeType="0" dbType="mysql" dbDriver="native" switchType="1" slaveThreshold="100"> <heartbeat>select user()</heartbeat> <writeHost host="hostM1" url="localhost:3306" user="root" password="123456"> </writeHost> </dataHost> </mycat:schema>
vim conf/rule.xml
<?xml version="1.0" encoding="UTF-8"?> <!DOCTYPE mycat:rule SYSTEM "rule.dtd"> <mycat:rule xmlns:mycat="http://io.mycat/"> <tableRule name="sharding-by-hour"> <rule> <columns>sharding_col</columns> <algorithm>sharding-by-hour</algorithm> </rule> </tableRule> <function name="sharding-by-hour" class="io.mycat.route.function.LatestMonthPartion"> <property name="splitOneDay">3</property> </function> </mycat:rule>
配置說明：
* columns標識將要分片的表字段，字符串類型（yyyymmddHH），格式須要符合Java標準。
* algorithm爲分片函數。
* splitOneDay爲一天切分的分片數。
測試SQL：
CREATE TABLE sharding_by_hour (create_time timestamp NULL ON UPDATE CURRENT_TIMESTAMP ,`db_nm` varchar(20) NULL,sharding_col varchar(10) null); INSERT INTO sharding_by_hour (sharding_col,create_time,db_nm) VALUES ('2017100101','2017-10-01', database()); INSERT INTO sharding_by_hour (sharding_col,create_time,db_nm) VALUES ('2017100108','2017-10-01', database()); INSERT INTO sharding_by_hour (sharding_col,create_time,db_nm) VALUES ('2017100109','2017-10-01', database()); INSERT INTO sharding_by_hour (sharding_col,create_time,db_nm) VALUES ('2017100116','2017-10-01', database()); INSERT INTO sharding_by_hour (sharding_col,create_time,db_nm) VALUES ('2017100117','2017-10-01', database()); INSERT INTO sharding_by_hour (sharding_col,create_time,db_nm) VALUES ('2017100123','2017-10-01', database()); select * from sharding_by_hour;
測試結果：
[root@testA conf]# mysql -uroot -p123456 -h 127.0.0.1 -P8066 test mysql: [Warning] Using a password on the command line interface can be insecure. Welcome to the MySQL monitor. Commands end with ; or \g. Your MySQL connection id is 1 Server version: 5.6.29-mycat-1.6-RELEASE-20161028204710 MyCat Server (OpenCloundDB) Copyright (c) 2000, 2017, Oracle and/or its affiliates. All rights reserved. Oracle is a registered trademark of Oracle Corporation and/or its affiliates. Other names may be trademarks of their respective owners. Type 'help;' or '\h' for help. Type '\c' to clear the current input statement. mysql> CREATE TABLE sharding_by_hour (create_time timestamp NULL ON UPDATE CURRENT_TIMESTAMP ,`db_nm` varchar(20) NULL,sharding_col varchar(10) null); INSERT INTO sharding_by_hour (sharding_col,create_time,db_nm) VALUES ('2017100101','2017-10-01', database()); INSERT INTO sharding_by_hour (sharding_col,create_time,db_nm) VALUES ('2017100108','2017-10-01', database()); INSERT INTO sharding_by_hour (sharding_col,create_time,db_nm) VALUES ('2017100109','2017-10-01', database()); INSERT INTO sharding_by_hour (sharding_col,create_time,db_nm) VALUES ('2017100116','2017-10-01', database()); INSERT INTO sharding_by_hour (sharding_col,create_time,db_nm) VALUES ('2017100117','2017-10-01', database()); INSERT INTO sharding_by_hour (sharding_col,create_time,db_nm) VALUES ('2017100123','2017-10-01', database()); select * from sharding_by_hour; Query OK, 0 rows affected (0.21 sec) mysql> INSERT INTO sharding_by_hour (sharding_col,create_time,db_nm) VALUES ('2017100101','2017-10-01', database()); Query OK, 1 row affected (0.11 sec) mysql> INSERT INTO sharding_by_hour (sharding_col,create_time,db_nm) VALUES ('2017100108','2017-10-01', database()); Query OK, 1 row affected (0.00 sec) mysql> INSERT INTO sharding_by_hour (sharding_col,create_time,db_nm) VALUES ('2017100109','2017-10-01', database()); Query OK, 1 row affected (0.01 sec) mysql> INSERT INTO sharding_by_hour (sharding_col,create_time,db_nm) VALUES ('2017100116','2017-10-01', database()); Query OK, 1 row affected (0.00 sec) mysql> INSERT INTO sharding_by_hour (sharding_col,create_time,db_nm) VALUES ('2017100117','2017-10-01', database()); Query OK, 1 row affected (0.01 sec) mysql> INSERT INTO sharding_by_hour (sharding_col,create_time,db_nm) VALUES ('2017100123','2017-10-01', database()); Query OK, 1 row affected (0.00 sec) mysql> select * from sharding_by_hour; +---------------------+-------+--------------+ | create_time | db_nm | sharding_col | +---------------------+-------+--------------+ | 2017-10-01 00:00:00 | db0 | 2017100101 | | 2017-10-01 00:00:00 | db1 | 2017100108 | | 2017-10-01 00:00:00 | db1 | 2017100109 | | 2017-10-01 00:00:00 | db2 | 2017100116 | | 2017-10-01 00:00:00 | db2 | 2017100117 | | 2017-10-01 00:00:00 | db2 | 2017100123 | +---------------------+-------+--------------+ 6 rows in set (0.06 sec)
注意
* 分片數必須跟dataNode節點數一致
* 最大分24個分片
* 分片字段的值必須是字符串，如：2017103112.(即爲2017年10月31日12點)
2.1.4 按天然月分片
使用場景爲按月份列分區，每一個天然月一個分片，查詢條件時使用between and。
字段爲時間類型。
vim conf/schema.xml
<?xml version="1.0"?> <!DOCTYPE mycat:schema SYSTEM "schema.dtd"> <mycat:schema xmlns:mycat="http://io.mycat/"> <schema name="test" checkSQLschema="false" sqlMaxLimit="100"> <table name="sharding_by_month" primaryKey="id" dataNode="dn$0-2" rule="sharding-by-month" /> </schema> <dataNode name="dn0" dataHost="dh-1" database="db0"/> <dataNode name="dn1" dataHost="dh-1" database="db1"/> <dataNode name="dn2" dataHost="dh-1" database="db2"/> <dataHost name="dh-1" maxCon="1000" minCon="10" balance="0" writeType="0" dbType="mysql" dbDriver="native" switchType="1" slaveThreshold="100"> <heartbeat>select user()</heartbeat> <writeHost host="hostM1" url="localhost:3306" user="root" password="123456"> </writeHost> </dataHost> </mycat:schema>
vim conf/rule.xml
<?xml version="1.0" encoding="UTF-8"?> <!DOCTYPE mycat:rule SYSTEM "rule.dtd"> <mycat:rule xmlns:mycat="http://io.mycat/"> <tableRule name="sharding-by-month"> <rule> <columns>create_time</columns> <algorithm>sharding-by-month</algorithm> </rule> </tableRule> <function name="sharding-by-month" class="io.mycat.route.function.PartitionByMonth"> <property name="dateFormat">yyyy-MM-dd</property> <property name="sBeginDate">2017-10-01</property> <property name="sEndDate">2017-12-30</property> </function> </mycat:rule>
配置說明：
* columns標識將要分片的表字段，字符串類型，與dateFormat格式一致。
* algorithm爲分片函數。
* dateFormat爲日期字符串格式。
* sBeginDate爲開始日期。
* sEndDate爲結束日期
測試SQL：
CREATE TABLE sharding_by_month (create_time timestamp NULL ON UPDATE CURRENT_TIMESTAMP ,`db_nm` varchar(20) NULL); INSERT INTO sharding_by_month (create_time,db_nm) VALUES ('2017-10-01', database()); INSERT INTO sharding_by_month (create_time,db_nm) VALUES ('2017-10-30', database()); INSERT INTO sharding_by_month (create_time,db_nm) VALUES ('2017-11-11', database()); INSERT INTO sharding_by_month (create_time,db_nm) VALUES ('2017-11-21', database()); INSERT INTO sharding_by_month (create_time,db_nm) VALUES ('2017-12-01', database()); INSERT INTO sharding_by_month (create_time,db_nm) VALUES ('2017-12-31', database()); INSERT INTO sharding_by_month (create_time,db_nm) VALUES ('2018-01-01', database()); INSERT INTO sharding_by_month (create_time,db_nm) VALUES ('2018-01-31', database()); select * from sharding_by_month
測試結果：
[root@testA conf]# mysql -uroot -p123456 -h 127.0.0.1 -P8066 test mysql: [Warning] Using a password on the command line interface can be insecure. Welcome to the MySQL monitor. Commands end with ; or \g. Your MySQL connection id is 1 Server version: 5.6.29-mycat-1.6-RELEASE-20161028204710 MyCat Server (OpenCloundDB) Copyright (c) 2000, 2017, Oracle and/or its affiliates. All rights reserved. Oracle is a registered trademark of Oracle Corporation and/or its affiliates. Other names may be trademarks of their respective owners. Type 'help;' or '\h' for help. Type '\c' to clear the current input statement. mysql> CREATE TABLE sharding_by_month (create_time timestamp NULL ON UPDATE CURRENT_TIMESTAMP ,`db_nm` varchar(20) NULL); INSERT INTO sharding_by_month (create_time,db_nm) VALUES ('2017-10-01', database()); INSERT INTO sharding_by_month (create_time,db_nm) VALUES ('2017-10-30', database()); INSERT INTO sharding_by_month (create_time,db_nm) VALUES ('2017-11-11', database()); INSERT INTO sharding_by_month (create_time,db_nm) VALUES ('2017-11-21', database()); INSERT INTO sharding_by_month (create_time,db_nm) VALUES ('2017-12-01', database()); INSERT INTO sharding_by_month (create_time,db_nm) VALUES ('2017-12-31', database()); INSERT INTO sharding_by_month (create_time,db_nm) VALUES ('2018-01-01', database()); INSERT INTO sharding_by_month (create_time,db_nm) VALUES ('2018-01-31', database()); Query OK, 0 rows affected (0.15 sec) mysql> INSERT INTO sharding_by_month (create_time,db_nm) VALUES ('2017-10-01', database()); select * from sharding_by_month; Query OK, 1 row affected (0.10 sec) mysql> INSERT INTO sharding_by_month (create_time,db_nm) VALUES ('2017-10-30', database()); Query OK, 1 row affected (0.00 sec) mysql> INSERT INTO sharding_by_month (create_time,db_nm) VALUES ('2017-11-11', database()); Query OK, 1 row affected (0.00 sec) mysql> INSERT INTO sharding_by_month (create_time,db_nm) VALUES ('2017-11-21', database()); Query OK, 1 row affected (0.01 sec) mysql> INSERT INTO sharding_by_month (create_time,db_nm) VALUES ('2017-12-01', database()); Query OK, 1 row affected (0.01 sec) mysql> INSERT INTO sharding_by_month (create_time,db_nm) VALUES ('2017-12-31', database()); Query OK, 1 row affected (0.00 sec) mysql> INSERT INTO sharding_by_month (create_time,db_nm) VALUES ('2018-01-01', database()); Query OK, 1 row affected (0.00 sec) mysql> INSERT INTO sharding_by_month (create_time,db_nm) VALUES ('2018-01-31', database()); Query OK, 1 row affected (0.01 sec) mysql> select * from sharding_by_month; +---------------------+-------+ | create_time | db_nm | +---------------------+-------+ | 2017-10-01 00:00:00 | db0 | | 2017-10-30 00:00:00 | db0 | | 2018-01-01 00:00:00 | db0 | | 2018-01-31 00:00:00 | db0 | | 2017-11-11 00:00:00 | db1 | | 2017-11-21 00:00:00 | db1 | | 2017-12-01 00:00:00 | db2 | | 2017-12-31 00:00:00 | db2 | +---------------------+-------+ 8 rows in set (0.06 sec)
注意
* schema裏的table的dataNode節點數必須：大於rule的開始時間按照分片數計算到如今的個數
* 按照天然月計算（不管是2八、30、31天都是一個月的）
* 分片節點個數能夠後增長，可是必須符合第一點說明。
2.2 離散分片
離散分片規則
* 枚舉分片
* 程序指定分區的分片
* 十進制求模分片
* 字符串hash解析分片
* 一致性哈希分片
2.2.1 枚舉分片
經過在配置文件中配置可能的枚舉id，指定數據分佈到不一樣的物理節點上，本規則適用於按照省份或區縣來拆分數據類業務。
vim conf/schema.xml
<?xml version="1.0"?> <!DOCTYPE mycat:schema SYSTEM "schema.dtd"> <mycat:schema xmlns:mycat="http://io.mycat/"> <schema name="test" checkSQLschema="false" sqlMaxLimit="100"> <table name="sharding_by_intfile" primaryKey="id" dataNode="dn$0-2" rule="sharding-by-intfile" /> </schema> <dataNode name="dn0" dataHost="dh-1" database="db0"/> <dataNode name="dn1" dataHost="dh-1" database="db1"/> <dataNode name="dn2" dataHost="dh-1" database="db2"/> <dataHost name="dh-1" maxCon="1000" minCon="10" balance="0" writeType="0" dbType="mysql" dbDriver="native" switchType="1" slaveThreshold="100"> <heartbeat>select user()</heartbeat> <writeHost host="hostM1" url="localhost:3306" user="root" password="123456"> </writeHost> </dataHost> </mycat:schema>
vim conf/rule.xml
<?xml version="1.0" encoding="UTF-8"?> <!DOCTYPE mycat:rule SYSTEM "rule.dtd"> <mycat:rule xmlns:mycat="http://io.mycat/"> <tableRule name="sharding-by-intfile"> <rule> <columns>age</columns> <algorithm>hash-int</algorithm> </rule> </tableRule> <function name="hash-int" class="io.mycat.route.function.PartitionByFileMap"> <property name="mapFile">partition-hash-int.txt</property> <property name="type">0</property> <property name="defaultNode">0</property> </function> </mycat:rule>
配置說明：
* columns標識將要分片的表字段，字符串類型，與dateFormat格式一致。
* algorithm爲分片函數。
* dateFormat爲日期字符串格式。
* sBeginDate爲開始日期。
* sEndDate爲結束日期
vim conf/partition-hash-int.txt
# 枚舉項=節點標識，枚舉項能夠是數字和字符（非漢字） 11=1 12=2
測試SQL：
CREATE TABLE sharding_by_intfile (`age` int NOT NULL ,`db_nm` varchar(20) NULL); INSERT INTO `sharding_by_intfile` (age,db_nm) VALUES (10, database()); INSERT INTO `sharding_by_intfile` (age,db_nm) VALUES (11, database()); INSERT INTO `sharding_by_intfile` (age,db_nm) VALUES (12, database()); select * from `sharding_by_intfile`;
測試結果：
[root@testA conf]# mysql -uroot -p123456 -h 127.0.0.1 -P8066 test mysql: [Warning] Using a password on the command line interface can be insecure. Welcome to the MySQL monitor. Commands end with ; or \g. Your MySQL connection id is 1 Server version: 5.6.29-mycat-1.6-RELEASE-20161028204710 MyCat Server (OpenCloundDB) Copyright (c) 2000, 2017, Oracle and/or its affiliates. All rights reserved. Oracle is a registered trademark of Oracle Corporation and/or its affiliates. Other names may be trademarks of their respective owners. Type 'help;' or '\h' for help. Type '\c' to clear the current input statement. mysql> CREATE TABLE sharding_by_intfile (`age` int NOT NULL ,`db_nm` varchar(20) NULL); INSERT INTO `sharding_by_intfile` (age,db_nm) VALUES (10, database()); INSERT INTO `sharding_by_intfile` (age,db_nm) VALUES (11, database()); INSERT INTO `sharding_by_intfile` (age,db_nm) VALUES (12, database()); select * from `sharding_by_intfile`; Query OK, 0 rows affected (0.21 sec) mysql> INSERT INTO `sharding_by_intfile` (age,db_nm) VALUES (10, database()); Query OK, 1 row affected (0.06 sec) mysql> INSERT INTO `sharding_by_intfile` (age,db_nm) VALUES (11, database()); Query OK, 1 row affected (0.00 sec) mysql> INSERT INTO `sharding_by_intfile` (age,db_nm) VALUES (12, database()); Query OK, 1 row affected (0.01 sec) mysql> select * from `sharding_by_intfile`; +-----+-------+ | age | db_nm | +-----+-------+ | 10 | db0 | | 12 | db2 | | 11 | db1 | +-----+-------+ 3 rows in set (0.21 sec)
注意
* schema裏的table的dataNode節點個數必須：大於等於 partition-hash-int.txt裏的配置個數
2.2.2 程序指定分區分片
在程序運行階段，由程序自主決定路由到哪一個分片。
vim conf/schema.xml
<?xml version="1.0"?> <!DOCTYPE mycat:schema SYSTEM "schema.dtd"> <mycat:schema xmlns:mycat="http://io.mycat/"> <schema name="test" checkSQLschema="false" sqlMaxLimit="100"> <table name="sharding_by_substring" primaryKey="id" dataNode="dn$0-2" rule="sharding-by-substring" /> </schema> <dataNode name="dn0" dataHost="dh-1" database="db0"/> <dataNode name="dn1" dataHost="dh-1" database="db1"/> <dataNode name="dn2" dataHost="dh-1" database="db2"/> <dataHost name="dh-1" maxCon="1000" minCon="10" balance="0" writeType="0" dbType="mysql" dbDriver="native" switchType="1" slaveThreshold="100"> <heartbeat>select user()</heartbeat> <writeHost host="hostM1" url="localhost:3306" user="root" password="123456"> </writeHost> </dataHost> </mycat:schema>
vim conf/rule.xml
<?xml version="1.0" encoding="UTF-8"?> <!DOCTYPE mycat:rule SYSTEM "rule.dtd"> <mycat:rule xmlns:mycat="http://io.mycat/"> <tableRule name="sharding-by-substring"> <rule> <columns>user_id</columns> <algorithm>sharding-by-substring</algorithm> </rule> </tableRule> <function name="sharding-by-substring" class="io.mycat.route.function.PartitionDirectBySubString"> <property name="startIndex">0</property> <property name="size">2</property> <property name="partitionCount">3</property> <property name="defaultPartition">0</property> </function> </mycat:rule>
配置說明：
* columns標識將要分片的表字段。
* algorithm爲分片函數。
* startIndex爲字符串開始截取位，從0開始。
* size爲截取的位數。
* partitionCount爲分片個數
* defaultPartition爲默認節點
測試SQL：
CREATE TABLE sharding_by_substring (`user_id` varchar(20) NOT NULL ,`db_nm` varchar(20) NULL); INSERT INTO `sharding_by_substring` (user_id,db_nm) VALUES ('05-10000', database()); INSERT INTO `sharding_by_substring` (user_id,db_nm) VALUES ('02-10001', database()); INSERT INTO `sharding_by_substring` (user_id,db_nm) VALUES ('03-10002', database()); select * from `sharding_by_substring`;
測試結果：
[root@testA conf]# mysql -uroot -p123456 -h 127.0.0.1 -P8066 test
mysql: [Warning] Using a password on the command line interface can be insecure.
Welcome to the MySQL monitor. Commands end with ; or \g.
Your MySQL connection id is 3
Server version: 5.6.29-mycat-1.6-RELEASE-20161028204710 MyCat Server (OpenCloundDB)
Copyright (c) 2000, 2017, Oracle and/or its affiliates. All rights reserved.
Oracle is a registered trademark of Oracle Corporation and/or its
affiliates. Other names may be trademarks of their respective
owners.
Type 'help;' or '\h' for help. Type '\c' to clear the current input statement. mysql> CREATE TABLE sharding_by_substring (`user_id` varchar(20) NOT NULL ,`db_nm` varchar(20) NULL); INSERT INTO `sharding_by_substring` (user_id,db_nm) VALUES ('05-10000', database()); INSERT INTO `sharding_by_substring` (user_id,db_nm) VALUES ('02-10001', database()); INSERT INTO `sharding_by_substring` (user_id,db_nm) VALUES ('03-10002', database()); select * from `sharding_by_substring`; Query OK, 0 rows affected (0.35 sec) mysql> INSERT INTO `sharding_by_substring` (user_id,db_nm) VALUES ('05-10000', database()); Query OK, 1 row affected (0.00 sec) mysql> INSERT INTO `sharding_by_substring` (user_id,db_nm) VALUES ('02-10001', database()); Query OK, 1 row affected (0.01 sec) mysql> INSERT INTO `sharding_by_substring` (user_id,db_nm) VALUES ('03-10002', database()); Query OK, 1 row affected (0.01 sec) mysql> select * from `sharding_by_substring`; +----------+-------+ | user_id | db_nm | +----------+-------+ | 05-10000 | db0 | | 03-10002 | db0 | | 02-10001 | db2 | +----------+-------+ 3 rows in set (0.00 sec)
注意
* 直接根據字符子串（必須是數字）計算分區號（由應用傳遞參數，指定分區號）。
例如user_id=05-10000，其中user_id是從startIndex=0開始的，截取長度爲兩位數字，即05，05就是獲取的分區，默認分配到defaultPartition。
2.2.3 十進制求模分片
對分片字段十進制取模運算，其數據分佈最均勻。
vim conf/schema.xml
<?xml version="1.0"?> <!DOCTYPE mycat:schema SYSTEM "schema.dtd"> <mycat:schema xmlns:mycat="http://io.mycat/"> <schema name="test" checkSQLschema="false" sqlMaxLimit="100"> <table name="sharding_by_substring" primaryKey="id" dataNode="dn$0-2" rule="sharding-by-substring" /> </schema> <dataNode name="dn0" dataHost="dh-1" database="db0"/> <dataNode name="dn1" dataHost="dh-1" database="db1"/> <dataNode name="dn2" dataHost="dh-1" database="db2"/> <dataHost name="dh-1" maxCon="1000" minCon="10" balance="0" writeType="0" dbType="mysql" dbDriver="native" switchType="1" slaveThreshold="100"> <heartbeat>select user()</heartbeat> <writeHost host="hostM1" url="localhost:3306" user="root" password="123456"> </writeHost> </dataHost> </mycat:schema>
vim conf/rule.xml
<?xml version="1.0" encoding="UTF-8"?> <!DOCTYPE mycat:rule SYSTEM "rule.dtd"> <mycat:rule xmlns:mycat="http://io.mycat/"> <tableRule name="sharding-by-substring"> <rule> <columns>user_id</columns> <algorithm>sharding-by-substring</algorithm> </rule> </tableRule> <function name="sharding-by-substring" class="io.mycat.route.function.PartitionDirectBySubString"> <property name="startIndex">0</property> <property name="size">2</property> <property name="partitionCount">3</property> <property name="defaultPartition">0</property> </function> </mycat:rule>
配置說明：
* columns標識將要分片的表字段。
* algorithm爲分片函數。
* startIndex爲字符串開始截取位，從0開始。
* size爲截取的位數。
* partitionCount爲分片個數
* defaultPartition爲默認節點
測試SQL：
CREATE TABLE `sharding_by_mod` (id int(10) null,`db_nm` varchar(20) NULL); INSERT INTO `sharding_by_mod` (id,db_nm) VALUES (1, database()); INSERT INTO `sharding_by_mod` (id,db_nm) VALUES (2, database()); INSERT INTO `sharding_by_mod` (id,db_nm) VALUES (3, database()); INSERT INTO `sharding_by_mod` (id,db_nm) VALUES (10, database()); select * from sharding_by_mod;
測試結果：
[root@testA conf]# mysql -uroot -p123456 -h 127.0.0.1 -P8066 test mysql: [Warning] Using a password on the command line interface can be insecure. Welcome to the MySQL monitor. Commands end with ; or \g. Your MySQL connection id is 2 Server version: 5.6.29-mycat-1.6-RELEASE-20161028204710 MyCat Server (OpenCloundDB) Copyright (c) 2000, 2017, Oracle and/or its affiliates. All rights reserved. Oracle is a registered trademark of Oracle Corporation and/or its affiliates. Other names may be trademarks of their respective owners. Type 'help;' or '\h' for help. Type '\c' to clear the current input statement. mysql> CREATE TABLE `sharding_by_mod` (id int(10) null,`db_nm` varchar(20) NULL); INSERT INTO `sharding_by_mod` (id,db_nm) VALUES (1, database()); INSERT INTO `sharding_by_mod` (id,db_nm) VALUES (2, database()); INSERT INTO `sharding_by_mod` (id,db_nm) VALUES (3, database()); INSERT INTO `sharding_by_mod` (id,db_nm) VALUES (10, database()); select * from sharding_by_mod; Query OK, 0 rows affected (0.22 sec) mysql> INSERT INTO `sharding_by_mod` (id,db_nm) VALUES (1, database()); Query OK, 1 row affected (0.04 sec) mysql> INSERT INTO `sharding_by_mod` (id,db_nm) VALUES (2, database()); Query OK, 1 row affected (0.02 sec) mysql> INSERT INTO `sharding_by_mod` (id,db_nm) VALUES (3, database()); Query OK, 1 row affected (0.00 sec) mysql> INSERT INTO `sharding_by_mod` (id,db_nm) VALUES (10, database()); Query OK, 1 row affected (0.01 sec) mysql> select * from sharding_by_mod; +------+-------+ | id | db_nm | +------+-------+ | 3 | db0 | | 1 | db1 | | 10 | db1 | | 2 | db2 | +------+-------+ 4 rows in set (0.10 sec)
注意
* 對分片字段十進制取模運算，dataNode=id%count
* 數據分佈最均勻。
* 擴容須要從新計算分片，數據遷移繁瑣。
* 儘可能的提早計算出來增量，建立足夠多的分片數。可是也不宜太多，給運維形成負擔。平衡點本身掌控。
2.2.4 字符串hash解析分片
此規則是截取字符串中的int數值的hash分片。
vim conf/schema.xml
<?xml version="1.0"?> <!DOCTYPE mycat:schema SYSTEM "schema.dtd"> <mycat:schema xmlns:mycat="http://io.mycat/"> <schema name="test" checkSQLschema="false" sqlMaxLimit="100"> <table name="sharding_by_stringhash" primaryKey="ord_no" dataNode="dn$0-2" rule="sharding-by-stringhash" /> </schema> <dataNode name="dn0" dataHost="dh-1" database="db0"/> <dataNode name="dn1" dataHost="dh-1" database="db1"/> <dataNode name="dn2" dataHost="dh-1" database="db2"/> <dataHost name="dh-1" maxCon="1000" minCon="10" balance="0" writeType="0" dbType="mysql" dbDriver="native" switchType="1" slaveThreshold="100"> <heartbeat>select user()</heartbeat> <writeHost host="hostM1" url="localhost:3306" user="root" password="123456"> </writeHost> </dataHost> </mycat:schema>
vim conf/rule.xml
<?xml version="1.0" encoding="UTF-8"?> <!DOCTYPE mycat:rule SYSTEM "rule.dtd"> <mycat:rule xmlns:mycat="http://io.mycat/"> <tableRule name="sharding-by-stringhash"> <rule> <columns>ord_no</columns> <algorithm>sharding-by-stringhash</algorithm> </rule> </tableRule> <function name="sharding-by-stringhash" class="io.mycat.route.function.PartitionByString"> <property name="partitionLength">512</property> <property name="partitionCount">2</property> <property name="hashSlice">-6:0</property> </function> </mycat:rule>
配置說明：
* columns標識將要分片的表字段。
* algorithm爲分片函數。
* partitionLength爲字符串hash的求模基數。
* partitionCount爲分區數。其中partitionLength*partitionCount=1024
* hashSlice爲預算位，即根據子字符串中的int值進行hash運算。
0 表明 str.length(), -1 表明 str.length()-1，大於0只表明數字自身
能夠理解爲substring（start，end），start爲0則只表示0
例1：值「45abc」，hash預算位0:2 ，取其中45進行計算
例2：值「aaaabbb2345」，hash預算位-4:0 ，取其中2345進行計算
測試SQL：
CREATE TABLE sharding_by_stringhash (ord_no varchar(20) NULL,`db_nm` varchar(20) NULL); INSERT INTO `sharding_by_stringhash` (ord_no,db_nm) VALUES (171022237582, database()); INSERT INTO `sharding_by_stringhash` (ord_no,db_nm) VALUES (171022553756, database()); select * from sharding_by_stringhash;
測試結果：
[root@testA conf]# mysql -uroot -p123456 -h 127.0.0.1 -P8066 test mysql: [Warning] Using a password on the command line interface can be insecure. Welcome to the MySQL monitor. Commands end with ; or \g. Your MySQL connection id is 1 Server version: 5.6.29-mycat-1.6-RELEASE-20161028204710 MyCat Server (OpenCloundDB) Copyright (c) 2000, 2017, Oracle and/or its affiliates. All rights reserved. Oracle is a registered trademark of Oracle Corporation and/or its affiliates. Other names may be trademarks of their respective owners. Type 'help;' or '\h' for help. Type '\c' to clear the current input statement. mysql> CREATE TABLE sharding_by_stringhash (ord_no varchar(20) NULL,`db_nm` varchar(20) NULL); INSERT INTO `sharding_by_stringhash` (ord_no,db_nm) VALUES (171022237582, database()); INSERT INTO `sharding_by_stringhash` (ord_no,db_nm) VALUES (171022553756, database()); select * from sharding_by_stringhash; Query OK, 0 rows affected (0.18 sec) mysql> INSERT INTO `sharding_by_stringhash` (ord_no,db_nm) VALUES (171022237582, database()); Query OK, 1 row affected (0.07 sec) mysql> INSERT INTO `sharding_by_stringhash` (ord_no,db_nm) VALUES (171022553756, database()); Query OK, 1 row affected (0.00 sec) mysql> select * from sharding_by_stringhash; +--------------+-------+ | ord_no | db_nm | +--------------+-------+ | 171022237582 | db0 | | 171022553756 | db1 | +--------------+-------+ 2 rows in set (0.09 sec)
注意
* 分片數量必須**小於等於**dataNode數
2.2.5 一致性hash分片
一致性hash算法有效解決了分佈式數據的擴容問題。由於此規則優勢在於擴容時遷移數據量比較少，前提是分片節點比較多，虛擬節點分配多些。虛擬節點分配的少就會形成數據分佈不夠均勻。但若是實際分片數據比較少，遷移量也會比較多。
vim conf/schema.xml
<?xml version="1.0"?> <!DOCTYPE mycat:schema SYSTEM "schema.dtd"> <mycat:schema xmlns:mycat="http://io.mycat/"> <schema name="test" checkSQLschema="false" sqlMaxLimit="100"> <table name="sharding_by_murmurhash" primaryKey="id" dataNode="dn$0-2" rule="sharding-by-murmurhash" /> </schema> <dataNode name="dn0" dataHost="dh-1" database="db0"/> <dataNode name="dn1" dataHost="dh-1" database="db1"/> <dataNode name="dn2" dataHost="dh-1" database="db2"/> <dataHost name="dh-1" maxCon="1000" minCon="10" balance="0" writeType="0" dbType="mysql" dbDriver="native" switchType="1" slaveThreshold="100"> <heartbeat>select user()</heartbeat> <writeHost host="hostM1" url="localhost:3306" user="root" password="123456"> </writeHost> </dataHost> </mycat:schema>
vim conf/rule.xml
<?xml version="1.0" encoding="UTF-8"?> <!DOCTYPE mycat:rule SYSTEM "rule.dtd"> <mycat:rule xmlns:mycat="http://io.mycat/"> <tableRule name="sharding-by-murmurhash"> <rule> <columns>id</columns> <algorithm>sharding-by-murmurhash</algorithm> </rule> </tableRule> <function name="sharding-by-murmurhash" class="io.mycat.route.function.PartitionByMurmurHash"> <property name="seed">0</property><!-- 默認是0 --> <property name="count">2</property><!-- 要分片的數據庫節點數量，必須指定，不然無法分片 --> <property name="virtualBucketTimes">160</property><!-- 一個實際的數據庫節點被映射爲這麼多虛擬節點，默認是160倍，也就是虛擬節點數是物理節點數的160倍 --> <!-- <property name="weightMapFile">weightMapFile</property> 節點的權重，沒有指定權重的節點默認是1。以properties文件的格式填寫，以從0開始到count-1的 整數值也就是節點索引爲key，以節點權重值爲值。全部權重值必須是正整數，不然以1代替 --> <!-- <property name="bucketMapPath">/etc/mycat/bucketMapPath</property> 用於測試時觀察各物理節點與虛擬節點的分佈狀況，若是指定了這個屬性，會把虛擬節點的murmur hash值與物理節點的映射按行輸出到 這個文件，沒有默認值> ，若是不指定，就不會輸出任何東西 --> </function> </mycat:rule>
測試SQL：
CREATE TABLE sharding_by_murmurhash (id int(10) null,`db_nm` varchar(20) NULL); INSERT INTO `sharding_by_murmurhash` (id,db_nm) VALUES (1, database()); INSERT INTO `sharding_by_murmurhash` (id,db_nm) VALUES (2, database()); INSERT INTO `sharding_by_murmurhash` (id,db_nm) VALUES (3, database()); INSERT INTO `sharding_by_murmurhash` (id,db_nm) VALUES (4, database()); INSERT INTO `sharding_by_murmurhash` (id,db_nm) VALUES (5, database()); INSERT INTO `sharding_by_murmurhash` (id,db_nm) VALUES (6, database()); INSERT INTO `sharding_by_murmurhash` (id,db_nm) VALUES (7, database()); INSERT INTO `sharding_by_murmurhash` (id,db_nm) VALUES (8, database()); INSERT INTO `sharding_by_murmurhash` (id,db_nm) VALUES (9, database()); INSERT INTO `sharding_by_murmurhash` (id,db_nm) VALUES (10, database()); select * from sharding_by_murmurhash;
測試結果：
[root@testA conf]# mysql -uroot -p123456 -h 127.0.0.1 -P8066 test mysql: [Warning] Using a password on the command line interface can be insecure. Welcome to the MySQL monitor. Commands end with ; or \g. Your MySQL connection id is 2 Server version: 5.6.29-mycat-1.6-RELEASE-20161028204710 MyCat Server (OpenCloundDB) Copyright (c) 2000, 2017, Oracle and/or its affiliates. All rights reserved. Oracle is a registered trademark of Oracle Corporation and/or its affiliates. Other names may be trademarks of their respective owners. Type 'help;' or '\h' for help. Type '\c' to clear the current input statement. mysql> CREATE TABLE sharding_by_murmurhash (id int(10) null,`db_nm` varchar(20) NULL); INSERT INTO `sharding_by_murmurhash` (id,db_nm) VALUES (1, database()); INSERT INTO `sharding_by_murmurhash` (id,db_nm) VALUES (2, database()); INSERT INTO `sharding_by_murmurhash` (id,db_nm) VALUES (3, database()); INSERT INTO `sharding_by_murmurhash` (id,db_nm) VALUES (4, database()); Query OK, 0 rows affected (0.11 sec) mysql> INSERT INTO `sharding_by_murmurhash` (id,db_nm) VALUES (1, database()); INSERT INTO `sharding_by_murmurhash` (id,db_nm) VALUES (5, database()); Query OK, 1 row affected (0.01 sec) mysql> INSERT INTO `sharding_by_murmurhash` (id,db_nm) VALUES (2, database()); INSERT INTO `sharding_by_murmurhash` (id,db_nm) VALUES (6, database()); INSERT INTO `sharding_by_murmurhash` (id,db_nm) VALUES (7, database()); INSERT INTO `sharding_by_murmurhash` (id,db_nm) VALUES (8, database()); Query OK, 1 row affected (0.05 sec) mysql> INSERT INTO `sharding_by_murmurhash` (id,db_nm) VALUES (3, database()); Query OK, 1 row affected (0.00 sec) mysql> INSERT INTO `sharding_by_murmurhash` (id,db_nm) VALUES (4, database()); INSERT INTO `sharding_by_murmurhash` (id,db_nm) VALUES (9, database()); Query OK, 1 row affected (0.01 sec) mysql> INSERT INTO `sharding_by_murmurhash` (id,db_nm) VALUES (5, database()); Query OK, 1 row affected (0.00 sec) mysql> INSERT INTO `sharding_by_murmurhash` (id,db_nm) VALUES (6, database()); Query OK, 1 row affected (0.01 sec) mysql> INSERT INTO `sharding_by_murmurhash` (id,db_nm) VALUES (7, database()); Query OK, 1 row affected (0.00 sec) mysql> INSERT INTO `sharding_by_murmurhash` (id,db_nm) VALUES (8, database()); INSERT INTO `sharding_by_murmurhash` (id,db_nm) VALUES (10, database()); Query OK, 1 row affected (0.01 sec) mysql> INSERT INTO `sharding_by_murmurhash` (id,db_nm) VALUES (9, database()); Query OK, 1 row affected (0.00 sec) mysql> INSERT INTO `sharding_by_murmurhash` (id,db_nm) VALUES (10, database()); Query OK, 1 row affected (0.01 sec) mysql> select * from sharding_by_murmurhash; +------+-------+ | id | db_nm | +------+-------+ | 1 | db1 | | 2 | db1 | | 3 | db1 | | 4 | db1 | | 7 | db1 | | 9 | db1 | | 10 | db1 | | 5 | db0 | | 6 | db0 | | 8 | db0 | +------+-------+ 10 rows in set (0.08 sec)
注意
* 分片數量必須**小於等於**dataNode數
3. 綜合類分片規則
- 範圍求模分片
- 日期範圍hash分片
- 取模範圍約束分片
- ASCII碼求模範圍約束（字符串）
- 固定分片hash（二進制）
3.1 範圍求模分片
該算法先進行範圍分片，計算出分片組，組內再求模，綜合了範圍分片和求模分片的優勢。分片組內使用求模能夠保證組內的數據分佈比較均勻，分片組之間採用範圍分片能夠兼顧範圍分片的特色。事先規定好分片的數量，數據擴容時按分片組擴容，則原有分片組的數據不須要遷移。因爲分片組內的數據分佈比較均勻，因此分片組內能夠避免熱點數據問題。
vim conf/schema.xml
<?xml version="1.0"?> <!DOCTYPE mycat:schema SYSTEM "schema.dtd"> <mycat:schema xmlns:mycat="http://io.mycat/"> <schema name="test" checkSQLschema="false" sqlMaxLimit="100"> <table name="sharding_by_rang_mod" primaryKey="id" dataNode="dn$0-10" rule="sharding-by-rang-mod" /> </schema> <dataNode name="dn0" dataHost="dh-1" database="db0"/> <dataNode name="dn1" dataHost="dh-1" database="db1"/> <dataNode name="dn2" dataHost="dh-1" database="db2"/> <dataNode name="dn3" dataHost="dh-1" database="db3"/> <dataNode name="dn4" dataHost="dh-1" database="db4"/> <dataNode name="dn5" dataHost="dh-1" database="db5"/> <dataNode name="dn6" dataHost="dh-1" database="db6"/> <dataNode name="dn7" dataHost="dh-1" database="db7"/> <dataNode name="dn8" dataHost="dh-1" database="db8"/> <dataNode name="dn9" dataHost="dh-1" database="db9"/> <dataNode name="dn10" dataHost="dh-1" database="db10"/> <dataHost name="dh-1" maxCon="1000" minCon="10" balance="0" writeType="0" dbType="mysql" dbDriver="native" switchType="1" slaveThreshold="100"> <heartbeat>select user()</heartbeat> <writeHost host="hostM1" url="localhost:3306" user="root" password="123456"> </writeHost> </dataHost> </mycat:schema>
vim conf/rule.xml
<?xml version="1.0" encoding="UTF-8"?> <!DOCTYPE mycat:rule SYSTEM "rule.dtd"> <mycat:rule xmlns:mycat="http://io.mycat/"> <tableRule name="sharding-by-rang-mod"> <rule> <columns>id</columns> <algorithm>sharding-by-rang-mod</algorithm> </rule> </tableRule> <function name="sharding-by-rang-mod" class="io.mycat.route.function.PartitionByRangeMod"> <property name="mapFile">partition-range-mod.txt</property> <property name="defaultNode">0</property> </function> </mycat:rule>
vim conf/partition-range-mod.txt
# range start-end ,data node group size 0-200M=3 200M1-400M=1 400M1-600M=4 600M1-800M=2
配置說明：
* columns標識將要分片的表字段。
* algorithm爲分片函數。
* mapFile爲指定分片規則的配置文件。
* defaultNode爲未包含以上規則的數據存儲節點，節點從0開始。
測試SQL：
CREATE TABLE sharding_by_rang_mod (id bigint null,`db_nm` varchar(20) NULL); INSERT INTO `sharding_by_rang_mod` (id,db_nm) VALUES (1000, database()); INSERT INTO `sharding_by_rang_mod` (id,db_nm) VALUES (1002, database()); INSERT INTO `sharding_by_rang_mod` (id,db_nm) VALUES (30000, database()); INSERT INTO `sharding_by_rang_mod` (id,db_nm) VALUES (30004, database()); INSERT INTO `sharding_by_rang_mod` (id,db_nm) VALUES (40000, database()); INSERT INTO `sharding_by_rang_mod` (id,db_nm) VALUES (40005, database()); INSERT INTO `sharding_by_rang_mod` (id,db_nm) VALUES (60005, database()); INSERT INTO `sharding_by_rang_mod` (id,db_nm) VALUES (60006, database()); INSERT INTO `sharding_by_rang_mod` (id,db_nm) VALUES (80006, database()); INSERT INTO `sharding_by_rang_mod` (id,db_nm) VALUES (60008, database()); select * from sharding_by_rang_mod;
測試結果：
[root@testA conf]# mysql -uroot -p123456 -h 127.0.0.1 -P8066 test mysql: [Warning] Using a password on the command line interface can be insecure. Welcome to the MySQL monitor. Commands end with ; or \g. Your MySQL connection id is 1 Server version: 5.6.29-mycat-1.6-RELEASE-20161028204710 MyCat Server (OpenCloundDB) Copyright (c) 2000, 2017, Oracle and/or its affiliates. All rights reserved. Oracle is a registered trademark of Oracle Corporation and/or its affiliates. Other names may be trademarks of their respective owners. Type 'help;' or '\h' for help. Type '\c' to clear the current input statement. mysql> CREATE TABLE sharding_by_rang_mod (id bigint null,`db_nm` varchar(20) NULL); INSERT INTO `sharding_by_rang_mod` (id,db_nm) VALUES (1000, database()); INSERT INTO `sharding_by_rang_mod` (id,db_nm) VALUES (1002, database()); INSERT INTO `sharding_by_rang_mod` (id,db_nm) VALUES (30000, database()); INSERT INTO `sharding_by_rang_mod` (id,db_nm) VALUES (30004, database()); INSERT INTO `sharding_by_rang_mod` (id,db_nm) VALUES (40000, database()); INSERT INTO `sharding_by_rang_mod` (id,db_nm) VALUES (40005, database()); INSERT INTO `sharding_by_rang_mod` (id,db_nm) VALUES (60005, database()); INSERT INTO `sharding_by_rang_mod` (id,db_nm) VALUES (60006, database()); INSERT INTO `sharding_by_rang_mod` (id,db_nm) VALUES (80006, database()); INSERT INTO `sharding_by_rang_mod` (id,db_nm) VALUES (60008, database()); select * from sharding_by_rang_mod; Query OK, 0 rows affected (0.58 sec) mysql> INSERT INTO `sharding_by_rang_mod` (id,db_nm) VALUES (1000, database()); Query OK, 1 row affected (0.08 sec) mysql> INSERT INTO `sharding_by_rang_mod` (id,db_nm) VALUES (1002, database()); Query OK, 1 row affected (0.00 sec) mysql> INSERT INTO `sharding_by_rang_mod` (id,db_nm) VALUES (30000, database()); Query OK, 1 row affected (0.00 sec) mysql> INSERT INTO `sharding_by_rang_mod` (id,db_nm) VALUES (30004, database()); Query OK, 1 row affected (0.01 sec) mysql> INSERT INTO `sharding_by_rang_mod` (id,db_nm) VALUES (40000, database()); Query OK, 1 row affected (0.00 sec) mysql> INSERT INTO `sharding_by_rang_mod` (id,db_nm) VALUES (40005, database()); Query OK, 1 row affected (0.01 sec) mysql> INSERT INTO `sharding_by_rang_mod` (id,db_nm) VALUES (60005, database()); Query OK, 1 row affected (0.00 sec) mysql> INSERT INTO `sharding_by_rang_mod` (id,db_nm) VALUES (60006, database()); Query OK, 1 row affected (0.01 sec) mysql> INSERT INTO `sharding_by_rang_mod` (id,db_nm) VALUES (80006, database()); Query OK, 1 row affected (0.00 sec) mysql> INSERT INTO `sharding_by_rang_mod` (id,db_nm) VALUES (60008, database()); Query OK, 1 row affected (0.00 sec) mysql> select * from sharding_by_rang_mod; +-------+-------+ | id | db_nm | +-------+-------+ | 1002 | db0 | | 80006 | db0 | | 30000 | db3 | | 30004 | db3 | | 40000 | db3 | | 1000 | db1 | | 40005 | db5 | | 60005 | db9 | | 60006 | db8 | | 60008 | db8 | +-------+-------+ 10 rows in set (0.12 sec) mysql>
注意
* 分片數量必須**小於等於**dataNode數
3.2 日期範圍hash分片
其思想與範圍求模一致，因爲日期取模方法會出現數據熱點問題，因此先根據日期分組，再根據時間hash使得短時間內數據分佈得更均勻。其優勢是能夠避免擴容時的數據遷移，又能夠在必定程度上避免範圍分片的熱點問題，要求日期格式儘可能精確，否則達不到局部均勻的目的。
vim conf/schema.xml
<?xml version="1.0"?> <!DOCTYPE mycat:schema SYSTEM "schema.dtd"> <mycat:schema xmlns:mycat="http://io.mycat/"> <schema name="test" checkSQLschema="false" sqlMaxLimit="100"> <table name="sharding_by_rang_mod" primaryKey="id" dataNode="dn$0-10" rule="sharding-by-rang-mod" /> </schema> <dataNode name="dn0" dataHost="dh-1" database="db0"/> <dataNode name="dn1" dataHost="dh-1" database="db1"/> <dataNode name="dn2" dataHost="dh-1" database="db2"/> <dataNode name="dn3" dataHost="dh-1" database="db3"/> <dataNode name="dn4" dataHost="dh-1" database="db4"/> <dataNode name="dn5" dataHost="dh-1" database="db5"/> <dataNode name="dn6" dataHost="dh-1" database="db6"/> <dataNode name="dn7" dataHost="dh-1" database="db7"/> <dataNode name="dn8" dataHost="dh-1" database="db8"/> <dataNode name="dn9" dataHost="dh-1" database="db9"/> <dataNode name="dn10" dataHost="dh-1" database="db10"/> <dataHost name="dh-1" maxCon="1000" minCon="10" balance="0" writeType="0" dbType="mysql" dbDriver="native" switchType="1" slaveThreshold="100"> <heartbeat>select user()</heartbeat> <writeHost host="hostM1" url="localhost:3306" user="root" password="123456"> </writeHost> </dataHost> </mycat:schema>
vim conf/rule.xml
<?xml version="1.0" encoding="UTF-8"?> <!DOCTYPE mycat:rule SYSTEM "rule.dtd"> <mycat:rule xmlns:mycat="http://io.mycat/"> <tableRule name="sharding-by-rang-mod"> <rule> <columns>id</columns> <algorithm>sharding-by-rang-mod</algorithm> </rule> </tableRule> <function name="sharding-by-rang-mod" class="io.mycat.route.function.PartitionByRangeMod"> <property name="mapFile">partition-range-mod.txt</property> <property name="defaultNode">0</property> </function> </mycat:rule>
配置說明：
* columns標識將要分片的表字段。
* algorithm爲分片函數。
* mapFile爲指定分片規則的配置文件。
* defaultNode爲未包含以上規則的數據存儲節點，節點從0開始。
vim conf/partition-range-mod.txt
# range start-end ,data node group size 0-200M=3 200M1-400M=1 400M1-600M=4 600M1-800M=2
測試SQL：
CREATE TABLE sharding_by_rang_mod (id bigint null,`db_nm` varchar(20) NULL); INSERT INTO `sharding_by_rang_mod` (id,db_nm) VALUES (1000, database()); INSERT INTO `sharding_by_rang_mod` (id,db_nm) VALUES (1002, database()); INSERT INTO `sharding_by_rang_mod` (id,db_nm) VALUES (30000, database()); INSERT INTO `sharding_by_rang_mod` (id,db_nm) VALUES (30004, database()); INSERT INTO `sharding_by_rang_mod` (id,db_nm) VALUES (40000, database()); INSERT INTO `sharding_by_rang_mod` (id,db_nm) VALUES (40005, database()); INSERT INTO `sharding_by_rang_mod` (id,db_nm) VALUES (60005, database()); INSERT INTO `sharding_by_rang_mod` (id,db_nm) VALUES (60006, database()); INSERT INTO `sharding_by_rang_mod` (id,db_nm) VALUES (80006, database()); INSERT INTO `sharding_by_rang_mod` (id,db_nm) VALUES (60008, database()); select * from sharding_by_rang_mod;
測試結果：
[root@testA conf]# mysql -uroot -p123456 -h 127.0.0.1 -P8066 test mysql: [Warning] Using a password on the command line interface can be insecure. Welcome to the MySQL monitor. Commands end with ; or \g. Your MySQL connection id is 1 Server version: 5.6.29-mycat-1.6-RELEASE-20161028204710 MyCat Server (OpenCloundDB) Copyright (c) 2000, 2017, Oracle and/or its affiliates. All rights reserved. Oracle is a registered trademark of Oracle Corporation and/or its affiliates. Other names may be trademarks of their respective owners. Type 'help;' or '\h' for help. Type '\c' to clear the current input statement. mysql> CREATE TABLE sharding_by_rang_mod (id bigint null,`db_nm` varchar(20) NULL); INSERT INTO `sharding_by_rang_mod` (id,db_nm) VALUES (1000, database()); INSERT INTO `sharding_by_rang_mod` (id,db_nm) VALUES (1002, database()); INSERT INTO `sharding_by_rang_mod` (id,db_nm) VALUES (30000, database()); INSERT INTO `sharding_by_rang_mod` (id,db_nm) VALUES (30004, database()); INSERT INTO `sharding_by_rang_mod` (id,db_nm) VALUES (40000, database()); INSERT INTO `sharding_by_rang_mod` (id,db_nm) VALUES (40005, database()); INSERT INTO `sharding_by_rang_mod` (id,db_nm) VALUES (60005, database()); INSERT INTO `sharding_by_rang_mod` (id,db_nm) VALUES (60006, database()); INSERT INTO `sharding_by_rang_mod` (id,db_nm) VALUES (80006, database()); INSERT INTO `sharding_by_rang_mod` (id,db_nm) VALUES (60008, database()); select * from sharding_by_rang_mod; Query OK, 0 rows affected (0.58 sec) mysql> INSERT INTO `sharding_by_rang_mod` (id,db_nm) VALUES (1000, database()); Query OK, 1 row affected (0.08 sec) mysql> INSERT INTO `sharding_by_rang_mod` (id,db_nm) VALUES (1002, database()); Query OK, 1 row affected (0.00 sec) mysql> INSERT INTO `sharding_by_rang_mod` (id,db_nm) VALUES (30000, database()); Query OK, 1 row affected (0.00 sec) mysql> INSERT INTO `sharding_by_rang_mod` (id,db_nm) VALUES (30004, database()); Query OK, 1 row affected (0.01 sec) mysql> INSERT INTO `sharding_by_rang_mod` (id,db_nm) VALUES (40000, database()); Query OK, 1 row affected (0.00 sec) mysql> INSERT INTO `sharding_by_rang_mod` (id,db_nm) VALUES (40005, database()); Query OK, 1 row affected (0.01 sec) mysql> INSERT INTO `sharding_by_rang_mod` (id,db_nm) VALUES (60005, database()); Query OK, 1 row affected (0.00 sec) mysql> INSERT INTO `sharding_by_rang_mod` (id,db_nm) VALUES (60006, database()); Query OK, 1 row affected (0.01 sec) mysql> INSERT INTO `sharding_by_rang_mod` (id,db_nm) VALUES (80006, database()); Query OK, 1 row affected (0.00 sec) mysql> INSERT INTO `sharding_by_rang_mod` (id,db_nm) VALUES (60008, database()); Query OK, 1 row affected (0.00 sec) mysql> select * from sharding_by_rang_mod; +-------+-------+ | id | db_nm | +-------+-------+ | 1002 | db0 | | 80006 | db0 | | 30000 | db3 | | 30004 | db3 | | 40000 | db3 | | 1000 | db1 | | 40005 | db5 | | 60005 | db9 | | 60006 | db8 | | 60008 | db8 | +-------+-------+ 10 rows in set (0.12 sec) mysql>
注意
* 分片數量必須**小於等於**dataNode數
3.3 取模範圍約束分片
取模運算與範圍約束的結合，主要爲了後續數據遷移作準備，便可以自主決定取模後數據的節點分佈。
vim conf/schema.xml
<?xml version="1.0"?> <!DOCTYPE mycat:schema SYSTEM "schema.dtd"> <mycat:schema xmlns:mycat="http://io.mycat/"> <schema name="test" checkSQLschema="false" sqlMaxLimit="100"> <table name="sharding_by_pattern" primaryKey="id" dataNode="dn$0-10" rule="sharding-by-pattern" /> </schema> <dataNode name="dn0" dataHost="dh-1" database="db0"/> <dataNode name="dn1" dataHost="dh-1" database="db1"/> <dataNode name="dn2" dataHost="dh-1" database="db2"/> <dataNode name="dn3" dataHost="dh-1" database="db3"/> <dataNode name="dn4" dataHost="dh-1" database="db4"/> <dataNode name="dn5" dataHost="dh-1" database="db5"/> <dataNode name="dn6" dataHost="dh-1" database="db6"/> <dataNode name="dn7" dataHost="dh-1" database="db7"/> <dataNode name="dn8" dataHost="dh-1" database="db8"/> <dataNode name="dn9" dataHost="dh-1" database="db9"/> <dataNode name="dn10" dataHost="dh-1" database="db10"/> <dataHost name="dh-1" maxCon="1000" minCon="10" balance="0" writeType="0" dbType="mysql" dbDriver="native" switchType="1" slaveThreshold="100"> <heartbeat>select user()</heartbeat> <writeHost host="hostM1" url="localhost:3306" user="root" password="123456"> </writeHost> </dataHost> </mycat:schema>
vim conf/rule.xml
<?xml version="1.0" encoding="UTF-8"?> <!DOCTYPE mycat:rule SYSTEM "rule.dtd"> <mycat:rule xmlns:mycat="http://io.mycat/"> <tableRule name="sharding-by-pattern"> <rule> <columns>id</columns> <algorithm>sharding-by-pattern</algorithm> </rule> </tableRule> <function name="sharding-by-pattern" class="io.mycat.route.function.PartitionByPattern"> <property name="mapFile">partition-pattern.txt</property> <property name="defaultNode">0</property> <property name="patternValue">256</property> </function> </mycat:rule>
配置說明：
* columns標識將要分片的表字段。
* algorithm爲分片函數。
* mapFile爲指定分片規則的配置文件。
* defaultNode爲未包含以上規則的數據存儲節點，節點從0開始。
* patternValue爲求模基數。
vim conf/partition-pattern.txt
# id patition range start-end, data node index ## first host configuration 1-32=0 33-64=1 65-96=2 97-128=3 ## second host configuration 129-160=4 161-192=5 193-224=6 225-256=7 0-0=7
配置說明：
* 1-32即表明id%256後分布的範圍，若是在1-32則分區在0，其餘類推。
* 若是id非數字，則會分配在defaultNode默認節點。
測試SQL：
CREATE TABLE sharding_by_pattern (id varchar(20) null,`db_nm` varchar(20) NULL); INSERT INTO `sharding_by_pattern` (id,db_nm) VALUES (1000, database()); INSERT INTO `sharding_by_pattern` (id,db_nm) VALUES (1002, database()); INSERT INTO `sharding_by_pattern` (id,db_nm) VALUES (30000, database()); INSERT INTO `sharding_by_pattern` (id,db_nm) VALUES (30004, database()); INSERT INTO `sharding_by_pattern` (id,db_nm) VALUES (40000, database()); INSERT INTO `sharding_by_pattern` (id,db_nm) VALUES (40005, database()); INSERT INTO `sharding_by_pattern` (id,db_nm) VALUES (60005, database()); INSERT INTO `sharding_by_pattern` (id,db_nm) VALUES (60006, database()); INSERT INTO `sharding_by_pattern` (id,db_nm) VALUES (80006, database()); INSERT INTO `sharding_by_pattern` (id,db_nm) VALUES ("abcd0", database()); select * from sharding_by_pattern;
測試結果：
[root@testA conf]# mysql -uroot -p123456 -h 127.0.0.1 -P8066 test mysql: [Warning] Using a password on the command line interface can be insecure. Welcome to the MySQL monitor. Commands end with ; or \g. Your MySQL connection id is 1 Server version: 5.6.29-mycat-1.6-RELEASE-20161028204710 MyCat Server (OpenCloundDB) Copyright (c) 2000, 2017, Oracle and/or its affiliates. All rights reserved. Oracle is a registered trademark of Oracle Corporation and/or its affiliates. Other names may be trademarks of their respective owners. Type 'help;' or '\h' for help. Type '\c' to clear the current input statement. mysql> CREATE TABLE sharding_by_pattern (id varchar(20) null,`db_nm` varchar(20) NULL); INSERT INTO `sharding_by_pattern` (id,db_nm) VALUES (1000, database()); INSERT INTO `sharding_by_pattern` (id,db_nm) VALUES (1002, database()); INSERT INTO `sharding_by_pattern` (id,db_nm) VALUES (30000, database()); INSERT INTO `sharding_by_pattern` (id,db_nm) VALUES (30004, database()); INSERT INTO `sharding_by_pattern` (id,db_nm) VALUES (40000, database()); INSERT INTO `sharding_by_pattern` (id,db_nm) VALUES (40005, database()); INSERT INTO `sharding_by_pattern` (id,db_nm) VALUES (60005, database()); INSERT INTO `sharding_by_pattern` (id,db_nm) VALUES (60006, database()); INSERT INTO `sharding_by_pattern` (id,db_nm) VALUES (80006, database()); INSERT INTO `sharding_by_pattern` (id,db_nm) VALUES ("abcd0", database()); select * from sharding_by_pattern; Query OK, 0 rows affected (0.72 sec) mysql> INSERT INTO `sharding_by_pattern` (id,db_nm) VALUES (1000, database()); Query OK, 1 row affected (0.08 sec) mysql> INSERT INTO `sharding_by_pattern` (id,db_nm) VALUES (1002, database()); Query OK, 1 row affected (0.01 sec) mysql> INSERT INTO `sharding_by_pattern` (id,db_nm) VALUES (30000, database()); Query OK, 1 row affected (0.00 sec) mysql> INSERT INTO `sharding_by_pattern` (id,db_nm) VALUES (30004, database()); Query OK, 1 row affected (0.00 sec) mysql> INSERT INTO `sharding_by_pattern` (id,db_nm) VALUES (40000, database()); Query OK, 1 row affected (0.01 sec) mysql> INSERT INTO `sharding_by_pattern` (id,db_nm) VALUES (40005, database()); Query OK, 1 row affected (0.00 sec) mysql> INSERT INTO `sharding_by_pattern` (id,db_nm) VALUES (60005, database()); Query OK, 1 row affected (0.00 sec) mysql> INSERT INTO `sharding_by_pattern` (id,db_nm) VALUES (60006, database()); Query OK, 1 row affected (0.01 sec) mysql> INSERT INTO `sharding_by_pattern` (id,db_nm) VALUES (80006, database()); Query OK, 1 row affected (0.00 sec) mysql> INSERT INTO `sharding_by_pattern` (id,db_nm) VALUES ("abcd0", database()); Query OK, 1 row affected (0.07 sec) mysql> select * from sharding_by_pattern; +-------+-------+ | id | db_nm | +-------+-------+ | 80006 | db4 | | 30000 | db1 | | 30004 | db1 | | 40000 | db1 | | 1000 | db7 | | 1002 | db7 | | 60005 | db3 | | 60006 | db3 | | 40005 | db2 | | abcd0 | db0 | +-------+-------+ 10 rows in set (0.14 sec) mysql>
注意
* 分片數量必須**小於等於**dataNode數
3.4 ASCII碼求模範圍約束分片（字符串）
此種規則相似於取模範圍約束，此規則支持數據符號字母取模。
vim conf/schema.xml
<?xml version="1.0"?> <!DOCTYPE mycat:schema SYSTEM "schema.dtd"> <mycat:schema xmlns:mycat="http://io.mycat/"> <schema name="test" checkSQLschema="false" sqlMaxLimit="100"> <table name="sharding_by_pattern" primaryKey="id" dataNode="dn$0-10" rule="sharding-by-pattern" /> </schema> <dataNode name="dn0" dataHost="dh-1" database="db0"/> <dataNode name="dn1" dataHost="dh-1" database="db1"/> <dataNode name="dn2" dataHost="dh-1" database="db2"/> <dataNode name="dn3" dataHost="dh-1" database="db3"/> <dataNode name="dn4" dataHost="dh-1" database="db4"/> <dataNode name="dn5" dataHost="dh-1" database="db5"/> <dataNode name="dn6" dataHost="dh-1" database="db6"/> <dataNode name="dn7" dataHost="dh-1" database="db7"/> <dataNode name="dn8" dataHost="dh-1" database="db8"/> <dataNode name="dn9" dataHost="dh-1" database="db9"/> <dataNode name="dn10" dataHost="dh-1" database="db10"/> <dataHost name="dh-1" maxCon="1000" minCon="10" balance="0" writeType="0" dbType="mysql" dbDriver="native" switchType="1" slaveThreshold="100"> <heartbeat>select user()</heartbeat> <writeHost host="hostM1" url="localhost:3306" user="root" password="123456"> </writeHost> </dataHost> </mycat:schema>
vim conf/rule.xml
<?xml version="1.0" encoding="UTF-8"?> <!DOCTYPE mycat:rule SYSTEM "rule.dtd"> <mycat:rule xmlns:mycat="http://io.mycat/"> <tableRule name="sharding-by-pattern"> <rule> <columns>id</columns> <algorithm>sharding-by-pattern</algorithm> </rule> </tableRule> <function name="sharding-by-pattern" class="io.mycat.route.function.PartitionByPattern"> <property name="mapFile">partition-pattern.txt</property> <property name="defaultNode">0</property> <property name="patternValue">256</property> </function> </mycat:rule>
配置說明：
* columns標識將要分片的表字段。
* algorithm爲分片函數。
* mapFile爲指定分片規則的配置文件。
* defaultNode爲未包含以上規則的數據存儲節點，節點從0開始。
* patternValue爲求模基數。
vim conf/partition-pattern.txt
# id patition range start-end, data node index ## first host configuration 1-32=0 33-64=1 65-96=2 97-128=3 ## second host configuration 129-160=4 161-192=5 193-224=6 225-256=7 0-0=7
配置說明：
* 1-32即表明id%256後分布的範圍，若是在1-32則分區在0，其餘類推。
* 若是id非數字，則會分配在defaultNode默認節點。
測試SQL：
CREATE TABLE sharding_by_prefix_pattern (id varchar(20) null,`db_nm` varchar(20) NULL); INSERT INTO `sharding_by_prefix_pattern` (id,db_nm) VALUES ("1000a", database()); INSERT INTO `sharding_by_prefix_pattern` (id,db_nm) VALUES ("1002A", database()); INSERT INTO `sharding_by_prefix_pattern` (id,db_nm) VALUES (30000, database()); INSERT INTO `sharding_by_prefix_pattern` (id,db_nm) VALUES (30004, database()); INSERT INTO `sharding_by_prefix_pattern` (id,db_nm) VALUES ("4000B", database()); INSERT INTO `sharding_by_prefix_pattern` (id,db_nm) VALUES ("4000b", database()); INSERT INTO `sharding_by_prefix_pattern` (id,db_nm) VALUES (60007, database()); INSERT INTO `sharding_by_prefix_pattern` (id,db_nm) VALUES (60006, database()); INSERT INTO `sharding_by_prefix_pattern` (id,db_nm) VALUES (80006, database()); INSERT INTO `sharding_by_prefix_pattern` (id,db_nm) VALUES ("abcd0", database()); select * from sharding_by_prefix_pattern;
測試結果：
[root@testA conf]# mysql -uroot -p123456 -h 127.0.0.1 -P8066 test mysql: [Warning] Using a password on the command line interface can be insecure. Welcome to the MySQL monitor. Commands end with ; or \g. Your MySQL connection id is 4 Server version: 5.6.29-mycat-1.6-RELEASE-20161028204710 MyCat Server (OpenCloundDB) Copyright (c) 2000, 2017, Oracle and/or its affiliates. All rights reserved. Oracle is a registered trademark of Oracle Corporation and/or its affiliates. Other names may be trademarks of their respective owners. Type 'help;' or '\h' for help. Type '\c' to clear the current input statement. mysql> CREATE TABLE sharding_by_prefix_pattern (id varchar(20) null,`db_nm` varchar(20) NULL); INSERT INTO `sharding_by_prefix_pattern` (id,db_nm) VALUES ("1000a", database()); INSERT INTO `sharding_by_prefix_pattern` (id,db_nm) VALUES ("1002A", database()); INSERT INTO `sharding_by_prefix_pattern` (id,db_nm) VALUES (30000, database()); INSERT INTO `sharding_by_prefix_pattern` (id,db_nm) VALUES (30004, database()); INSERT INTO `sharding_by_prefix_pattern` (id,db_nm) VALUES ("4000B", database()); INSERT INTO `sharding_by_prefix_pattern` (id,db_nm) VALUES ("4000b", database()); INSERT INTO `sharding_by_prefix_pattern` (id,db_nm) VALUES (60007, database()); INSERT INTO `sharding_by_prefix_pattern` (id,db_nm) VALUES (60006, database()); INSERT INTO `sharding_by_prefix_pattern` (id,db_nm) VALUES (80006, database()); INSERT INTO `sharding_by_prefix_pattern` (id,db_nm) VALUES ("abcd0", database()); select * from sharding_by_prefix_pattern; Query OK, 0 rows affected (0.27 sec) mysql> INSERT INTO `sharding_by_prefix_pattern` (id,db_nm) VALUES ("1000a", database()); Query OK, 1 row affected (0.03 sec) mysql> INSERT INTO `sharding_by_prefix_pattern` (id,db_nm) VALUES ("1002A", database()); Query OK, 1 row affected (0.01 sec) mysql> INSERT INTO `sharding_by_prefix_pattern` (id,db_nm) VALUES (30000, database()); Query OK, 1 row affected (0.00 sec) mysql> INSERT INTO `sharding_by_prefix_pattern` (id,db_nm) VALUES (30004, database()); Query OK, 1 row affected (0.01 sec) mysql> INSERT INTO `sharding_by_prefix_pattern` (id,db_nm) VALUES ("4000B", database()); Query OK, 1 row affected (0.00 sec) mysql> INSERT INTO `sharding_by_prefix_pattern` (id,db_nm) VALUES ("4000b", database()); Query OK, 1 row affected (0.00 sec) mysql> INSERT INTO `sharding_by_prefix_pattern` (id,db_nm) VALUES (60007, database()); Query OK, 1 row affected (0.01 sec) mysql> INSERT INTO `sharding_by_prefix_pattern` (id,db_nm) VALUES (60006, database()); Query OK, 1 row affected (0.00 sec) mysql> INSERT INTO `sharding_by_prefix_pattern` (id,db_nm) VALUES (80006, database()); Query OK, 1 row affected (0.00 sec) mysql> INSERT INTO `sharding_by_prefix_pattern` (id,db_nm) VALUES ("abcd0", database()); Query OK, 1 row affected (0.01 sec) mysql> select * from sharding_by_prefix_pattern; +-------+-------+ | id | db_nm | +-------+-------+ | 1000a | db3 | | 1002A | db3 | | 30000 | db3 | | 30004 | db3 | | 4000B | db3 | | 4000b | db3 | | 60007 | db3 | | 60006 | db3 | | 80006 | db3 | | abcd0 | db6 | +-------+-------+ 10 rows in set (0.01 sec) mysql>
注意
* 分片數量必須**小於等於**dataNode數
3.5 固定分片hash算法分片（二進制）
本條規則相似於十進制的求模運算，區別在因而二進制的操做，是取id的二進制低10位，即id二進制&1111111111。此算法的優勢在於若是按照十進制取模運算，則在連續插入1~10時，1~10會被分到1~10個分片，增大了插入事務的控制難度。而此算法根據二進制則可能會分到鏈接的分片，下降了插入事務的控制難度。
vim conf/schema.xml
<?xml version="1.0"?> <!DOCTYPE mycat:schema SYSTEM "schema.dtd"> <mycat:schema xmlns:mycat="http://io.mycat/"> <schema name="test" checkSQLschema="false" sqlMaxLimit="100"> <table name="sharding_by_long" primaryKey="id" dataNode="dn$0-10" rule="sharding-by-long" /> </schema> <dataNode name="dn0" dataHost="dh-1" database="db0"/> <dataNode name="dn1" dataHost="dh-1" database="db1"/> <dataNode name="dn2" dataHost="dh-1" database="db2"/> <dataNode name="dn3" dataHost="dh-1" database="db3"/> <dataNode name="dn4" dataHost="dh-1" database="db4"/> <dataNode name="dn5" dataHost="dh-1" database="db5"/> <dataNode name="dn6" dataHost="dh-1" database="db6"/> <dataNode name="dn7" dataHost="dh-1" database="db7"/> <dataNode name="dn8" dataHost="dh-1" database="db8"/> <dataNode name="dn9" dataHost="dh-1" database="db9"/> <dataNode name="dn10" dataHost="dh-1" database="db10"/> <dataHost name="dh-1" maxCon="1000" minCon="10" balance="0" writeType="0" dbType="mysql" dbDriver="native" switchType="1" slaveThreshold="100"> <heartbeat>select user()</heartbeat> <writeHost host="hostM1" url="localhost:3306" user="root" password="123456"> </writeHost> </dataHost> </mycat:schema>
vim conf/rule.xml
<?xml version="1.0" encoding="UTF-8"?> <!DOCTYPE mycat:rule SYSTEM "rule.dtd"> <mycat:rule xmlns:mycat="http://io.mycat/"> <tableRule name="sharding-by-long"> <rule> <columns>id</columns> <algorithm>sharding-by-long</algorithm> </rule> </tableRule> <function name="sharding-by-long" class="io.mycat.route.function.PartitionByLong"> <property name="partitionCount">2,1</property> <property name="partitionLength">256,512</property> </function> </mycat:rule>
配置說明：
* columns標識將要分片的表字段。
* algorithm爲分片函數。
* partitionCount爲指定分片個數列表。
* partitionLength爲分片範圍列表。
分區長度:默認爲最大2^n=1024 ,即最大支持1024分區
約束 :
count,length兩個數組的長度必須是一致的。
1024 = sum((count[i]*length[i])). count和length兩個向量的點積恆等於1024
本例的分區策略：但願將數據水平分紅3份，前兩份各佔25%，第三份佔50%。（故本例非均勻分區）
// |<———————1024———————————>|
// |<—-256—>|<—-256—>|<———-512————->|
// | partition0 | partition1 | partition2 |
// | 共2份,故count[0]=2 | 共1份，故count[1]=1 |
int[] count = new int[] { 2, 1 };
int[] length = new int[] { 256, 512 };
vim conf/partition-pattern.txt
測試SQL：
CREATE TABLE sharding_by_long (id int(10) null,`db_nm` varchar(20) NULL); INSERT INTO `sharding_by_long` (id,db_nm) VALUES (1000, database()); INSERT INTO `sharding_by_long` (id,db_nm) VALUES (1002, database()); INSERT INTO `sharding_by_long` (id,db_nm) VALUES (30000, database()); INSERT INTO `sharding_by_long` (id,db_nm) VALUES (30004, database()); INSERT INTO `sharding_by_long` (id,db_nm) VALUES (4000, database()); INSERT INTO `sharding_by_long` (id,db_nm) VALUES (4000, database()); INSERT INTO `sharding_by_long` (id,db_nm) VALUES (60007, database()); INSERT INTO `sharding_by_long` (id,db_nm) VALUES (60006, database()); INSERT INTO `sharding_by_long` (id,db_nm) VALUES (80006, database()); INSERT INTO `sharding_by_long` (id,db_nm) VALUES (0, database()); select * from sharding_by_long;
測試結果：
[root@testA conf]# mysql -uroot -p123456 -h 127.0.0.1 -P8066 test mysql: [Warning] Using a password on the command line interface can be insecure. Welcome to the MySQL monitor. Commands end with ; or \g. Your MySQL connection id is 4 Server version: 5.6.29-mycat-1.6-RELEASE-20161028204710 MyCat Server (OpenCloundDB) Copyright (c) 2000, 2017, Oracle and/or its affiliates. All rights reserved. Oracle is a registered trademark of Oracle Corporation and/or its affiliates. Other names may be trademarks of their respective owners. Type 'help;' or '\h' for help. Type '\c' to clear the current input statement. mysql> CREATE TABLE sharding_by_long (id int(10) null,`db_nm` varchar(20) NULL); INSERT INTO `sharding_by_long` (id,db_nm) VALUES (1000, database()); INSERT INTO `sharding_by_long` (id,db_nm) VALUES (1002, database()); INSERT INTO `sharding_by_long` (id,db_nm) VALUES (30000, database()); INSERT INTO `sharding_by_long` (id,db_nm) VALUES (30004, database()); INSERT INTO `sharding_by_long` (id,db_nm) VALUES (4000, database()); INSERT INTO `sharding_by_long` (id,db_nm) VALUES (4000, database()); INSERT INTO `sharding_by_long` (id,db_nm) VALUES (60007, database()); INSERT INTO `sharding_by_long` (id,db_nm) VALUES (60006, database()); INSERT INTO `sharding_by_long` (id,db_nm) VALUES (80006, database()); INSERT INTO `sharding_by_long` (id,db_nm) VALUES (0, database()); select * from sharding_by_long; Query OK, 0 rows affected (0.22 sec) mysql> INSERT INTO `sharding_by_long` (id,db_nm) VALUES (1000, database()); Query OK, 1 row affected (0.00 sec) mysql> INSERT INTO `sharding_by_long` (id,db_nm) VALUES (1002, database()); Query OK, 1 row affected (0.00 sec) mysql> INSERT INTO `sharding_by_long` (id,db_nm) VALUES (30000, database()); Query OK, 1 row affected (0.04 sec) mysql> INSERT INTO `sharding_by_long` (id,db_nm) VALUES (30004, database()); Query OK, 1 row affected (0.00 sec) mysql> INSERT INTO `sharding_by_long` (id,db_nm) VALUES (4000, database()); Query OK, 1 row affected (0.01 sec) mysql> INSERT INTO `sharding_by_long` (id,db_nm) VALUES (4000, database()); Query OK, 1 row affected (0.00 sec) mysql> INSERT INTO `sharding_by_long` (id,db_nm) VALUES (60007, database()); Query OK, 1 row affected (0.00 sec) mysql> INSERT INTO `sharding_by_long` (id,db_nm) VALUES (60006, database()); Query OK, 1 row affected (0.01 sec) mysql> INSERT INTO `sharding_by_long` (id,db_nm) VALUES (80006, database()); Query OK, 1 row affected (0.00 sec) mysql> INSERT INTO `sharding_by_long` (id,db_nm) VALUES (0, database()); Query OK, 1 row affected (0.00 sec) mysql> select * from sharding_by_long; +-------+-------+ | id | db_nm | +-------+-------+ | 80006 | db0 | | 0 | db0 | | 30000 | db1 | | 30004 | db1 | | 1000 | db2 | | 1002 | db2 | | 4000 | db2 | | 4000 | db2 | | 60007 | db2 | | 60006 | db2 | +-------+-------+ 10 rows in set (0.01 sec) mysql>
注意
* 分片數量必須**小於等於**dataNode數
4. Mycat分片場景和策略
- 根據表數據量判斷是否須要切分，確保切分後單分片表數據量爲1000W左右
- 根據業務的狀況選擇合適的分片字段： 最頻繁的或者最重要的查詢條件
- 須要考慮擴容數據遷移問題（範圍類，範圍取模類不須要遷移，哈希類須要遷移）
- 有關聯關係的表配置相同分片規則（ER思想，爲了應用join等複雜sql），一對多對應關係通常按多的那一方切分
- 若是配置類數據， 更新頻率比較少，考慮全局表
- 參考資料：
- [1] http://mycat.io/
相關文章
- 1. MyCat分片策略
- 2. sharding （五） Hint 表達式分片策略
- 3. sharding-jdbc分片策略
- 4. Sharding-JDBC分片策略詳解（二）
- 5. 分庫分表 sharding-jdbc—分片策略
- 6. 【分庫分表】sharding-jdbc—分片策略
- 7. mycat 分片
- 8. mycat-ER分片
- 9. 數據庫系列課程（05）-MyCat分庫分表策略
- 10. Mycat的分片join
- 更多相關文章...
- • Redis內存回收策略 - Redis教程
- • 二級緩存的併發訪問策略和常用插件 - Hibernate教程
- • Git五分鐘教程
- • 算法總結-二分查找法
相關標籤/搜索
每日一句
-
每一个你不满意的现在，都有一个你没有努力的曾经。
歡迎關注本站公眾號,獲取更多信息
相關文章