1、Sharding-JDBC依賴java
2、分片策略mysql
1. 標準分片策略git
2. 複合分片策略github
3. Inline表達式分片策略算法
4. 經過Hint而非SQL解析的方式分片的策略spring
5. 不分片的策略sql
3、分片鍵數據庫
4、分片算法express
1. 精確分片算法apache
2. 範圍分片算法
3. 複合分片算法
4. Hint分片算法
5、代碼實踐
上一篇博文以Sharding-JDBC爲契機,瞭解了ShardingSphere你們族的全貌概覽,以及Sharding-JDBC的角色定位。
Sharding-JDBC定位爲輕量級Java框架,在Java的JDBC層提供的額外服務。 它使用客戶端直連數據庫,以jar包形式提供服務,無需額外部署和依賴,可理解爲加強版的JDBC驅動,徹底兼容JDBC和各類ORM框架。
Sharding-JDBC目前已經迭代到4.0.0版本,而且進入了Apache孵化器,但Sharding-JDBC各個大版本之間不支持向下兼容(這裏主要描述的是2.x到4.x),大版本和大版本之間猶如斷層,若是涉及到升級,就須要改動代碼,而且2.x到4.x多多少少都有點BUG,致使在業務系統引進Sharding-JDBC的過程略顯艱難。
接下來將介紹Sharding-JDBC的2.0.3版本,以及它的長處和短板。
<!-- sharding-jdbc-core --> <dependency> <groupId>io.shardingjdbc</groupId> <artifactId>sharding-jdbc-core</artifactId> <version>2.0.3</version> </dependency> <!-- sharding-jdbc-spring-namespace --> <dependency> <groupId>io.shardingjdbc</groupId> <artifactId>sharding-jdbc-core-spring-namespace</artifactId> <version>2.0.3</version> </dependency>
Sharding-JDBC沒有將Spring Schema上傳至公網,需引入namespace依賴包。
包含分片鍵和分片算法,因爲分片算法的獨立性,將其獨立抽離。真正可用於分片操做的是分片鍵 + 分片算法,也就是分片策略。目前提供5種分片策略:標準分片策略、複合分片策略、Inline表達式分片策略、經過Hint而非SQL解析的方式分片的策略和不分片的策略。
對應StandardShardingStrategy。提供對SQL語句中的=、IN和BETWEEN AND的分片操做支持。StandardShardingStrategy只支持單分片鍵,提供PreciseShardingAlgorithm和RangeShardingAlgorithm兩個分片算法。PreciseShardingAlgorithm是必選的,用於處理=和IN的分片。RangeShardingAlgorithm是可選的,用於處理BETWEEN AND分片,若是不配置RangeShardingAlgorithm,SQL中的BETWEEN AND將按照全庫路由處理。
對應ComplexShardingStrategy。複合分片策略。提供對SQL語句中的=、IN和BETWEEN AND的分片操做支持。ComplexShardingStrategy支持多分片鍵,因爲多分片鍵之間的關係複雜,所以並未進行過多的封裝,而是直接將分片鍵值組合以及分片操做符透傳至分片算法,徹底由應用開發者實現,提供最大的靈活度。
對應InlineShardingStrategy。使用Groovy的表達式,提供對SQL語句中的=和IN的分片操做支持,只支持單分片鍵。對於簡單的分片算法,能夠經過簡單的配置使用,從而避免繁瑣的Java代碼開發,如 t_user_$->{u_id % 8} 表示t_user表根據u_id模8,而分紅8張表,表名稱爲 t_user_0 到 t_user_7 。
對應HintShardingStrategy。經過Hint而非SQL解析的方式分片的策略。
對應NoneShardingStrategy。不分片的策略。
標準分片策略、複合分片策略和Inline表達式分片策略要求分片鍵(即分片字段)必須存在於SQL中和數據表結構中,不然沒法正確分片;相反,若分片鍵不存在於SQL中和數據表結構中,則可使用Hint方式進行強制路由。
用於分片的數據庫字段,是將數據庫(表)水平拆分的關鍵字段。例:將訂單表中的訂單主鍵的尾數取模分片,則訂單主鍵爲分片字段。 SQL中若是無分片字段,將執行全路由,性能較差。 除了對單分片字段的支持,ShardingSphere也支持根據多個字段進行分片。
經過分片算法將數據分片,支持經過=、IN和BETWEEN AND分片。分片算法須要應用方開發者自行實現,可實現的靈活度很是高。
目前提供4種分片算法。因爲分片算法和業務實現緊密相關,所以並未提供內置分片算法,而是經過分片策略將各類場景提煉出來,提供更高層級的抽象,並提供接口讓應用開發者自行實現分片算法。
對應PreciseShardingAlgorithm,用於處理使用單一鍵做爲分片鍵的=與IN進行分片的場景。須要配合StandardShardingStrategy使用。
對應RangeShardingAlgorithm,用於處理使用單一鍵做爲分片鍵的BETWEEN AND進行分片的場景。須要配合StandardShardingStrategy使用。
對應ComplexKeysShardingAlgorithm,用於處理使用多鍵做爲分片鍵進行分片的場景,包含多個分片鍵的邏輯較複雜,須要應用開發者自行處理其中的複雜度。須要配合ComplexShardingStrategy使用。
對應HintShardingAlgorithm,用於處理使用Hint行分片的場景。須要配合HintShardingStrategy使用。
在正式介紹代碼以前,有必要先描述下業務背景,便於理解代碼的用途。
數據表結構:
CREATE TABLE `settlement_20180403` ( `id` bigint(20) NOT NULL AUTO_INCREMENT, `pay_serial_number` varchar(32) DEFAULT NULL COMMENT '支付流水號', `user_mobile` varchar(15) DEFAULT NULL COMMENT '用戶手機號', `money` int(10) DEFAULT '0' COMMENT '支付或退款金額, 單位: 分', `operate_tm` timestamp NULL DEFAULT NULL COMMENT '操做時間', `refund` tinyint(1) DEFAULT NULL COMMENT '是否退款 1: 是 2: 否', `remark` varchar(32) DEFAULT NULL COMMENT '備註', PRIMARY KEY (`id`), UNIQUE KEY `PAY_SERIAL_NUM_UNIQUE` (`pay_serial_number`) ) ENGINE=InnoDB DEFAULT CHARSET=utf8;
表結構中比較有趣的字段是 pay_serial_number ,以 232042018040353088 爲例, pay_serial_number 由三部分組成:前5位是固定編碼,樣例中是23204,中間8位是日期,包含年月日,樣例中是20180403,最後5位是隨機數,樣例中是53088,以上三部分組成了支付流水號。
隨着業務量逐漸擴增, settlement 表數據量急劇,須要以時間做爲分表條件水平切分數據,分表字段則爲 pay_serial_number ,其中的日期則爲分表條件。接下來的代碼都是以此業務背景展開。
以下代碼配置了標準分片策略中的精確分片算法PreciseShardingAlgorithm和Hint分片算法HintShardingAlgorithm。
XML配置:
<?xml version="1.0" encoding="UTF-8"?> <beans xmlns="http://www.springframework.org/schema/beans" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:tx="http://www.springframework.org/schema/tx" xmlns:sharding="http://shardingjdbc.io/schema/shardingjdbc/sharding" xsi:schemaLocation="http://www.springframework.org/schema/beans http://www.springframework.org/schema/beans/spring-beans.xsd http://www.springframework.org/schema/tx http://www.springframework.org/schema/tx/spring-tx.xsd http://shardingjdbc.io/schema/shardingjdbc/sharding http://shardingjdbc.io/schema/shardingjdbc/sharding/sharding.xsd"> <!-- 標準分片策略 --> <sharding:standard-strategy id="settlementTableShardingStandardStrategy" sharding-column="pay_serial_number" precise-algorithm-class="org.cellphone.finance.repo.sharding.PreciseTableShardingAlgorithm"/> <!-- 基於暗示(Hint)的數據分片策略 --> <sharding:hint-strategy id="settlementTableShardingHintStrategy" algorithm-class="org.cellphone.finance.repo.sharding.HintTableShardingAlgorithm"/> <!-- 加了分庫分表插件,spring boot 啓動會實例化兩個dataSource,不知是否正常? --> <sharding:data-source id="shardingDataSource"> <!-- dataSource是數據庫鏈接池 --> <sharding:sharding-rule data-source-names="dataSource"> <sharding:table-rules> <sharding:table-rule logic-table="settlement" table-strategy-ref="settlementTableShardingStandardStrategy"/> <!-- logic-table 必定要和SQL中的表名以及HintManager中配置的logicTable一致,不然沒法找到對應的table rule --> <sharding:table-rule logic-table="settlement_hint" table-strategy-ref="settlementTableShardingHintStrategy"/> </sharding:table-rules> </sharding:sharding-rule> <sharding:props> <prop key="sql.show">true</prop> </sharding:props> </sharding:data-source> <bean id="transactionManager" class="org.springframework.jdbc.datasource.DataSourceTransactionManager"> <property name="dataSource" ref="shardingDataSource"/> </bean> <tx:annotation-driven/> </beans>
精確分片算法:
package org.cellphone.finance.repo.sharding; import com.google.common.collect.Lists; import io.shardingjdbc.core.api.algorithm.sharding.PreciseShardingValue; import io.shardingjdbc.core.api.algorithm.sharding.standard.PreciseShardingAlgorithm; import org.apache.commons.collections.CollectionUtils; import org.cellphone.common.constant.CommonConst; import java.util.Collection; /** * 精確分片算法,屬於標準分片算法,用於處理=和IN的分片 * <p> * 使用精確分片算法的前提:分片字段必須存在與SQL中、數據庫表結構中,不然沒法使用精確分片算法 * <p> * 此分片算法應用於SETTLEMENT數據表,這裏是按天分表 * <p> * 特別說明:Sharding Jdbc版本:2.0.3 * <p> * Created by on 2018/4/9. */ public class PreciseTableShardingAlgorithm implements PreciseShardingAlgorithm<String> { /** * 精確分片算法 * * @param availableTargetNames 目標數據源名稱或數據表名稱,注意:是邏輯數據源名或邏輯數據表名,來自SQL * @param shardingValue 分片值,來自SQL中分片字段對應的值 * @return 真實數據源名稱或數據表名稱 */ @Override public String doSharding(final Collection<String> availableTargetNames, final PreciseShardingValue<String> shardingValue) { // 默認數據表名稱,有可能數據庫中不存在這張表 String tableName = "settlement"; // 邏輯表名爲空,返回默認表名 if (CollectionUtils.isEmpty(availableTargetNames)) return tableName; // availableTargetNames來自SQL,只有一個元素 tableName = Lists.newArrayList(availableTargetNames).get(0); String paySerialNumber = shardingValue.getValue(); String suffix = paySerialNumber.substring(5, 13); return tableName + CommonConst.UNDERLINE + suffix; } }
上面說過,精確分片算法能夠很完美的支持SQL語句中的=和IN的分片操做,所以PreciseTableShardingAlgorithm能夠很好的支持簡單的INSERT、UPDATE、DELETE和SELECT操做。
但若是要經過 pay_serial_number 查詢數據就行不通了,由於分片鍵並不直接存在與SQL中和數據表結構中,而是間接存在於 pay_serial_number 中的時間數據,所以,就須要經過Hint方式來進行分片才能定位到真實的數據表。
Hint分片算法:
package org.cellphone.finance.repo.sharding; import com.google.common.collect.Lists; import io.shardingjdbc.core.api.algorithm.sharding.ListShardingValue; import io.shardingjdbc.core.api.algorithm.sharding.ShardingValue; import io.shardingjdbc.core.api.algorithm.sharding.hint.HintShardingAlgorithm; import org.apache.commons.collections.CollectionUtils; import org.apache.commons.lang3.StringUtils; import org.apache.commons.lang3.time.DateFormatUtils; import org.apache.commons.lang3.time.DateUtils; import org.cellphone.common.constant.CommonConst; import org.cellphone.common.constant.DateConst; import java.text.ParseException; import java.util.*; /** * Created by on 2019/4/27. */ public class HintTableShardingAlgorithm implements HintShardingAlgorithm { @Override public Collection<String> doSharding(Collection<String> availableTargetNames, ShardingValue shardingValue) { // 數據表名稱列表 List<String> tableNames = new ArrayList<>(); // 邏輯表名爲空 if (CollectionUtils.isEmpty(availableTargetNames)) return tableNames; // availableTargetNames來自SQL,只有一個元素 String tableName = Lists.newArrayList(availableTargetNames).get(0); tableName = tableName.replace("_hint", StringUtils.EMPTY); ListShardingValue<String> listShardingValue = (ListShardingValue<String>) shardingValue; List<String> list = Lists.newArrayList(listShardingValue.getValues()); Date startTm, endTm; try { startTm = DateUtils.parseDate(list.get(0), DateConst.DATE_FORMAT_NORMAL); endTm = DateUtils.parseDate(list.get(1), DateConst.DATE_FORMAT_NORMAL); } catch (ParseException e) { return tableNames; } Calendar calendar = Calendar.getInstance(); while (startTm.getTime() <= endTm.getTime()) { String suffix = DateFormatUtils.format(startTm, DateConst.DATE_FORMAT_YYYY_MM_DD); tableNames.add(tableName + CommonConst.UNDERLINE + suffix); calendar.setTime(startTm); calendar.add(Calendar.DATE, 1); startTm = calendar.getTime(); } return tableNames; } }
與Hint分片算法對應的Java查詢方法 settlementMapper.selectByExample(example) :
public List<Settlement> querySettlements(SettlementExample example, String startTime, String endTime) { HintManager hintManager = HintManager.getInstance(); // logicTable必須和XML中的logic-table以及SQL中的邏輯表名稱一致 hintManager.addTableShardingValue("settlement_hint", "start_time", startTime); hintManager.addTableShardingValue("settlement_hint", "end_time", endTime); List<Settlement> settlements = settlementMapper.selectByExample(example); hintManager.close(); return settlements; }
以及該查詢方法對應的SQL語句:
select * from settlement_hint t where t.pay_serial_number = ?
單元測試代碼:
@Test public void test004QuerySettlements() throws ParseException { String startTime = "2018-04-03 00:00:00", endTime = "2018-04-05 00:00:00"; SettlementExample example = new SettlementExample(); SettlementExample.Criteria criteria = example.createCriteria(); criteria.andPaySerialNumberEqualTo("232042018040353088"); List<Settlement> settlements = repository.querySettlements(example, startTime, endTime); Assert.assertEquals("136********", settlements.get(0).getUserMobile()); }
通過上述一波代碼後,原覺得單元測試能夠完美經過,但天有不測風雲,月有陰晴圓缺,單元測試居然失敗了,報錯日誌提示數據表不存在:
org.springframework.jdbc.BadSqlGrammarException: ### Error querying database. Cause: java.sql.SQLSyntaxErrorException: Table 'warehouse.settlement_hint' doesn't exist ### The error may exist in file [D:\programs\IdeaProjects\warehouse\warehouse-finance\warehouse-finance-service\target\classes\mapper\SettlementMapper.xml] ### The error may involve org.cellphone.finance.repo.mapper.SettlementMapper.selectByExample-Inline ### The error occurred while setting parameters ### SQL: select id, pay_serial_number, user_mobile, money, operate_tm, refund, remark from settlement_hint WHERE ( pay_serial_number = ? ) ### Cause: java.sql.SQLSyntaxErrorException: Table 'warehouse.settlement_hint' doesn't exist ; bad SQL grammar []; nested exception is java.sql.SQLSyntaxErrorException: Table 'warehouse.settlement_hint' doesn't exist at org.springframework.jdbc.support.SQLErrorCodeSQLExceptionTranslator.doTranslate(SQLErrorCodeSQLExceptionTranslator.java:234) at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:72) at org.mybatis.spring.MyBatisExceptionTranslator.translateExceptionIfPossible(MyBatisExceptionTranslator.java:73) at org.mybatis.spring.SqlSessionTemplate$SqlSessionInterceptor.invoke(SqlSessionTemplate.java:446) at com.sun.proxy.$Proxy94.selectList(Unknown Source) at org.mybatis.spring.SqlSessionTemplate.selectList(SqlSessionTemplate.java:230) at org.apache.ibatis.binding.MapperMethod.executeForMany(MapperMethod.java:139) at org.apache.ibatis.binding.MapperMethod.execute(MapperMethod.java:76) at org.apache.ibatis.binding.MapperProxy.invoke(MapperProxy.java:59) at com.sun.proxy.$Proxy95.selectByExample(Unknown Source) at org.cellphone.finance.repo.SettlementRepository.querySettlements(SettlementRepository.java:62) at org.cellphone.finance.repo.SettlementRepository$$FastClassBySpringCGLIB$$c5ad592f.invoke(<generated>) at org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:204) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.invokeJoinpoint(CglibAopProxy.java:746) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:163) at org.springframework.dao.support.PersistenceExceptionTranslationInterceptor.invoke(PersistenceExceptionTranslationInterceptor.java:139) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:185) at org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.intercept(CglibAopProxy.java:688) at org.cellphone.finance.repo.SettlementRepository$$EnhancerBySpringCGLIB$$9133dcc1.querySettlements(<generated>) at org.cellphone.finance.SettlementRepositoryTest.test004QuerySettlements(SettlementRepositoryTest.java:89) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50) at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12) at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:47) at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17) at org.springframework.test.context.junit4.statements.RunBeforeTestExecutionCallbacks.evaluate(RunBeforeTestExecutionCallbacks.java:73) at org.springframework.test.context.junit4.statements.RunAfterTestExecutionCallbacks.evaluate(RunAfterTestExecutionCallbacks.java:83) at org.springframework.test.context.junit4.statements.RunBeforeTestMethodCallbacks.evaluate(RunBeforeTestMethodCallbacks.java:75) at org.springframework.test.context.junit4.statements.RunAfterTestMethodCallbacks.evaluate(RunAfterTestMethodCallbacks.java:86) at org.springframework.test.context.junit4.statements.SpringRepeat.evaluate(SpringRepeat.java:84) at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:325) at org.springframework.test.context.junit4.SpringJUnit4ClassRunner.runChild(SpringJUnit4ClassRunner.java:251) at org.springframework.test.context.junit4.SpringJUnit4ClassRunner.runChild(SpringJUnit4ClassRunner.java:97) at org.junit.runners.ParentRunner$3.run(ParentRunner.java:290) at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:71) at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:288) at org.junit.runners.ParentRunner.access$000(ParentRunner.java:58) at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:268) at org.springframework.test.context.junit4.statements.RunBeforeTestClassCallbacks.evaluate(RunBeforeTestClassCallbacks.java:61) at org.springframework.test.context.junit4.statements.RunAfterTestClassCallbacks.evaluate(RunAfterTestClassCallbacks.java:70) at org.junit.runners.ParentRunner.run(ParentRunner.java:363) at org.springframework.test.context.junit4.SpringJUnit4ClassRunner.run(SpringJUnit4ClassRunner.java:190) at org.junit.runner.JUnitCore.run(JUnitCore.java:137) at com.intellij.junit4.JUnit4IdeaTestRunner.startRunnerWithArgs(JUnit4IdeaTestRunner.java:68) at com.intellij.rt.execution.junit.IdeaTestRunner$Repeater.startRunnerWithArgs(IdeaTestRunner.java:47) at com.intellij.rt.execution.junit.JUnitStarter.prepareStreamsAndStart(JUnitStarter.java:242) at com.intellij.rt.execution.junit.JUnitStarter.main(JUnitStarter.java:70) Caused by: java.sql.SQLSyntaxErrorException: Table 'warehouse.settlement_hint' doesn't exist at com.mysql.cj.jdbc.exceptions.SQLError.createSQLException(SQLError.java:120) at com.mysql.cj.jdbc.exceptions.SQLError.createSQLException(SQLError.java:97) at com.mysql.cj.jdbc.exceptions.SQLExceptionsMapping.translateException(SQLExceptionsMapping.java:122) at com.mysql.cj.jdbc.ClientPreparedStatement.executeInternal(ClientPreparedStatement.java:975) at com.mysql.cj.jdbc.ClientPreparedStatement.execute(ClientPreparedStatement.java:392) at com.alibaba.druid.filter.FilterChainImpl.preparedStatement_execute(FilterChainImpl.java:3409) at com.alibaba.druid.filter.FilterEventAdapter.preparedStatement_execute(FilterEventAdapter.java:440) at com.alibaba.druid.filter.FilterChainImpl.preparedStatement_execute(FilterChainImpl.java:3407) at com.alibaba.druid.proxy.jdbc.PreparedStatementProxyImpl.execute(PreparedStatementProxyImpl.java:167) at com.alibaba.druid.pool.DruidPooledPreparedStatement.execute(DruidPooledPreparedStatement.java:498) at io.shardingjdbc.core.executor.type.prepared.PreparedStatementExecutor$3.execute(PreparedStatementExecutor.java:101) at io.shardingjdbc.core.executor.type.prepared.PreparedStatementExecutor$3.execute(PreparedStatementExecutor.java:97) at io.shardingjdbc.core.executor.ExecutorEngine.executeInternal(ExecutorEngine.java:187) at io.shardingjdbc.core.executor.ExecutorEngine.syncExecute(ExecutorEngine.java:167) at io.shardingjdbc.core.executor.ExecutorEngine.execute(ExecutorEngine.java:131) at io.shardingjdbc.core.executor.ExecutorEngine.executePreparedStatement(ExecutorEngine.java:98) at io.shardingjdbc.core.executor.type.prepared.PreparedStatementExecutor.execute(PreparedStatementExecutor.java:97) at io.shardingjdbc.core.jdbc.core.statement.ShardingPreparedStatement.execute(ShardingPreparedStatement.java:141) at org.apache.ibatis.executor.statement.PreparedStatementHandler.query(PreparedStatementHandler.java:63) at org.apache.ibatis.executor.statement.RoutingStatementHandler.query(RoutingStatementHandler.java:79) at org.apache.ibatis.executor.SimpleExecutor.doQuery(SimpleExecutor.java:63) at org.apache.ibatis.executor.BaseExecutor.queryFromDatabase(BaseExecutor.java:326) at org.apache.ibatis.executor.BaseExecutor.query(BaseExecutor.java:156) at org.apache.ibatis.executor.CachingExecutor.query(CachingExecutor.java:109) at com.github.pagehelper.PageInterceptor.intercept(PageInterceptor.java:108) at org.apache.ibatis.plugin.Plugin.invoke(Plugin.java:61) at com.sun.proxy.$Proxy117.query(Unknown Source) at org.apache.ibatis.session.defaults.DefaultSqlSession.selectList(DefaultSqlSession.java:148) at org.apache.ibatis.session.defaults.DefaultSqlSession.selectList(DefaultSqlSession.java:141) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.mybatis.spring.SqlSessionTemplate$SqlSessionInterceptor.invoke(SqlSessionTemplate.java:433) ... 46 more
拋出錯誤堆棧後,很納悶爲什麼沒有將邏輯表名 settlement_hint 轉換爲真實表名 settlement_20180403 ,因而就跟蹤了Sharding-JDBC源碼。
通過DEBUG,得知全部JDBC操做都會調用 io.shardingjdbc.core.routing.type.simple.SimpleRoutingEngine#route 方法:
/* * Copyright 1999-2015 dangdang.com. * <p> * Licensed under the Apache License, Version 2.0 (the "License"); * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * * http://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. * See the License for the specific language governing permissions and * limitations under the License. * </p> */ package io.shardingjdbc.core.routing.type.simple; import io.shardingjdbc.core.api.algorithm.sharding.ShardingValue; import io.shardingjdbc.core.hint.HintManagerHolder; import io.shardingjdbc.core.hint.ShardingKey; import io.shardingjdbc.core.parsing.parser.context.condition.Column; import io.shardingjdbc.core.parsing.parser.context.condition.Condition; import io.shardingjdbc.core.parsing.parser.sql.SQLStatement; import io.shardingjdbc.core.routing.strategy.ShardingStrategy; import io.shardingjdbc.core.routing.type.RoutingEngine; import io.shardingjdbc.core.routing.type.RoutingResult; import io.shardingjdbc.core.routing.type.TableUnit; import io.shardingjdbc.core.rule.DataNode; import io.shardingjdbc.core.rule.ShardingRule; import io.shardingjdbc.core.rule.TableRule; import com.google.common.base.Optional; import com.google.common.base.Preconditions; import lombok.RequiredArgsConstructor; import java.util.ArrayList; import java.util.Collection; import java.util.LinkedList; import java.util.List; /** * Simple routing engine. * * @author zhangliang */ @RequiredArgsConstructor public final class SimpleRoutingEngine implements RoutingEngine { private final ShardingRule shardingRule; private final List<Object> parameters; private final String logicTableName; private final SQLStatement sqlStatement; /** * Sharding-JDBC路由方法,全部JDBC操做都會調用此方法,路由到真實表 * * @return */ @Override public RoutingResult route() { // 從XML中根據邏輯表名獲取配置的數據表規則 TableRule tableRule = shardingRule.getTableRule(logicTableName); // 獲取數據源分片值,分片值來自邏輯SQL中的數據,因爲XML只配置了單個數據源,暫時不深刻分析此環節 List<ShardingValue> databaseShardingValues = getDatabaseShardingValues(tableRule); // 獲取數據表分片值,分片值來自邏輯SQL中的數據,重點分析這部分代碼 List<ShardingValue> tableShardingValues = getTableShardingValues(tableRule); // 計算須要路由的真實數據源 Collection<String> routedDataSources = routeDataSources(tableRule, databaseShardingValues); Collection<DataNode> routedDataNodes = new LinkedList<>(); for (String each : routedDataSources) { // routeTables方法計算須要路由的真實數據表,重點分析這部分代碼 routedDataNodes.addAll(routeTables(tableRule, each, tableShardingValues)); } // 產生路由結果 return generateRoutingResult(routedDataNodes); } private List<ShardingValue> getDatabaseShardingValues(final TableRule tableRule) { ShardingStrategy strategy = shardingRule.getDatabaseShardingStrategy(tableRule); return HintManagerHolder.isUseShardingHint() ? getDatabaseShardingValuesFromHint(strategy.getShardingColumns()) : getShardingValues(strategy.getShardingColumns()); } /** * 獲取數據表分片值 * 分片值由上面提到的querySettlements經過hintManager傳入 * * @param tableRule 數據表規則,從XML文件中獲取 * @return 分片值列表 */ private List<ShardingValue> getTableShardingValues(final TableRule tableRule) { // 從XML文件中獲取數據表規則 ShardingStrategy strategy = shardingRule.getTableShardingStrategy(tableRule); /* * 判斷是否使用了Hint分片算法: * 1. 是:根據ShardingStrategy中拿到的shardingColumns來獲取shardingValues,問題就出在這裏!!! * 2. 否:從SQL中根據shardingColumns來獲取shardingValues */ return HintManagerHolder.isUseShardingHint() ? getTableShardingValuesFromHint(strategy.getShardingColumns()) : getShardingValues(strategy.getShardingColumns()); } private List<ShardingValue> getDatabaseShardingValuesFromHint(final Collection<String> shardingColumns) { List<ShardingValue> result = new ArrayList<>(shardingColumns.size()); for (String each : shardingColumns) { Optional<ShardingValue> shardingValue = HintManagerHolder.getDatabaseShardingValue(new ShardingKey(logicTableName, each)); if (shardingValue.isPresent()) { result.add(shardingValue.get()); } } return result; } /** * 從Hint分片算法中獲取shardingValues * * @param shardingColumns 分片鍵 * @return 分片值列表 */ private List<ShardingValue> getTableShardingValuesFromHint(final Collection<String> shardingColumns) { List<ShardingValue> result = new ArrayList<>(shardingColumns.size()); for (String each : shardingColumns) { // 從HintManagerHolder獲取sharding values Optional<ShardingValue> shardingValue = HintManagerHolder.getTableShardingValue(new ShardingKey(logicTableName, each)); if (shardingValue.isPresent()) { result.add(shardingValue.get()); } } return result; } private List<ShardingValue> getShardingValues(final Collection<String> shardingColumns) { List<ShardingValue> result = new ArrayList<>(shardingColumns.size()); for (String each : shardingColumns) { Optional<Condition> condition = sqlStatement.getConditions().find(new Column(each, logicTableName)); if (condition.isPresent()) { result.add(condition.get().getShardingValue(parameters)); } } return result; } private Collection<String> routeDataSources(final TableRule tableRule, final List<ShardingValue> databaseShardingValues) { Collection<String> availableTargetDatabases = tableRule.getActualDatasourceNames(); if (databaseShardingValues.isEmpty()) { return availableTargetDatabases; } Collection<String> result = shardingRule.getDatabaseShardingStrategy(tableRule).doSharding(availableTargetDatabases, databaseShardingValues); Preconditions.checkState(!result.isEmpty(), "no database route info"); return result; } /** * 計算須要路由的真實數據表 * * @param tableRule 數據表規則,從XML中獲取 * @param routedDataSource 須要路由的數據源 * @param tableShardingValues 數據表分片值 * @return 須要路由的真實數據節點 */ private Collection<DataNode> routeTables(final TableRule tableRule, final String routedDataSource, final List<ShardingValue> tableShardingValues) { // 全部的真實數據表名,在XML中能夠顯式配置,也能夠省略 Collection<String> availableTargetTables = tableRule.getActualTableNames(routedDataSource); /* * table sharding values是否爲空: * 1. 是:路由到全部真實表 * 2. 否:路由到自定義的分表策略算法定義的數據表 * * 問題就出在這裏:tableShardingValues.isEmpty()爲true,致使沒法使用自定義分表策略算法 */ Collection<String> routedTables = tableShardingValues.isEmpty() ? availableTargetTables : shardingRule.getTableShardingStrategy(tableRule).doSharding(availableTargetTables, tableShardingValues); Preconditions.checkState(!routedTables.isEmpty(), "no table route info"); Collection<DataNode> result = new LinkedList<>(); for (String each : routedTables) { result.add(new DataNode(routedDataSource, each)); } return result; } private RoutingResult generateRoutingResult(final Collection<DataNode> routedDataNodes) { RoutingResult result = new RoutingResult(); for (DataNode each : routedDataNodes) { result.getTableUnits().getTableUnits().add(new TableUnit(each.getDataSourceName(), logicTableName, each.getTableName())); } return result; } }
上述代碼中, getTableShardingValuesFromHint 返回的 List<ShardingValue> 爲空,緣由是 getTableShardingValuesFromHint 的入參 strategy.getShardingColumns() 爲空列表,而此參數來自Sharding-JDBC的默認實現HintShardingStrategy:
/* * Copyright 1999-2015 dangdang.com. * <p> * Licensed under the Apache License, Version 2.0 (the "License"); * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * * http://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. * See the License for the specific language governing permissions and * limitations under the License. * </p> */ package io.shardingjdbc.core.routing.strategy.hint; import io.shardingjdbc.core.api.algorithm.sharding.hint.HintShardingAlgorithm; import io.shardingjdbc.core.api.algorithm.sharding.ShardingValue; import io.shardingjdbc.core.routing.strategy.ShardingStrategy; import lombok.Getter; import java.util.Collection; import java.util.TreeSet; /** * Hint sharding strategy. * final類,沒法繼承此類來重定義內部邏輯 * * * @author zhangliang */ public final class HintShardingStrategy implements ShardingStrategy { /** * 沒有提供Setter方法和能夠配置的入口,沒法設置shardingColumns,這是2.0.3版本的BUG * * github issue: https://github.com/apache/incubator-shardingsphere/issues/1980 */ @Getter private final Collection<String> shardingColumns; private final HintShardingAlgorithm shardingAlgorithm; public HintShardingStrategy(final HintShardingAlgorithm shardingAlgorithm) { this.shardingColumns = new TreeSet<>(String.CASE_INSENSITIVE_ORDER); this.shardingAlgorithm = shardingAlgorithm; } @Override public Collection<String> doSharding(final Collection<String> availableTargetNames, final Collection<ShardingValue> shardingValues) { Collection<String> shardingResult = shardingAlgorithm.doSharding(availableTargetNames, shardingValues.iterator().next()); Collection<String> result = new TreeSet<>(String.CASE_INSENSITIVE_ORDER); result.addAll(shardingResult); return result; } }
跟蹤到此,Hint分片算法沒法使用的緣由一路瞭然,這是Sharding-JDBC 2.0.3版本的BUG,但在後續版本4.x中獲得了修復。