ShardingSphere初探1 --Sharding-JDBC

時間  2019-11-24
標籤 shardingsphere 初探 sharding jdbc 欄目 Java 简体版
原文   原文鏈接

Sharding-JDBCjava

引入maven依賴:node

<dependency>
    <groupId>org.apache.shardingsphere</groupId>
    <artifactId>sharding-jdbc-core</artifactId>
    <version>${latest.release.version}</version>
</dependency>

注意將version換爲實際的版本號.mysql

簡介

Sharding-JDBC是ShardingSphere的第一個產品，也是ShardingSphere的前身。 它定位爲輕量級Java框架，在Java的JDBC層提供的額外服務。它使用客戶端直連數據庫，以jar包形式提供服務，無需額外部署和依賴，可理解爲加強版的JDBC驅動，徹底兼容JDBC和各類ORM框架。git

  • 適用於任何基於Java的ORM框架，如：JPA, Hibernate, Mybatis, Spring JDBC Template或直接使用JDBC。
  • 基於任何第三方的數據庫鏈接池，如：DBCP, C3P0, BoneCP, Druid, HikariCP等。
  • 支持任意實現JDBC規範的數據庫。目前支持MySQL，Oracle，SQLServer和PostgreSQL。

 

對比:github

Sharding-JDBC的優點在於對Java應用的友好度.算法

內部結構spring

 

圖片解析:sql

黃色部分數據庫

圖中黃色部分表示的是Sharding-JDBC的入口API,採用工廠方法的形式提供.express

目前有ShardingDataSourceFactory 和 MasterSlaveDataSourceFactory 兩個工廠類.

ShardingDataSourceFactory 用於建立分庫分表或分庫分表+讀寫分離的JDBC驅動,

MasterSlaveDataSourceFactory用於建立獨立使用讀寫分離的JDBC驅動.

藍色部分

圖中藍色部分表示的是Sharding-JDBC的配置對象,提供靈活多變的配置方式.

ShardingRuleConfiguration是分庫分表配置的核心和入口,它能夠包含多個

TableRuleConfiguration 和 MasterSlaveRuleConfiguration .

每一組相同規則分片的表配置一個TableRuleConfiguration. 若是須要分庫分表和讀寫分離共同使用,

每個讀寫分離的邏輯庫配置一個 MasterSlaveRuleConfiguration .

每一個TableRuleConfiguration對應一個ShardingStrategyConfiguration,它有5種實現類可供選擇.

* 僅讀寫分離使用MasterSlaveRuleConfiguration便可.

紅色部分

圖中紅色部分表示的是內部對象,由Sharding-JDBC內部使用,應用開發者無需關注.

Sharding-JDBC經過ShardingRuleConfiguration和MasterSlaveRuleConfiguration生成真正

供ShardingDataSource和MasterSlaveDataSource使用的規則對象.

ShardingDataSource和MasterSlaveDataSource實現了DataSource接口, 是JDBC的完整實現方案.

 

初始化流程:

1. 配置Configuration對象.

2. 經過Factory對象將Configuration對象轉化爲Rule對象.

3. 經過Factory對象將Rule對象與DataSource對象裝配.

4. Sharding-JDBC使用DataSource對象進行分庫.

使用約定

在org.apache.shardingsphere.api和org.apache.shardingsphere.shardingjdbc.api 包中的類是面向用戶的API，每次修改都會在release notes中明確聲明。 其餘包中的類屬於內部實現，可能隨時進行調整，請勿直接使用。

 

官方Sharding-JDBC相關使用示例 : https://github.com/apache/incubator-shardingsphere-example

數據分片  什麼是數據分片?

首先看不使用Spring的方式:

引入Maven依賴

<dependency>
    <groupId>org.apache.shardingsphere</groupId>
    <artifactId>sharding-jdbc-core</artifactId>
    <version>${sharding-sphere.version}</version>
</dependency>

基於Java編碼的規則配置

Sharding-JDBC的分庫分表經過規則配置描述, 如下例子是根據user_id取模分庫,且根據order_id取模分表的兩庫兩表的配置.

    // 配置真實數據源
    Map<String, DataSource> dataSourceMap = new HashMap<>();
    
    // 配置第一個數據源
    BasicDataSource dataSource1 = new BasicDataSource();
    dataSource1.setDriverClassName("com.mysql.jdbc.Driver");
    dataSource1.setUrl("jdbc:mysql://localhost:3306/ds0");
    dataSource1.setUsername("root");
    dataSource1.setPassword("");
    dataSourceMap.put("ds0", dataSource1);
    
    // 配置第二個數據源
    BasicDataSource dataSource2 = new BasicDataSource();
    dataSource2.setDriverClassName("com.mysql.jdbc.Driver");
    dataSource2.setUrl("jdbc:mysql://localhost:3306/ds1");
    dataSource2.setUsername("root");
    dataSource2.setPassword("");
    dataSourceMap.put("ds1", dataSource2);
    
    // 配置Order表規則
    TableRuleConfiguration orderTableRuleConfig = new TableRuleConfiguration();
    orderTableRuleConfig.setLogicTable("t_order");
    orderTableRuleConfig.setActualDataNodes("ds${0..1}.t_order${0..1}");
    
    // 配置分庫 + 分表策略
    orderTableRuleConfig.setDatabaseShardingStrategyConfig(new InlineShardingStrategyConfiguration("user_id", "ds${user_id % 2}"));
    orderTableRuleConfig.setTableShardingStrategyConfig(new InlineShardingStrategyConfiguration("order_id", "t_order${order_id % 2}"));
    
    // 配置分片規則
    ShardingRuleConfiguration shardingRuleConfig = new ShardingRuleConfiguration();
    shardingRuleConfig.getTableRuleConfigs().add(orderTableRuleConfig);
    
    // 省略配置order_item表規則...
    // ...
    
    // 獲取數據源對象
    DataSource dataSource = ShardingDataSourceFactory.createDataSource(dataSourceMap, shardingRuleConfig, new ConcurrentHashMap(), new Properties());

基於yaml的規則配置:

或經過Yaml方式配置,與以上配置等價:

dataSources:
  ds0: !!org.apache.commons.dbcp.BasicDataSource
    driverClassName: com.mysql.jdbc.Driver
    url: jdbc:mysql://localhost:3306/ds0
    username: root
    password: 
  ds1: !!org.apache.commons.dbcp.BasicDataSource
    driverClassName: com.mysql.jdbc.Driver
    url: jdbc:mysql://localhost:3306/ds1
    username: root
    password: 
    
shardingRule:
  tables:
    t_order: 
      actualDataNodes: ds${0..1}.t_order${0..1}
      databaseStrategy: 
        inline:
          shardingColumn: user_id
          algorithmInlineExpression: ds${user_id % 2}
      tableStrategy: 
        inline:
          shardingColumn: order_id
          algorithmInlineExpression: t_order${order_id % 2}
    t_order_item: 
      actualDataNodes: ds${0..1}.t_order_item${0..1}
      databaseStrategy: 
        inline:
          shardingColumn: user_id
          algorithmInlineExpression: ds${user_id % 2}
      tableStrategy: 
        inline:
          shardingColumn: order_id
          algorithmInlineExpression: t_order_item${order_id % 2}
    DataSource dataSource = YamlShardingDataSourceFactory.createDataSource(yamlFile);

使用原生JDBC:

經過ShardingDataSourceFactory或者YamlShardingDataSourceFactory工廠和規則配置對象獲取ShardingDataSource, ShardingDataSource實現自JDBC的

標準接口DataSource. 而後可經過DataSource選擇使用原生JDBC開發, 或者使用JPA, MyBatis等ORM工具. 以JDBC原生實現爲例:

DataSource dataSource = YamlShardingDataSourceFactory.createDataSource(yamlFile);
String sql = "SELECT i.* FROM t_order o JOIN t_order_item i ON o.order_id=i.order_id WHERE o.user_id=? AND o.order_id=?";
try (
        Connection conn = dataSource.getConnection();
        PreparedStatement preparedStatement = conn.prepareStatement(sql)) {
    preparedStatement.setInt(1, 10);
    preparedStatement.setInt(2, 1001);
    try (ResultSet rs = preparedStatement.executeQuery()) {
        while(rs.next()) {
            System.out.println(rs.getInt(1));
            System.out.println(rs.getInt(2));
        }
    }
}

使用Spring

引入Maven依賴

<!-- for spring boot -->
<dependency>
    <groupId>org.apache.shardingsphere</groupId>
    <artifactId>sharding-jdbc-spring-boot-starter</artifactId>
    <version>${sharding-sphere.version}</version>
</dependency>

<!-- for spring namespace -->
<dependency>
    <groupId>org.apache.shardingsphere</groupId>
    <artifactId>sharding-jdbc-spring-namespace</artifactId>
    <version>${sharding-sphere.version}</version>
</dependency>

基於Spring boot的規則配置

spring.shardingsphere.datasource.names=ds0,ds1

spring.shardingsphere.datasource.ds0.type=org.apache.commons.dbcp2.BasicDataSource
spring.shardingsphere.datasource.ds0.driver-class-name=com.mysql.jdbc.Driver
spring.shardingsphere.datasource.ds0.url=jdbc:mysql://localhost:3306/ds0
spring.shardingsphere.datasource.ds0.username=root
spring.shardingsphere.datasource.ds0.password=

spring.shardingsphere.datasource.ds1.type=org.apache.commons.dbcp2.BasicDataSource
spring.shardingsphere.datasource.ds1.driver-class-name=com.mysql.jdbc.Driver
spring.shardingsphere.datasource.ds1.url=jdbc:mysql://localhost:3306/ds1
spring.shardingsphere.datasource.ds1.username=root
spring.shardingsphere.datasource.ds1.password=

spring.shardingsphere.sharding.default-database-strategy.inline.sharding-column=user_id
spring.shardingsphere.sharding.default-database-strategy.inline.algorithm-expression=ds$->{user_id % 2}

spring.shardingsphere.sharding.tables.t_order.actual-data-nodes=ds$->{0..1}.t_order$->{0..1}
spring.shardingsphere.sharding.tables.t_order.table-strategy.inline.sharding-column=order_id
spring.shardingsphere.sharding.tables.t_order.table-strategy.inline.algorithm-expression=t_order$->{order_id % 2}

spring.shardingsphere.sharding.tables.t_order_item.actual-data-nodes=ds$->{0..1}.t_order_item$->{0..1}
spring.shardingsphere.sharding.tables.t_order_item.table-strategy.inline.sharding-column=order_id
spring.shardingsphere.sharding.tables.t_order_item.table-strategy.inline.algorithm-expression=t_order_item$->{order_id % 2}

基於Spring boot + JNDI的規則配置

若是您計劃使用Spring boot + JNDI的方式, 在應用容器(如Tomcat)中使用Sharding-JDBC時,可以使用

spring.shardingsphere.datasource.${datasourceName}.jndiName來替代數據源的一系列配置.如:

spring.shardingsphere.datasource.names=ds0,ds1

spring.shardingsphere.datasource.ds0.jndi-name=java:comp/env/jdbc/ds0
spring.shardingsphere.datasource.ds1.jndi-name=jdbc/ds1

spring.shardingsphere.sharding.default-database-strategy.inline.sharding-column=user_id
spring.shardingsphere.sharding.default-database-strategy.inline.algorithm-expression=ds$->{user_id % 2}

spring.shardingsphere.sharding.tables.t_order.actual-data-nodes=ds$->{0..1}.t_order$->{0..1}
spring.shardingsphere.sharding.tables.t_order.table-strategy.inline.sharding-column=order_id
spring.shardingsphere.sharding.tables.t_order.table-strategy.inline.algorithm-expression=t_order$->{order_id % 2}

spring.shardingsphere.sharding.tables.t_order_item.actual-data-nodes=ds$->{0..1}.t_order_item$->{0..1}
spring.shardingsphere.sharding.tables.t_order_item.table-strategy.inline.sharding-column=order_id
spring.shardingsphere.sharding.tables.t_order_item.table-strategy.inline.algorithm-expression=t_order_item$->{order_id % 2}

基於Spring命名空間的規則配置:

<?xml version="1.0" encoding="UTF-8"?>
<beans xmlns="http://www.springframework.org/schema/beans"
    xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" 
    xmlns:sharding="http://shardingsphere.apache.org/schema/shardingsphere/sharding" 
    xsi:schemaLocation="http://www.springframework.org/schema/beans 
                        http://www.springframework.org/schema/beans/spring-beans.xsd
                        http://shardingsphere.apache.org/schema/shardingsphere/sharding 
                        http://shardingsphere.apache.org/schema/shardingsphere/sharding/sharding.xsd 
                        ">
    <bean id="ds0" class="org.apache.commons.dbcp.BasicDataSource" destroy-method="close">
        <property name="driverClassName" value="com.mysql.jdbc.Driver" />
        <property name="url" value="jdbc:mysql://localhost:3306/ds0" />
        <property name="username" value="root" />
        <property name="password" value="" />
    </bean>
    <bean id="ds1" class="org.apache.commons.dbcp.BasicDataSource" destroy-method="close">
        <property name="driverClassName" value="com.mysql.jdbc.Driver" />
        <property name="url" value="jdbc:mysql://localhost:3306/ds1" />
        <property name="username" value="root" />
        <property name="password" value="" />
    </bean>
    
    <sharding:inline-strategy id="databaseStrategy" sharding-column="user_id" algorithm-expression="ds$->{user_id % 2}" />
    <sharding:inline-strategy id="orderTableStrategy" sharding-column="order_id" algorithm-expression="t_order$->{order_id % 2}" />
    <sharding:inline-strategy id="orderItemTableStrategy" sharding-column="order_id" algorithm-expression="t_order_item$->{order_id % 2}" />
    
    <sharding:data-source id="shardingDataSource">
        <sharding:sharding-rule data-source-names="ds0,ds1">
            <sharding:table-rules>
                <sharding:table-rule logic-table="t_order" actual-data-nodes="ds$->{0..1}.t_order$->{0..1}" database-strategy-ref="databaseStrategy" table-strategy-ref="orderTableStrategy" />
                <sharding:table-rule logic-table="t_order_item" actual-data-nodes="ds$->{0..1}.t_order_item$->{0..1}" database-strategy-ref="databaseStrategy" table-strategy-ref="orderItemTableStrategy" />
            </sharding:table-rules>
        </sharding:sharding-rule>
    </sharding:data-source>
</beans>

在Spring中使用DataSource

直接經過注入的方式便可使用DataSource, 或者將DataSource配置在JPA,Hibernate或MyBatis中使用.

@Resource
private DataSource dataSource;

規則配置包括數據源配置,表規則配置,分庫策略和分表策略組成.這只是最簡單的配置方式,實際使用可更加靈活,

如:多分片鍵, 分片策略直接和表規則配置綁定等.

 

讀寫分離

一樣讀寫分離的也是單獨的依賴

不使用Spring

引入Maven依賴:

<dependency>
    <groupId>org.apache.shardingsphere</groupId>
    <artifactId>sharding-jdbc-core</artifactId>
    <version>${sharding-sphere.version}</version>
</dependency>

基於Java編碼的規則配置

    // 配置真實數據源
    Map<String, DataSource> dataSourceMap = new HashMap<>();
    
    // 配置主庫
    BasicDataSource masterDataSource = new BasicDataSource();
    masterDataSource.setDriverClassName("com.mysql.jdbc.Driver");
    masterDataSource.setUrl("jdbc:mysql://localhost:3306/ds_master");
    masterDataSource.setUsername("root");
    masterDataSource.setPassword("");
    dataSourceMap.put("ds_master", masterDataSource);
    
    // 配置第一個從庫
    BasicDataSource slaveDataSource1 = new BasicDataSource();
    slaveDataSource1.setDriverClassName("com.mysql.jdbc.Driver");
    slaveDataSource1.setUrl("jdbc:mysql://localhost:3306/ds_slave0");
    slaveDataSource1.setUsername("root");
    slaveDataSource1.setPassword("");
    dataSourceMap.put("ds_slave0", slaveDataSource1);
    
    // 配置第二個從庫
    BasicDataSource slaveDataSource2 = new BasicDataSource();
    slaveDataSource2.setDriverClassName("com.mysql.jdbc.Driver");
    slaveDataSource2.setUrl("jdbc:mysql://localhost:3306/ds_slave1");
    slaveDataSource2.setUsername("root");
    slaveDataSource2.setPassword("");
    dataSourceMap.put("ds_slave1", slaveDataSource2);
    
    // 配置讀寫分離規則
    MasterSlaveRuleConfiguration masterSlaveRuleConfig = new MasterSlaveRuleConfiguration("ds_master_slave", "ds_master", Arrays.asList("ds_slave0", "ds_slave1"));
    
    // 獲取數據源對象
    DataSource dataSource = MasterSlaveDataSourceFactory.createDataSource(createDataSourceMap(), masterSlaveRuleConfig, new HashMap<String, Object>(), new Properties());

基於Yaml的規則配置:

或經過Yaml方式配置,與以上配置等價:

 

dataSources:
  ds_master: !!org.apache.commons.dbcp.BasicDataSource
    driverClassName: com.mysql.jdbc.Driver
    url: jdbc:mysql://localhost:3306/ds_master
    username: root
    password: 
  ds_slave0: !!org.apache.commons.dbcp.BasicDataSource
    driverClassName: com.mysql.jdbc.Driver
    url: jdbc:mysql://localhost:3306/ds_slave0
    username: root
    password:
  ds_slave1: !!org.apache.commons.dbcp.BasicDataSource
    driverClassName: com.mysql.jdbc.Driver
    url: jdbc:mysql://localhost:3306/ds_slave1
    username: root
    password: 

masterSlaveRule:
  name: ds_ms
  masterDataSourceName: ds_master
  slaveDataSourceNames: [ds_slave0, ds_slave1]
  
props:
  sql.show: true
    DataSource dataSource = MasterSlaveDataSourceFactory.createDataSource(yamlFile);

使用原生JDBC

經過MasterSlaveDataSourceFactory工廠和規則配置對象獲取MasterSlaveDataSource, MasterSlaveDataSource實現自JDBC的標準接口DataSource.

而後可經過DataSource選擇使用原生JDBC開發,或者使用JPA, MyBatis等ORM工具. 以JDBC原生實現爲例:

DataSource dataSource = MasterSlaveDataSourceFactory.createDataSource(yamlFile);
String sql = "SELECT i.* FROM t_order o JOIN t_order_item i ON o.order_id=i.order_id WHERE o.user_id=? AND o.order_id=?";
try (
        Connection conn = dataSource.getConnection();
        PreparedStatement preparedStatement = conn.prepareStatement(sql)) {
    preparedStatement.setInt(1, 10);
    preparedStatement.setInt(2, 1001);
    try (ResultSet rs = preparedStatement.executeQuery()) {
        while(rs.next()) {
            System.out.println(rs.getInt(1));
            System.out.println(rs.getInt(2));
        }
    }
}

使用Spring

引入Maven依賴:

<!-- for spring boot -->
<dependency>
    <groupId>org.apache.shardingsphere</groupId>
    <artifactId>sharding-jdbc-spring-boot-starter</artifactId>
    <version>${sharding-sphere.version}</version>
</dependency>

<!-- for spring namespace -->
<dependency>
    <groupId>org.apache.shardingsphere</groupId>
    <artifactId>sharding-jdbc-spring-namespace</artifactId>
    <version>${sharding-sphere.version}</version>
</dependency>

基於Spring boot的規則配置:

spring.shardingsphere.datasource.names=master,slave0,slave1

spring.shardingsphere.datasource.master.type=org.apache.commons.dbcp.BasicDataSource
spring.shardingsphere.datasource.master.driver-class-name=com.mysql.jdbc.Driver
spring.shardingsphere.datasource.master.url=jdbc:mysql://localhost:3306/master
spring.shardingsphere.datasource.master.username=root
spring.shardingsphere.datasource.master.password=

spring.shardingsphere.datasource.slave0.type=org.apache.commons.dbcp.BasicDataSource
spring.shardingsphere.datasource.slave0.driver-class-name=com.mysql.jdbc.Driver
spring.shardingsphere.datasource.slave0.url=jdbc:mysql://localhost:3306/slave0
spring.shardingsphere.datasource.slave0.username=root
spring.shardingsphere.datasource.slave0.password=

spring.shardingsphere.datasource.slave1.type=org.apache.commons.dbcp.BasicDataSource
spring.shardingsphere.datasource.slave1.driver-class-name=com.mysql.jdbc.Driver
spring.shardingsphere.datasource.slave1.url=jdbc:mysql://localhost:3306/slave1
spring.shardingsphere.datasource.slave1.username=root
spring.shardingsphere.datasource.slave1.password=

spring.shardingsphere.masterslave.name=ms
spring.shardingsphere.masterslave.master-data-source-name=master
spring.shardingsphere.masterslave.slave-data-source-names=slave0,slave1

spring.shardingsphere.props.sql.show=true

基於Spring boot + JNDI的規則配置:

若是您計劃使用Spring boot + JNDI的方式, 在應用容器(如Tomcat)中使用Sharding-JDBC時,可以使用spring.shardingsphere.datasource.${datasourceName}.jndiName來代替數據源的一系列配置.如:

spring.shardingsphere.datasource.names=master,slave0,slave1

spring.shardingsphere.datasource.master.jndi-name=java:comp/env/jdbc/master
spring.shardingsphere.datasource.slave0.jndi-name=jdbc/slave0
spring.shardingsphere.datasource.slave1.jndi-name=jdbc/slave1

spring.shardingsphere.masterslave.name=ms
spring.shardingsphere.masterslave.master-data-source-name=master
spring.shardingsphere.masterslave.slave-data-source-names=slave0,slave1

spring.shardingsphere.props.sql.show=true

基於Spring命名空間的規則配置:

<?xml version="1.0" encoding="UTF-8"?>
<beans xmlns="http://www.springframework.org/schema/beans"
    xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" 
    xmlns:master-slave="http://shardingsphere.apache.org/schema/shardingsphere/masterslave" 
    xsi:schemaLocation="http://www.springframework.org/schema/beans 
                        http://www.springframework.org/schema/beans/spring-beans.xsd
                        http://shardingsphere.apache.org/schema/shardingsphere/masterslave 
                        http://shardingsphere.apache.org/schema/shardingsphere/masterslave/master-slave.xsd 
                        ">
    <bean id="ds_master" class="org.apache.commons.dbcp.BasicDataSource" destroy-method="close">
        <property name="driverClassName" value="com.mysql.jdbc.Driver" />
        <property name="url" value="jdbc:mysql://localhost:3306/ds_master" />
        <property name="username" value="root" />
        <property name="password" value="" />
    </bean>
    <bean id="ds_slave0" class="org.apache.commons.dbcp.BasicDataSource" destroy-method="close">
        <property name="driverClassName" value="com.mysql.jdbc.Driver" />
        <property name="url" value="jdbc:mysql://localhost:3306/ds_slave0" />
        <property name="username" value="root" />
        <property name="password" value="" />
    </bean>
    <bean id="ds_slave1" class="org.apache.commons.dbcp.BasicDataSource" destroy-method="close">
        <property name="driverClassName" value="com.mysql.jdbc.Driver" />
        <property name="url" value="jdbc:mysql://localhost:3306/ds_slave1" />
        <property name="username" value="root" />
        <property name="password" value="" />
    </bean>
    
    <master-slave:data-source id="masterSlaveDataSource" master-data-source-name="ds_master" slave-data-source-names="ds_slave0, ds_slave1" >
        <master-slave:props>
                <prop key="sql.show">${sql_show}</prop>
                <prop key="executor.size">10</prop>
                <prop key="foo">bar</prop>
            </master-slave:props>
    </master-slave:data-source>
</beans>

在Spring中使用DataSource

直接經過注入的方式便可使用DataSource, 或者將DataSource配置在JPA, Hibernate或MyBatis中使用.

@Resource
private DataSource dataSource;

 

配置:

配置是整個Sharding-JDBC的核心,是Sharding-JDBC中惟一與應用開發者打交道的模塊. 配置模塊也是Sharding-JDBC的門戶,

經過它能夠快速清晰的理解Sharding-JDBC所提供的功能.

本部分是Sharding-JDBC的配置參考手冊,須要時可當作字典查閱.

Sharding-JDBC提供了4種配置方式,用於不一樣的使用場景.經過配置,應用開發者能夠靈活的使用分庫分表,讀寫分離以及分庫分表+讀寫分離共用.

 

強制路由

ShardingSphere使用ThreadLocal管理分片鍵值進行Hint強制路由. 能夠經過編程的方式向HintManager中添加分片值,該分片值僅在當前線程內生效.

Hint方式主要使用場景:

1. 分片字段不存在SQL中, 數據庫表結構中, 而存在於外部業務邏輯.

2. 強制在主庫進行某些數據操做.

基於暗示(Hint)的數據分片

配置Hint分片算法

Hint分片算法須要用戶實現org.apache.shardingsphere.api.sharding.hint.HintShardingAlgorithm接口.

ShardingSphere在進行Routing時,若是發現LogicTable的TableRule採用了Hint的分片算法,將會從HintManager中獲取分片值進行路由操做.

參考配置以下:

shardingRule:
  tables:
   t_order:
        actualDataNodes: demo_ds_${0..1}.t_order_${0..1}
        databaseStrategy:
          hint:
            algorithmClassName: org.apache.shardingsphere.userAlgo.HintAlgorithm
        tableStrategy:
          hint:
            algorithmClassName: org.apache.shardingsphere.userAlgo.HintAlgorithm
  defaultTableStrategy:
    none:
  defaultKeyGenerator:
    type: SNOWFLAKE
    column: order_id
  props:
      sql.show: true

獲取HintManager

HintManager hintManager = HintManager.getInstance();

添加分片鍵值

* 使用hintManager.addDatabaseShardingValue來添加數據源分片鍵值.

* 使用hintManager.addTableShardingValue來添加表分片鍵值.

分庫不分表狀況下, 強制路由至某一個分庫時, 可以使用hintManager.setDatabaseShardingValue方式添加分片.

經過此方式添加分片鍵值後,將跳過SQL解析和改寫階段, 從而提升總體執行效率.

清除分片鍵值

分片鍵值保存存在ThreadLocal中,因此須要在操做結束時調用hintManager.close()來清除ThreadLocal中的內容.

hintManager實現了AutoCloseable接口, 可推薦使用 try with resource 自動關閉.

完整代碼示例:

// Sharding database and table with using hintManager.
        String sql = "SELECT * FROM t_order";
        try (HintManager hintManager = HintManager.getInstance();
             Connection conn = dataSource.getConnection();
             PreparedStatement preparedStatement = conn.prepareStatement(sql)) {
            hintManager.addDatabaseShardingValue("t_order", 1);
            hintManager.addTableShardingValue("t_order", 2);
            try (ResultSet rs = preparedStatement.executeQuery()) {
                while (rs.next()) {
                    // ...
                }
            }
        }

// Sharding database without sharding table and routing to only one database with using hintManger.
        String sql = "SELECT * FROM t_order";
        try (HintManager hintManager = HintManager.getInstance();
             Connection conn = dataSource.getConnection();
             PreparedStatement preparedStatement = conn.prepareStatement(sql)) {
            hintManager.setDatabaseShardingValue(3);
            try (ResultSet rs = preparedStatement.executeQuery()) {
                while (rs.next()) {
                    // ...
                }
            }
        }

基於暗示(Hint)的強制主庫路由

獲取HintManager

與基於暗示(Hint)的數據分片相同.

設置主庫路由

* 使用hintManager.setMasterRouteOnly設置主庫路由.

清除分片鍵值

與基於暗示(Hint)的數據分片相同.

完整代碼示例:

String sql = "SELECT * FROM t_order";
try (
        HintManager hintManager = HintManager.getInstance();
        Connection conn = dataSource.getConnection();
        PreparedStatement preparedStatement = conn.prepareStatement(sql)) {
    hintManager.setMasterRouteOnly();
    try (ResultSet rs = preparedStatement.executeQuery()) {
        while (rs.next()) {
            // ...
        }
    }
}

 

編排治理(想起了mooc服務計算與服務生態系統)

使用治理功能須要指定一個註冊中心. 配置將所有存入註冊中心, 能夠在每次啓動時使用本地配置覆蓋註冊中心配置,

也能夠只經過註冊中心讀取配置.

不使用Spring

引入Maven依賴:

<dependency>
    <groupId>org.apache.shardingsphere</groupId>
    <artifactId>sharding-jdbc-orchestration</artifactId>
    <version>${sharding-sphere.version}</version>
</dependency>

<!--若使用zookeeper, 請加入下面Maven座標-->
<dependency>
    <groupId>org.apache.shardingsphere</groupId>
    <artifactId>sharding-orchestration-reg-zookeeper-curator</artifactId>
</dependency>

<!--若使用etcd, 請下面Maven座標-->
<dependency>
    <groupId>org.apache.shardingsphere</groupId>
    <artifactId>sharding-orchestration-reg-etcd</artifactId>
</dependency>

基於Java編碼的規則配置:

    // 省略配置dataSourceMap以及shardingRuleConfig
    // ...

    // 配置註冊中心
    RegistryCenterConfiguration regConfig = new RegistryCenterConfiguration();
    regConfig.setServerLists("localhost:2181");
    regConfig.setNamespace("sharding-sphere-orchestration");

    // 配置治理
    OrchestrationConfiguration orchConfig = new OrchestrationConfiguration("orchestration-sharding-data-source", regConfig, false);

    // 獲取數據源對象
    DataSource dataSource = OrchestrationShardingDataSourceFactory.createDataSource(dataSourceMap, shardingRuleConfig, new ConcurrentHashMap(), new Properties(), orchConfig);

基於Yaml的規則配置

或經過Yaml方式配置,與以上配置等價:

orchestration:
  name: orchestration-sharding-data-source
  overwrite: false
  registry:
    serverLists: localhost:2181
    namespace: sharding-sphere-orchestration
    DataSource dataSource = YamlOrchestrationShardingDataSourceFactory.createDataSource(yamlFile);

使用Spring

引入Maven依賴:

<!-- for spring boot -->
<dependency>
    <groupId>org.apache.shardingsphere</groupId>
    <artifactId>sharding-jdbc-orchestration-spring-boot-starter</artifactId>
    <version>${sharding-sphere.version}</version>
</dependency>

<!--若使用zookeeper, 請加入下面Maven座標-->
<dependency>
    <groupId>org.apache.shardingsphere</groupId>
    <artifactId>sharding-orchestration-reg-zookeeper-curator</artifactId>
    <version>${sharding-sphere.version}</version>
</dependency>

<!--若使用etcd, 請加入下面Maven座標-->
<dependency>
    <groupId>org.apache.shardingsphere</groupId>
    <artifactId>sharding-orchestration-reg-etcd</artifactId>
    <version>${sharding-sphere.version}</version>
</dependency>
<!-- for spring namespace -->
<dependency>
    <groupId>org.apache.shardingsphere</groupId>
    <artifactId>sharding-jdbc-orchestration-spring-namespace</artifactId>
    <version>${sharding-sphere.version}</version>
</dependency>

<!--若使用zookeeper, 請加入下面Maven座標-->
<dependency>
    <groupId>org.apache.shardingsphere</groupId>
    <artifactId>sharding-orchestration-reg-zookeeper-curator</artifactId>
    <version>${sharding-sphere.version}</version>
</dependency>

<!--若使用etcd, 請加入下面Maven座標-->
<dependency>
    <groupId>org.apache.shardingsphere</groupId>
    <artifactId>sharding-orchestration-reg-etcd</artifactId>
    <version>${sharding-sphere.version}</version>
</dependency>

基於Spring boot的規則配置

spring.shardingsphere.orchestration.name=orchestration-sharding-data-source
spring.shardingsphere.orchestration.overwrite=false
spring.shardingsphere.orchestration.registry.server-lists=localhost:2181
spring.shardingsphere.orchestration.registry.namespace=sharding-jdbc-orchestration

基於Spring命名空間的規則配置:

<?xml version="1.0" encoding="UTF-8"?>
<beans xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
       xmlns:orchestraion="http://shardingsphere.apache.org/schema/shardingsphere/orchestration"
       xmlns="http://www.springframework.org/schema/beans"
       xsi:schemaLocation="http://www.springframework.org/schema/beans
                           http://www.springframework.org/schema/beans/spring-beans.xsd
                           http://shardingsphere.apache.org/schema/shardingsphere/orchestration
                           http://shardingsphere.apache.org/schema/shardingsphere/orchestration/orchestration.xsd">
     <import resource="namespace/shardingDataSourceNamespace.xml" />
     <orchestraion:registry-center id="regCenter" server-lists="localhost:3181" namespace="orchestration-spring-namespace-test" operation-timeout-milliseconds="1000" max-retries="3" />
     <orchestraion:sharding-data-source id="simpleShardingOrchestration" data-source-ref="simpleShardingDataSource" registry-center-ref="regCenter" />
</beans>

分佈式事務

兩階段提交-XA

引入Maven依賴

<dependency>
    <groupId>org.apache.shardingsphere</groupId>
    <artifactId>sharding-transaction-xa-core</artifactId>
    <version>${shardingsphere.version}</version>
</dependency>

XA事務管理器將以SPI的方式被Sharding-JDBC所加載.

Atomikos參數配置(可選)

ShardingSphere默認的XA事務管理器爲Atomikos, 在項目的logs目錄中會生成 xa_tx.log,

這是XA崩潰恢復時所需的日誌,請勿刪除.

也能夠經過在項目的classpath中添加 jta.properties 來訂製化Atomikos配置項.

第三方BASE實現-Saga

目前Apache/incubator-shardingsphere暫無BASE事務的實現,可是仍然能夠使用第三方實現的Saga事務.

https://shardingsphere.apache.org/document/current/cn/manual/sharding-jdbc/usage/transaction/

相關文章
相關標籤/搜索
每日一句
    每一个你不满意的现在，都有一个你没有努力的曾经。
本站公眾號
   歡迎關注本站公眾號,獲取更多信息
聯繫我們 最近搜索 最新文章 沪ICP备13005482号-10 MyBatis教程 SQL 教程 MySQL教程 Java 教程 Thymeleaf 教程 Hibernate教程 Spring教程 Redis教程