Sharding-JDBC 按日期時間分庫分表

時間  2021-01-30
標籤 java mysql git github web spring sql 數據庫 apache api 欄目 Java 简体版
原文   原文鏈接

簡介

Sharding-JDBC

定位爲輕量級Java框架，在Java的JDBC層提供的額外服務。 它使用客戶端直連數據庫，以jar包形式提供服務，無需額外部署和依賴，可理解爲加強版的JDBC驅動，徹底兼容JDBC和各類ORM框架。java

  • 適用於任何基於Java的ORM框架，如：JPA, Hibernate, Mybatis, Spring JDBC Template或直接使用JDBC。
  • 基於任何第三方的數據庫鏈接池，如：DBCP, C3P0, BoneCP, Druid, HikariCP等。
  • 支持任意實現JDBC規範的數據庫。目前支持MySQL，Oracle，SQLServer和PostgreSQL。

集成sharding-jdbc

首先建立建立數據庫和表這是sharding-jdbc所要求的。mysql

create database db_201906; create database db_201907; use db_201906; create table t_order_20190614(id int not null auto_increment,order_no varchar(16) NOT NULL,sys_time datetime,PRIMARY KEY (id)); create table t_order_20190615(id int not null auto_increment,order_no varchar(16) NOT NULL,sys_time datetime,PRIMARY KEY (id)); insert into t_order_20190614 values(0,'0123456789','2019-06-14 0:0:0'); insert into t_order_20190615 values(0,'0123456789','2019-06-15 0:0:0'); use db_201907; create table t_order_20190714(id int not null auto_increment,order_no varchar(16) NOT NULL,sys_time datetime,PRIMARY KEY (id)); create table t_order_20190715(id int not null auto_increment,order_no varchar(16) NOT NULL,sys_time datetime,PRIMARY KEY (id)); insert into t_order_20190714 values(0,'0123456789','2019-07-14 0:0:0'); insert into t_order_20190715 values(0,'0123456789','2019-07-15 0:0:0');

接着是pom.xml文件，添加sharding-jdbc到工程中，主要是下面兩個依賴：git

<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
  <modelVersion>4.0.0</modelVersion>

  <groupId>org.osource.aurora</groupId>
  <artifactId>shardingjdbc</artifactId>
  <version>0.0.1-SNAPSHOT</version>
  <packaging>jar</packaging>

  <name>shardingjdbc</name>
  <url>http://maven.apache.org</url>
 
  <parent>
      <groupId>org.springframework.boot</groupId>
      <artifactId>spring-boot-starter-parent</artifactId>
      <version>2.0.5.RELEASE</version>
      <relativePath/> <!-- lookup parent from repository -->
  </parent>

  <properties>
      <project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
      <project.reporting.outputEncoding>UTF-8</project.reporting.outputEncoding>
      <java.version>1.8</java.version>
  </properties>

  <dependencies>
    
    <dependency>
        <groupId>org.springframework.boot</groupId>
        <artifactId>spring-boot-starter-jdbc</artifactId>
    </dependency>
    
    <dependency>
        <groupId>org.springframework.boot</groupId>
        <artifactId>spring-boot-starter-web</artifactId>
    </dependency>
    
    <dependency>
        <groupId>org.mybatis.spring.boot</groupId>
        <artifactId>mybatis-spring-boot-starter</artifactId>
        <version>1.3.2</version>
    </dependency>
    
    <dependency>
        <groupId>mysql</groupId>
        <artifactId>mysql-connector-java</artifactId>
    </dependency>
    
    <dependency>
        <groupId>org.springframework.boot</groupId>
        <artifactId>spring-boot-starter-test</artifactId>
        <scope>test</scope>
    </dependency>
    
    <!--sharding-jdbc --> 
    <dependency>
        <groupId>io.shardingjdbc</groupId>
        <artifactId>sharding-jdbc-core</artifactId>
        <version>2.0.3</version>
    </dependency>

    <dependency>
        <groupId>commons-dbcp</groupId>
        <artifactId>commons-dbcp</artifactId>
        <version>1.3</version>
    </dependency>
        
  </dependencies>
  <build>
      <plugins>
      
          <plugin>
              <groupId>org.apache.maven.plugins</groupId>
              <artifactId>maven-scm-plugin</artifactId>
              <version>1.9.4</version>
              <configuration>
                  <connectionType>developerConnection</connectionType>
              </configuration>
          </plugin>

          <plugin>
              <groupId>org.apache.maven.plugins</groupId>
              <artifactId>maven-release-plugin</artifactId>
              <version>2.5.3</version>
              <configuration>
                  <releaseProfiles>release</releaseProfiles>
                  <autoVersionSubmodules>true</autoVersionSubmodules>
                  <tagBase>https://github.com/sharding/shardingjdbc-framework.git</tagBase>
                  <tagNameFormat>v@{project.version}</tagNameFormat>
              </configuration>
          </plugin>

          <plugin>
              <groupId>org.apache.maven.plugins</groupId>
              <artifactId>maven-javadoc-plugin</artifactId>
              <configuration>
                  <skip>true</skip>
                  <aggregate>true</aggregate>
                  <charset>UTF-8</charset>
                  <encoding>UTF-8</encoding>
                  <docencoding>UTF-8</docencoding>
              </configuration>
          </plugin>

          <plugin>
              <groupId>org.apache.maven.plugins</groupId>
              <artifactId>maven-jar-plugin</artifactId>
              <configuration>
                  <excludes>
                      <exclude>**/*.xml</exclude>
                  </excludes>
              </configuration>
          </plugin>
          
          <plugin>
              <groupId>org.apache.maven.plugins</groupId>
              <artifactId>maven-compiler-plugin</artifactId>
              <configuration>
                  <source>1.8</source>
                  <target>1.8</target>
                  <encoding>UTF-8</encoding>
              </configuration>
          </plugin>
          
          <plugin>
              <groupId>org.apache.maven.plugins</groupId>
              <artifactId>maven-surefire-plugin</artifactId>
              <configuration>
                  <!-- 設置成true在使用maven打包發佈時不作junit測試 -->
                  <skip>true</skip>
              </configuration>
          </plugin>
          
      </plugins>
  </build>
 
</project>

首先是數據源配置和庫策略、表策略：github

import java.sql.SQLException; import java.util.HashMap; import java.util.Map; import java.util.Properties; import java.util.concurrent.ConcurrentHashMap; import javax.sql.DataSource; import org.apache.commons.dbcp.BasicDataSource; import org.apache.ibatis.session.SqlSessionFactory; import org.mybatis.spring.SqlSessionFactoryBean; import org.mybatis.spring.SqlSessionTemplate; import org.springframework.beans.factory.annotation.Value; import org.springframework.boot.context.properties.ConfigurationProperties; import org.springframework.context.annotation.Bean; import org.springframework.context.annotation.Configuration; import org.springframework.context.annotation.Primary; import org.springframework.core.io.support.PathMatchingResourcePatternResolver; import org.springframework.jdbc.datasource.DataSourceTransactionManager; import io.shardingjdbc.core.api.config.ShardingRuleConfiguration; import io.shardingjdbc.core.api.config.TableRuleConfiguration; import io.shardingjdbc.core.api.config.strategy.StandardShardingStrategyConfiguration; import io.shardingjdbc.core.jdbc.core.datasource.ShardingDataSource; @Configuration public class ShardingDataSourceConfiguration { @Value("${spring.datasource.username:root}") private String username; @Value("${spring.datasource.password:123456}") private String password; @Value("${spring.datasource.url}") private String jdbcUrl; @Value("${spring.datasource.driver-class-name}") private String driverClassName; @Value("${shardingjdbc.sql.show:true}") private String sqlShow; @Value("${mybatis.mapper-locations:mappper/**/*.xml}") private String mapperLocations; // 配置sharding-jdbc的DataSource，給上層應用使用，這個DataSource包含全部的邏輯庫和邏輯表，應用增刪改查時，修改對應sql // 而後選擇合適的數據庫繼續操做。所以這個DataSource建立很重要。
 @Bean @Primary @ConfigurationProperties(prefix = "spring.datasource") public DataSource shardingDataSource() throws SQLException { ShardingRuleConfiguration shardingRuleConfig = new ShardingRuleConfiguration(); // 訂單表配置，能夠累計添加多個配置
 shardingRuleConfig.getTableRuleConfigs().add(getOrderTableRuleConfiguration()); // shardingRuleConfig.getTableRuleConfigs().add(getUserTableRuleConfiguration()); // 打印SQL
        Properties props = new Properties(); props.put("sql.show", sqlShow); return new ShardingDataSource(shardingRuleConfig.build(createDataSourceMap()), new ConcurrentHashMap<String, Object>(), props); } // 建立用戶表規則
 @Bean TableRuleConfiguration getOrderTableRuleConfiguration() { TableRuleConfiguration orderTableRuleConfig = new TableRuleConfiguration(); orderTableRuleConfig.setLogicTable("t_order"); orderTableRuleConfig.setLogicIndex("sys_time"); // 設置數據庫策略，傳入的是sys_time
 orderTableRuleConfig.setDatabaseShardingStrategyConfig( new StandardShardingStrategyConfiguration("sys_time", DatabaseShardingAlgorithm.class.getName())); // 設置數據表策略，傳入的是sys_time
 orderTableRuleConfig.setTableShardingStrategyConfig( new StandardShardingStrategyConfiguration("sys_time", TableShardingAlgorithm.class.getName())); // 設置數據節點，格式爲dbxx.tablexx。這裏的名稱要和map的別名一致。下面兩種方式均可以 // orderTableRuleConfig.setActualDataNodes("db_${0..1}.t_order_${0..1}");
 orderTableRuleConfig.setActualDataNodes( "db_201906.t_order_20190614,db_201906.t_order_20190615,db_201907.t_order_20190714,db_201907.t_order_20190715"); // 設置縱列名稱 // orderTableRuleConfig.setKeyGeneratorColumnName("ID");
        return orderTableRuleConfig; } // 下面函數是獲取數據源，即包含有多少個數據庫，讀入到系統中存放於map中
    private Map<String, DataSource> createDataSourceMap() { Map<String, DataSource> result = new HashMap<>(); result.put("db_201906", createDataSource("jdbc:mysql://localhost:3306/db_201906?characterEncoding=utf8&useSSL=false")); result.put("db_201907", createDataSource("jdbc:mysql://localhost:3306/db_201907?characterEncoding=utf8&useSSL=false")); return result; } private DataSource createDataSource(final String jdbcUrl) { // 使用默認鏈接池
        BasicDataSource result = new BasicDataSource(); // 指定driver的類名，默認從jdbc url中自動探測
        result.setDriverClassName(com.mysql.jdbc.Driver.class.getName()); // 設置數據庫路徑
 result.setUrl(jdbcUrl); // 設置數據庫用戶名
 result.setUsername(username); // 設置數據密碼
 result.setPassword(password); return result; } @Bean("sqlSessionFactory") @Primary public SqlSessionFactory sqlSessionFactory(DataSource shardingDataSource) throws Exception { SqlSessionFactoryBean sessionFactory = new SqlSessionFactoryBean(); sessionFactory.setDataSource(shardingDataSource); PathMatchingResourcePatternResolver resolver = new PathMatchingResourcePatternResolver(); sessionFactory.setMapperLocations(resolver.getResources(mapperLocations)); return sessionFactory.getObject(); } /** * - 須要手動配置事務管理器 */ @Bean public DataSourceTransactionManager transactitonManager(DataSource shardingDataSource) { return new DataSourceTransactionManager(shardingDataSource); } @Bean public SqlSessionTemplate sqlSessionTmplate(SqlSessionFactory sqlSessionFactory) { SqlSessionTemplate sqlSessionTmplate = new SqlSessionTemplate(sqlSessionFactory); return sqlSessionTmplate; } }

數據庫分庫策略 DatabaseShardingAlgorithmweb

import java.text.ParseException; import java.text.SimpleDateFormat; import java.util.Collection; import java.util.Date; import io.shardingjdbc.core.api.algorithm.sharding.PreciseShardingValue; import io.shardingjdbc.core.api.algorithm.sharding.standard.PreciseShardingAlgorithm; public class DatabaseShardingAlgorithm implements PreciseShardingAlgorithm<String> { @Override public String doSharding(Collection<String> collection, PreciseShardingValue<String> preciseShardingValue) { String db_name = "db_"; try { Date date = new SimpleDateFormat("yyyy-MM-dd HH:mm:ss").parse(preciseShardingValue.getValue()); String year = String.format("%tY", date); String mon = String.format("%tm", date); db_name = db_name + year + mon; System.out.println("db_name:" + db_name); } catch (ParseException e) { e.printStackTrace(); } for (String each : collection) { System.out.println("db:" + each); if (each.equals(db_name)) { return each; } } throw new IllegalArgumentException(); } }

數據表分表策略 TableShardingAlgorithmspring

import java.text.ParseException; import java.text.SimpleDateFormat; import java.util.Collection; import java.util.Date; import io.shardingjdbc.core.api.algorithm.sharding.PreciseShardingValue; import io.shardingjdbc.core.api.algorithm.sharding.standard.PreciseShardingAlgorithm; public class TableShardingAlgorithm implements PreciseShardingAlgorithm<String> { @Override public String doSharding(Collection<String> collection, PreciseShardingValue<String> preciseShardingValue) { String tb_name = preciseShardingValue.getLogicTableName() + "_"; try { Date date = new SimpleDateFormat("yyyy-MM-dd HH:mm:ss").parse(preciseShardingValue.getValue()); String year = String.format("%tY", date); String mon = String.format("%tm", date); String dat = String.format("%td", date); tb_name = tb_name + year + mon + dat; System.out.println("tb_name:" + tb_name); } catch (ParseException e) { e.printStackTrace(); } for (String each : collection) { System.out.println("t_order_:" + each); if (each.equals(tb_name)) { return each; } } throw new IllegalArgumentException(); } }

 測試並使用sharding-jdbc

上面基本已經完成了sharding-jdbc的集成，下面將進行測試sql

建立實體類 Order數據庫

import java.io.Serializable; public class Order implements Serializable { /** * */
    private static final long serialVersionUID = -8759492936340749287L; private String orderNo; private String sysTime; public String getOrderNo() { return orderNo; } public void setOrderNo(String orderNo) { this.orderNo = orderNo; } public String getSysTime() { return sysTime; } public void setSysTime(String sysTime) { this.sysTime = sysTime; } @Override public String toString() { return "Order [orderNo=" + orderNo + ", sysTime=" + sysTime + "]"; } }

建立服務接口類OrderService apache

import java.util.List; public interface OrderService { List<Order> getAll(String sysTime); }

建立服務接口類實現 OrderServiceImplapi

import java.util.List; import org.springframework.beans.factory.annotation.Autowired; import org.springframework.stereotype.Service; @Service("orderService") public class OrderServiceImpl implements OrderService { @Autowired private OrderMapper orderMapper; @Override public List<Order> getAll(String sysTime) { return orderMapper.findAll(sysTime); } }

建立 OrderMapper 數據訪問層

import java.util.List; import org.apache.ibatis.annotations.Mapper; import org.apache.ibatis.annotations.Param; import org.apache.ibatis.annotations.Select; @Mapper public interface OrderMapper { @Select("select * from order where sys_time = #{sysTime}") List<Order> findAll(@Param("sys_time") String sysTime); }

建立 Spring Boot 啓動類 SpringBootApplicationTest

import org.mybatis.spring.annotation.MapperScan; import org.springframework.boot.SpringApplication; import org.springframework.boot.autoconfigure.EnableAutoConfiguration; import org.springframework.boot.autoconfigure.SpringBootApplication; import org.springframework.context.annotation.ComponentScan; @SpringBootApplication //開啓通用註解掃描
@ComponentScan(basePackages = { "org.shardingjdbc" }) @MapperScan(basePackages = { "org.shardingjdbc.mapper.user,org.shardingjdbc.mapper.order" }) @EnableAutoConfiguration public class SpringBootApplicationTest { public static void main(String[] args) { SpringApplication.run(SpringBootApplicationTest.class, args); } }

RestController 測試代碼

import java.util.List; import org.springframework.beans.factory.annotation.Autowired; import org.springframework.beans.factory.annotation.Qualifier; import org.springframework.web.bind.annotation.RequestMapping; import org.springframework.web.bind.annotation.RestController; @RestController public class OrderController { @Autowired @Qualifier("orderService") private OrderService orderService; @RequestMapping("/get") public String getOrder() { List<Order> list = orderService.getAll("2019-06-14 0:0:0"); System.out.println(list); return "OK"; } }

SpringBootTest 測試方法

import java.util.List; import org.junit.Test; import org.junit.runner.RunWith; import org.springframework.beans.factory.annotation.Autowired; import org.springframework.beans.factory.annotation.Qualifier; import org.springframework.boot.autoconfigure.EnableAutoConfiguration; import org.springframework.boot.test.context.SpringBootTest; import org.springframework.test.context.junit4.SpringRunner; /** * Unit test for simple App. */ @RunWith(SpringRunner.class) @SpringBootTest(classes = SpringBootApplicationTest.class) @EnableAutoConfiguration public class AppTest { @Autowired @Qualifier("orderService") private OrderService orderService; @Test public void getOrder() { List<Order> list = orderService.getAll("2019-06-14 0:0:0"); System.out.println(list); } }

sharding-jdbc 分庫分表實現方式到此基本完成

相關文章
相關標籤/搜索
每日一句
    每一个你不满意的现在，都有一个你没有努力的曾经。
本站公眾號
   歡迎關注本站公眾號,獲取更多信息
聯繫我們 最近搜索 最新文章 沪ICP备13005482号-10 MyBatis教程 SQL 教程 MySQL教程 Java 教程 Thymeleaf 教程 Hibernate教程 Spring教程 Redis教程