springboot(十三)-分庫分表-手動配置

時間  2019-11-08
標籤 springboot 十三 分庫 分表 手動 配置 欄目 Spring 简体版
原文   原文鏈接

sharding-jdbc簡介

Sharding-JDBC直接封裝JDBC API，能夠理解爲加強版的JDBC驅動，舊代碼遷移成本幾乎爲零：
可適用於任何基於java的ORM框架，如：JPA, Hibernate, Mybatis, Spring JDBC Template或直接使用JDBC。
可基於任何第三方的數據庫鏈接池，如：DBCP, C3P0, BoneCP, Druid等。
理論上可支持任意實現JDBC規範的數據庫。雖然目前僅支持MySQL，但已有支持Oracle，SQLServer，DB2等數據庫的計劃。
Sharding-JDBC定位爲輕量級java框架，使用客戶端直連數據庫，以jar包形式提供服務，未使用中間層，無需額外部署，無其餘依賴，DBA也無需改變原有的運維方式。SQL解析使用Druid解析器，是目前性能最高的SQL解析器。
具體的介紹能夠上它的文檔那裏看看，簡單概括起來就是，它是一個加強版的JDBC，對使用者透明，邏輯代碼什麼的都不用動，它來完成分庫分表的操做；而後它還支持分佈式事務（不完善）。看起來很不錯的樣子。
下面用個小例子來看一下分庫分表的使用。使用的是SpringBoot，mybatis，DBCP鏈接池。

java

1.新建一個springboot項目

ArtifactId爲sharding-jdbc-manualConfiguration.本身配置好目錄結構。mysql

 

 

2.pom.xml

<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
  xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
  <modelVersion>4.0.0</modelVersion>

  <groupId>com.itmuch.boot</groupId>
  <artifactId>sharding-jdbc-manualConfiguration</artifactId>
  <version>0.0.1-SNAPSHOT</version>
  <packaging>jar</packaging>

  <name>sharding-jdbc-manualConfiguration</name>
  <url>http://maven.apache.org</url>

  <parent>
        <groupId>org.springframework.boot</groupId>
        <artifactId>spring-boot-starter-parent</artifactId>
        <version>1.5.9.RELEASE</version>
        <relativePath/> <!-- lookup parent from repository -->
    </parent>

    <properties>
        <project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
        <project.reporting.outputEncoding>UTF-8</project.reporting.outputEncoding>
        <java.version>1.8</java.version>
    </properties>

    <dependencies>
        <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-starter-web</artifactId>
        </dependency>
        <dependency>
            <groupId>org.mybatis.spring.boot</groupId>
            <artifactId>mybatis-spring-boot-starter</artifactId>
            <version>1.3.1</version>
        </dependency>

        <dependency>
            <groupId>mysql</groupId>
            <artifactId>mysql-connector-java</artifactId>
            <scope>runtime</scope>
        </dependency>
        <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-starter-test</artifactId>
            <scope>test</scope>
        </dependency>

        <!--sharding-jdbc -->
        <dependency>
            <groupId>io.shardingjdbc</groupId>
            <artifactId>sharding-jdbc-core</artifactId>
            <version>2.0.3</version>
        </dependency>
        <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-devtools</artifactId>
        </dependency>
<!--         <dependency> -->
<!--             <groupId>com.alibaba</groupId> -->
<!--             <artifactId>druid</artifactId> -->
<!--             <version>1.1.3</version> -->
<!--         </dependency> -->
        <dependency>
            <groupId>commons-dbcp</groupId>
            <artifactId>commons-dbcp</artifactId>
        </dependency>
        <dependency>
            <groupId>mysql</groupId>
            <artifactId>mysql-connector-java</artifactId>
        </dependency>

        <!-- https://mvnrepository.com/artifact/com.alibaba/fastjson -->
        <dependency>
            <groupId>com.alibaba</groupId>
            <artifactId>fastjson</artifactId>
            <version>1.2.47</version>
        </dependency>

    </dependencies>

    <build>
        <plugins>
            <plugin>
                <groupId>org.springframework.boot</groupId>
                <artifactId>spring-boot-maven-plugin</artifactId>
            </plugin>
        </plugins>
    </build>
</project>

簡單提一下：git

這裏主要是這個依賴web

　　　　 <dependency>
            <groupId>io.shardingjdbc</groupId>
            <artifactId>sharding-jdbc-core</artifactId>
            <version>2.0.3</version>
        </dependency>

它是噹噹網開源的sharding-jdbc，固然這個不重要啦！算法

順便說一下：spring

<!--         <dependency> -->
<!--             <groupId>com.alibaba</groupId> -->
<!--             <artifactId>druid</artifactId> -->
<!--             <version>1.1.3</version> -->
<!--         </dependency> -->
        <dependency>
            <groupId>commons-dbcp</groupId>
            <artifactId>commons-dbcp</artifactId>
        </dependency>

我是用的DBCP數據庫鏈接池，這裏的druid鏈接池被我注掉了，固然你也能夠使用它，把DBCP注掉。它們都差很少。我想不少其餘博友應該有寫。你們找找看咯！sql

在這裏我要吐槽下：TMD，CSDN博客上面的文章叫什麼東西啊，我讀過幾回，在上面down下來的代碼都是不能運行的，而後在博客園上面再找相同內容的東西，一比對，發現很明顯的配置漏洞。我也是無語了。數據庫

因此我是推薦咱博客園的文章的，你們都很優秀！！！apache

application.yml

咱們是手動配置數據源，那這裏咱們能夠上面都不用謝了呀！直接放個空文件得了，固然你若是不想使用8080端口，想在這裏配置一下當前項目的使用端口號，你能夠在這裏配置下咯！！！json

 數據庫

咱們要準備三個數據庫user_0,user_1,user_2,每一個數據庫裏準備兩張表user_info_0,user_info_1.一共使用六張同樣同樣的表。

DROP TABLE IF EXISTS `user_info_0`;
CREATE TABLE `user_info_0` (
  `user_id` bigint(19) NOT NULL,
  `user_name` varchar(45) DEFAULT NULL,
  `account` varchar(45) DEFAULT NULL,
  `password` varchar(45) DEFAULT NULL,
  PRIMARY KEY (`user_id`)
) ENGINE=InnoDB DEFAULT CHARSET=utf8;

-- ----------------------------
-- Table structure for user_info_1
-- ----------------------------
DROP TABLE IF EXISTS `user_info_1`;
CREATE TABLE `user_info_1` (
  `user_id` bigint(19) NOT NULL,
  `user_name` varchar(45) DEFAULT NULL,
  `account` varchar(45) DEFAULT NULL,
  `password` varchar(45) DEFAULT NULL,
  PRIMARY KEY (`user_id`)
) ENGINE=InnoDB DEFAULT CHARSET=utf8;

Application.java

@SpringBootApplication
@EnableAutoConfiguration(exclude = { DataSourceAutoConfiguration.class })
public class Application {

    public static void main(String[] args) {
        SpringApplication.run(Application.class, args);
    }
}

啓動類嘛，咱們都同樣！不不不。。。注意了，多了個註解，看到它要幹嗎了嗎？它要把自動配置數據源的功能排除掉。

這裏你要是看過我前面的文章「爲何用springboot」中的springboot自動配置的原理，這個東西，so easy啦！

 testController.java

@RestController
public class testController {
    @Resource
    UserInfoMapper userInfoMaper;

    @Resource
    DemoService demoService;

    @GetMapping("insert/{id}")
    public String insertData(@PathVariable Long id) {
        demoService.demo();
        return "success";
    }

    @GetMapping("get/{id}")
    public String getData(@PathVariable Long id) {
        UserInfo userInfoByUserId = demoService.getUserInfoByUserId(id);
        System.out.println("獲得的結果爲：" + JSON.toJSON(userInfoByUserId));
        return JSON.toJSON(userInfoByUserId).toString();
    }
}

對外接口，咱們提供兩個吧，向數據庫插入數據傳的id沒用，我傳了玩兒的！！！

插入的內容請看下面的service.

DemoService.java

@Service
public class DemoService {

    @Resource
    UserInfoMapper userInfoMapper;

    public static Long userId = 100L;

    public void demo() {
        System.out.println("Insert--------------");

        for (int i = 1; i <= 100; i++) {
            UserInfo userInfo = new UserInfo();
            userInfo.setUserId(userId);
            System.out.println(userId);

            userInfo.setAccount("Account" + i);
            userInfo.setPassword("pass" + i);
            userInfo.setUserName("name" + i);
            userId++;
            userInfoMapper.insert(userInfo);
            System.out.println("第" + i + "條");
        }
        System.out.println("over..........");
    }

    public UserInfo getUserInfoByUserId(Long id) {
        return userInfoMapper.selectByPrimaryKey(id);
    }
}

咱們就試驗向數據庫插100條數據咯，再提供下查詢方法。哦哦，Entity在下面

UserInfo.java

 1 public class UserInfo {
 2     private Long userId;
 3 
 4     private String userName;
 5 
 6     private String account;
 7 
 8     private String password;
 9 
10     public Long getUserId() {
11         return userId;
12     }
13 
14     public void setUserId(Long userId) {
15         this.userId = userId;
16     }
17 
18     public String getUserName() {
19         return userName;
20     }
21 
22     public void setUserName(String userName) {
23         this.userName = userName == null ? null : userName.trim();
24     }
25 
26     public String getAccount() {
27         return account;
28     }
29 
30     public void setAccount(String account) {
31         this.account = account == null ? null : account.trim();
32     }
33 
34     public String getPassword() {
35         return password;
36     }
37 
38     public void setPassword(String password) {
39         this.password = password == null ? null : password.trim();
40     }
41 }
點擊展開

UserInfoMapper.java

@Mapper
public interface UserInfoMapper {
    /**
     * This method was generated by MyBatis Generator. This method corresponds to
     * the database table user_info
     *
     * @mbg.generated Tue Mar 13 23:47:19 CST 2018
     */
    int insert(UserInfo record);

    /**
     * This method was generated by MyBatis Generator. This method corresponds to
     * the database table user_info
     *
     * @mbg.generated Tue Mar 13 23:47:19 CST 2018
     */
    int insertSelective(UserInfo record);

    /**
     * This method was generated by MyBatis Generator. This method corresponds to
     * the database table user_info
     *
     * @mbg.generated Tue Mar 13 23:47:19 CST 2018
     */
    UserInfo selectByPrimaryKey(Long userId);

    /**
     * This method was generated by MyBatis Generator. This method corresponds to
     * the database table user_info
     *
     * @mbg.generated Tue Mar 13 23:47:19 CST 2018
     */
    int updateByPrimaryKeySelective(UserInfo record);

    /**
     * This method was generated by MyBatis Generator. This method corresponds to
     * the database table user_info
     *
     * @mbg.generated Tue Mar 13 23:47:19 CST 2018
     */
    int updateByPrimaryKey(UserInfo record);
}

由於是用mybatis的逆向工程自動生成的，東西多餘了點。懶得刪了。可是注意下，類名上面的@Mapper註解是我加的。

這裏我展開說一下，我在網上看到不少人在springboot項目中使用Mybatis做爲持久層框架的時候，都不這麼用。都是在resource根目錄下面建一個文件夾mapper,而後把全部的XXXMapper.xml文件放到裏面，XXXMapper.java仍是放到com...mapper中不變。而後在啓動類上面添加一個針對這個com...mapper文件夾的@ComponentScan(「com...mapper」)註解。還要在根目錄resource下面配置mybatis-config.xml文件。更有甚者，竟然有人在application.properties文件或application.yml文件中配置mybatis的東西，例如：

mybatis:
  mapper- locations: classpath:mybatis/mapper/*.xml
  type-aliases- package: com.sun.shard.bean

雖然springboot也能支持這樣使用，可是我是不建議也不喜歡這樣的風格，由於你想啊，咱們爲何要用springboot?不就是想盡可能少寫點配置嘛，把這些繁瑣的東西交給springboot來作就得了！要否則springboot出mybatis-spring-boot-starter這個啓動器幹啥！

對了，你們想了解爲啥，建議你們本身看下源碼，你加了mybatis-spring-boot-starter這個依賴，在Maven Dependencies下面找到

看明白這個類，基本就沒啥東西了。

 扯遠了，咱們繼續咱們的分庫分表。

UserInfoMapper.xml

  1 <?xml version="1.0" encoding="UTF-8"?>
  2 <!DOCTYPE mapper PUBLIC "-//mybatis.org//DTD Mapper 3.0//EN" "http://mybatis.org/dtd/mybatis-3-mapper.dtd">
  3 <mapper namespace="com.itmuch.boot.mapper.UserInfoMapper">
  4   <resultMap id="BaseResultMap" type="com.itmuch.boot.entity.UserInfo">
  5     <!--
  6       WARNING - @mbg.generated
  7       This element is automatically generated by MyBatis Generator, do not modify.
  8       This element was generated on Tue Mar 13 23:47:19 CST 2018.
  9     -->
 10     <id column="user_id" jdbcType="BIGINT" property="userId" />
 11     <result column="user_name" jdbcType="VARCHAR" property="userName" />
 12     <result column="account" jdbcType="VARCHAR" property="account" />
 13     <result column="password" jdbcType="VARCHAR" property="password" />
 14   </resultMap>
 15   <sql id="Base_Column_List">
 16     <!--
 17       WARNING - @mbg.generated
 18       This element is automatically generated by MyBatis Generator, do not modify.
 19       This element was generated on Tue Mar 13 23:47:19 CST 2018.
 20     -->
 21     user_id, user_name, account, password
 22   </sql>
 23   <select id="selectByPrimaryKey" parameterType="java.lang.Long" resultMap="BaseResultMap">
 24     <!--
 25       WARNING - @mbg.generated
 26       This element is automatically generated by MyBatis Generator, do not modify.
 27       This element was generated on Tue Mar 13 23:47:19 CST 2018.
 28     -->
 29     select 
 30     <include refid="Base_Column_List" />
 31     from user_info
 32     where user_id = #{userId,jdbcType=BIGINT}
 33   </select>
 34   <insert id="insert" parameterType="com.itmuch.boot.entity.UserInfo">
 35     <!--
 36       WARNING - @mbg.generated
 37       This element is automatically generated by MyBatis Generator, do not modify.
 38       This element was generated on Tue Mar 13 23:47:19 CST 2018.
 39     -->
 40     insert into user_info (user_id, user_name, account, 
 41       password)
 42     values (#{userId,jdbcType=BIGINT}, #{userName,jdbcType=VARCHAR}, #{account,jdbcType=VARCHAR}, 
 43       #{password,jdbcType=VARCHAR})
 44   </insert>
 45   <insert id="insertSelective" parameterType="com.itmuch.boot.entity.UserInfo">
 46     <!--
 47       WARNING - @mbg.generated
 48       This element is automatically generated by MyBatis Generator, do not modify.
 49       This element was generated on Tue Mar 13 23:47:19 CST 2018.
 50     -->
 51     insert into user_info
 52     <trim prefix="(" suffix=")" suffixOverrides=",">
 53       <if test="userId != null">
 54         user_id,
 55       </if>
 56       <if test="userName != null">
 57         user_name,
 58       </if>
 59       <if test="account != null">
 60         account,
 61       </if>
 62       <if test="password != null">
 63         password,
 64       </if>
 65     </trim>
 66     <trim prefix="values (" suffix=")" suffixOverrides=",">
 67       <if test="userId != null">
 68         #{userId,jdbcType=BIGINT},
 69       </if>
 70       <if test="userName != null">
 71         #{userName,jdbcType=VARCHAR},
 72       </if>
 73       <if test="account != null">
 74         #{account,jdbcType=VARCHAR},
 75       </if>
 76       <if test="password != null">
 77         #{password,jdbcType=VARCHAR},
 78       </if>
 79     </trim>
 80   </insert>
 81   <update id="updateByPrimaryKeySelective" parameterType="com.itmuch.boot.entity.UserInfo">
 82     <!--
 83       WARNING - @mbg.generated
 84       This element is automatically generated by MyBatis Generator, do not modify.
 85       This element was generated on Tue Mar 13 23:47:19 CST 2018.
 86     -->
 87     update user_info
 88     <set>
 89       <if test="userName != null">
 90         user_name = #{userName,jdbcType=VARCHAR},
 91       </if>
 92       <if test="account != null">
 93         account = #{account,jdbcType=VARCHAR},
 94       </if>
 95       <if test="password != null">
 96         password = #{password,jdbcType=VARCHAR},
 97       </if>
 98     </set>
 99     where user_id = #{userId,jdbcType=BIGINT}
100   </update>
101   <update id="updateByPrimaryKey" parameterType="com.itmuch.boot.entity.UserInfo">
102     <!--
103       WARNING - @mbg.generated
104       This element is automatically generated by MyBatis Generator, do not modify.
105       This element was generated on Tue Mar 13 23:47:19 CST 2018.
106     -->
107     update user_info
108     set user_name = #{userName,jdbcType=VARCHAR},
109       account = #{account,jdbcType=VARCHAR},
110       password = #{password,jdbcType=VARCHAR}
111     where user_id = #{userId,jdbcType=BIGINT}
112   </update>
113 </mapper>
點擊展開

 

 

 到這裏咱們一套流程算是下來了。接下來是重點，手動添加數據源配置。

DataSourceConfig.java

 1 @Configuration
 2 @MapperScan(basePackages = "com.itmuch.boot.mapper")
 3 public class DataSourceConfig {
 4 
 5     @Bean(name = "shardingDataSource")
 6     DataSource getShardingDataSource() throws SQLException {
 7         ShardingRuleConfiguration shardingRuleConfig;
 8         shardingRuleConfig = new ShardingRuleConfiguration();
 9         shardingRuleConfig.getTableRuleConfigs().add(getUserTableRuleConfiguration());
10         shardingRuleConfig.getBindingTableGroups().add("user_info");
11         shardingRuleConfig.setDefaultDatabaseShardingStrategyConfig(
12                 new StandardShardingStrategyConfiguration("user_id", DemoDatabaseShardingAlgorithm.class.getName()));
13         shardingRuleConfig.setDefaultTableShardingStrategyConfig(
14                 new StandardShardingStrategyConfiguration("user_id", DemoTableShardingAlgorithm.class.getName()));
15         return new ShardingDataSource(shardingRuleConfig.build(createDataSourceMap()));
16     }
17 
18     @Bean
19     TableRuleConfiguration getUserTableRuleConfiguration() {
20         TableRuleConfiguration orderTableRuleConfig = new TableRuleConfiguration();
21         orderTableRuleConfig.setLogicTable("user_info");
22         orderTableRuleConfig.setActualDataNodes("user_${0..2}.user_info_${0..1}");
23         orderTableRuleConfig.setKeyGeneratorColumnName("user_id");
24         return orderTableRuleConfig;
25     }
26 
27     private Map<String, DataSource> createDataSourceMap() {
28         Map<String, DataSource> result = new HashMap<>();
29         result.put("user_0", createDataSource("user_0"));
30         result.put("user_1", createDataSource("user_1"));
31         result.put("user_2", createDataSource("user_2"));
32         return result;
33     }
34 
35     private DataSource createDataSource(final String dataSourceName) {
36         BasicDataSource result = new BasicDataSource();
37         result.setDriverClassName(com.mysql.jdbc.Driver.class.getName());
38         result.setUrl(String.format("jdbc:mysql://localhost:3306/%s", dataSourceName));
39 
40         result.setUsername("root");
41         result.setPassword("root");
42         return result;
43     }
44 
45 }

 

 

 

 這個數據源配置類你們也看到了，並不複雜，你看啊，咱們準備了三個數據庫user_0,user_1,user_2；每一個數據中準備了兩張相同的業務表user_info_0和user_info_1.帶着這個信息去看這個類，不用多說。

DemoDatabaseShardingAlgorithm.java

 

public class DemoDatabaseShardingAlgorithm implements PreciseShardingAlgorithm<Long> {
    @Override
    public String doSharding(Collection<String> collection, PreciseShardingValue<Long> preciseShardingValue) {
        for (String each : collection) {
            if (each.endsWith(Long.parseLong(preciseShardingValue.getValue().toString()) % 3 + "")) {
                return each;
            }
        }
        throw new IllegalArgumentException();
    }
}

 

 顧名思義，這是分庫算法，根據選擇的字段（上面DataSourceConfig.java第12行咱們選的字段是user_id）對3取餘。

 DemoTableShardingAlgorithm.java

public class DemoTableShardingAlgorithm implements PreciseShardingAlgorithm<Long> {
    @Override
    public String doSharding(Collection<String> collection, PreciseShardingValue<Long> preciseShardingValue) {
        for (String each : collection) {
            if (each.endsWith(Long.parseLong(preciseShardingValue.getValue().toString()) % 2 + "")) {
                return each;
            }
        }
        throw new IllegalArgumentException();
    }
}

 

 顧名思義，這是分表算法，根據選擇的字段（上面DataSourceConfig.java第14行咱們選的字段也是user_id）對2取餘。

 

測試

代碼就這麼多，啓動application.java類，咱們訪問localhost:8080/insert/1試試。

而後看數據庫：咱們以前建的三個庫，共六張表，四張表都是17條記錄，兩張表裏面分別有16條，一共100條。根據user_id判斷，沒有重複的，那基本哦了呀！

 

 

代碼地址：https://gitee.com/fengyuduke/my_open_resources/blob/master/sharding-jdbc-manualConfiguration.rar

相關文章
相關標籤/搜索
每日一句
    每一个你不满意的现在，都有一个你没有努力的曾经。
本站公眾號
   歡迎關注本站公眾號,獲取更多信息
聯繫我們 最近搜索 最新文章 沪ICP备13005482号-10 MyBatis教程 SQL 教程 MySQL教程 Java 教程 Thymeleaf 教程 Hibernate教程 Spring教程 Redis教程