Jedis的Sharded源代碼分析

本專欄與Redis相關的文章

Redis Sentinel機制與用法（一）
Redis Sentinel機制與用法（二）
Jedis的JedisSentinelPool源代碼分析
Jedis的Sharded源代碼分析
Redis 主從 Replication 的配置
詳解Redis SORT命令
JedisCommand接口說明

概述

Jedis是Redis官方推薦的Java客戶端，更多Redis的客戶端能夠參考Redis官網客戶端列表。當業務的數據量很是龐大時，須要考慮將數據存儲到多個緩存節點上，如何定位數據應該存儲的節點，通常用的是一致性哈希算法。Jedis在客戶端角度實現了一致性哈希算法，對數據進行分片，存儲到對應的不一樣的redis實例中。
Jedis對Sharded的實現主要是在ShardedJedis.java和ShardedJedisPool.java中。本文主要介紹ShardedJedis的實現，ShardedJedisPool是基於apache的common-pool2的對象池實現。

繼承關係

ShardedJedis--->BinaryShardedJedis--->Sharded <Jedis, JedisShardInfo>

構造函數

查看其構造函數

public ShardedJedis(List<JedisShardInfo> shards, Hashing algo, Pattern keyTagPattern) {
        super(shards, algo, keyTagPattern);
    }

構造器參數解釋：

• shards是一個JedisShardInfo的列表，一個JedisShardedInfo類表明一個數據分片的主體。
• algo是用來進行數據分片的算法
• keyTagPattern，自定義分片算法所依據的key的形式。例如，能夠不針對整個key的字符串作哈希計算，而是相似對thisisa{key}中包含在大括號內的字符串進行哈希計算。

JedisShardInfo是什麼樣的？

public class JedisShardInfo extends ShardInfo<Jedis> {

  public String toString() {
    return host + ":" + port + "*" + getWeight();
  }

  private int connectionTimeout;
  private int soTimeout;
  private String host;
  private int port;
  private String password = null;
  private String name = null;
  // Default Redis DB
  private int db = 0;

  public String getHost() {
    return host;
  }

  public int getPort() {
    return port;
  }

  public JedisShardInfo(String host) {
    super(Sharded.DEFAULT_WEIGHT);
    URI uri = URI.create(host);
    if (JedisURIHelper.isValid(uri)) {
      this.host = uri.getHost();
      this.port = uri.getPort();
      this.password = JedisURIHelper.getPassword(uri);
      this.db = JedisURIHelper.getDBIndex(uri);
    } else {
      this.host = host;
      this.port = Protocol.DEFAULT_PORT;
    }
  }

  public JedisShardInfo(String host, String name) {
    this(host, Protocol.DEFAULT_PORT, name);
  }

  public JedisShardInfo(String host, int port) {
    this(host, port, 2000);
  }

  public JedisShardInfo(String host, int port, String name) {
    this(host, port, 2000, name);
  }

  public JedisShardInfo(String host, int port, int timeout) {
    this(host, port, timeout, timeout, Sharded.DEFAULT_WEIGHT);
  }

  public JedisShardInfo(String host, int port, int timeout, String name) {
    this(host, port, timeout, timeout, Sharded.DEFAULT_WEIGHT);
    this.name = name;
  }

  public JedisShardInfo(String host, int port, int connectionTimeout, int soTimeout, int weight) {
    super(weight);
    this.host = host;
    this.port = port;
    this.connectionTimeout = connectionTimeout;
    this.soTimeout = soTimeout;
  }

  public JedisShardInfo(String host, String name, int port, int timeout, int weight) {
    super(weight);
    this.host = host;
    this.name = name;
    this.port = port;
    this.connectionTimeout = timeout;
    this.soTimeout = timeout;
  }

  public JedisShardInfo(URI uri) {
    super(Sharded.DEFAULT_WEIGHT);
    if (!JedisURIHelper.isValid(uri)) {
      throw new InvalidURIException(String.format(
        "Cannot open Redis connection due invalid URI. %s", uri.toString()));
    }

    this.host = uri.getHost();
    this.port = uri.getPort();
    this.password = JedisURIHelper.getPassword(uri);
    this.db = JedisURIHelper.getDBIndex(uri);
  }

@Override
  public Jedis createResource() {
    return new Jedis(this);
  }
    /**
    *    省略setters和getters
    **/
}

可見JedisShardInfo包含了一個redis節點ip地址，端口號，name，密碼等等相關信息。要構造一個ShardedJedis，提供一個或多個JedisShardInfo。

最終構造函數的實如今其父類Sharded裏面

public Sharded(List<S> shards, Hashing algo, Pattern tagPattern) {
        this.algo = algo;
        this.tagPattern = tagPattern;
        initialize(shards);
    }

哈希環的初始化

Sharded類裏面維護了一個TreeMap，基於紅黑樹實現，用來盛放通過一致性哈希計算後的redis節點，另外維護了一個LinkedHashMap，用來保存ShardInfo與Jedis實例的對應關係。
定位的流程以下
先在TreeMap中找到對應key所對應的ShardInfo，而後經過ShardInfo在LinkedHashMap中找到對應的Jedis實例。

Sharded類對這些實例變量的定義以下所示：

public static final int DEFAULT_WEIGHT = 1;
    private TreeMap<Long, S> nodes;
    private final Hashing algo;
    private final Map<ShardInfo<R>, R> resources = new LinkedHashMap<ShardInfo<R>, R>();

    /**
     * The default pattern used for extracting a key tag. The pattern must have
     * a group (between parenthesis), which delimits the tag to be hashed. A
     * null pattern avoids applying the regular expression for each lookup,
     * improving performance a little bit is key tags aren't being used.
     */
    private Pattern tagPattern = null;
    // the tag is anything between {}
    public static final Pattern DEFAULT_KEY_TAG_PATTERN = Pattern.compile("\\{(.+?)\\}");

接下來看其構造函數中的initialize方法

private void initialize(List<S> shards) {
        nodes = new TreeMap<Long, S>();

        for (int i = 0; i != shards.size(); ++i) {
            final S shardInfo = shards.get(i);
            if (shardInfo.getName() == null)
                for (int n = 0; n < 160 * shardInfo.getWeight(); n++) {
                    nodes.put(this.algo.hash("SHARD-" + i + "-NODE-" + n), shardInfo);
                }
            else
                for (int n = 0; n < 160 * shardInfo.getWeight(); n++) {
                    nodes.put(
                            this.algo.hash(shardInfo.getName() + "*"
                                    + shardInfo.getWeight() + n), shardInfo);
                }
            resources.put(shardInfo, shardInfo.createResource());
        }
    }

能夠看到，它對每個ShardInfo經過必定規則計算其哈希值，而後存到TreeMap中，這裏它實現了一致性哈希算法中虛擬節點的概念，由於咱們能夠看到同一個ShardInfo不止一次被放到TreeMap中，數量是，權重*160。
增長了虛擬節點的一致性哈希有不少好處，能避免數據在redis節點間分佈不均勻。

而後，在LinkedHashMap中放入ShardInfo以及其對應的Jedis實例，經過調用其自身的createSource()來獲得jedis實例。

數據定位

從ShardedJedis的代碼中能夠看到，不管進行什麼操做，都要先根據key來找到對應的Redis，而後返回一個可供操做的Jedis實例。

例如其set方法：

public String set(String key, String value) {
        Jedis j = getShard(key);
        return j.set(key, value);
    }

而getShard方法則在Sharded.java中實現，其源代碼以下所示：

public R getShard(byte[] key) {
        return resources.get(getShardInfo(key));
    }

    public R getShard(String key) {
        return resources.get(getShardInfo(key));
    }

    public S getShardInfo(byte[] key) {
        SortedMap<Long, S> tail = nodes.tailMap(algo.hash(key));
        if (tail.isEmpty()) {
            return nodes.get(nodes.firstKey());
        }
        return tail.get(tail.firstKey());
    }

    public S getShardInfo(String key) {
        return getShardInfo(SafeEncoder.encode(getKeyTag(key)));
    }

能夠看到，先經過getShardInfo方法從TreeMap中得到對應的ShardInfo，而後根據這個ShardInfo就可以再LinkedHashMap中得到對應的Jedis實例了。