Jedis的Sharded源代碼分析

概述

Jedis是Redis官方推薦的Java客戶端，更多Redis的客戶端能夠參考Redis官網客戶端列表。當業務的數據量很是龐大時，須要考慮將數據存儲到多個緩存節點上，如何定位數據應該存儲的節點，通常用的是一致性哈希算法。Jedis在客戶端角度實現了一致性哈希算法，對數據進行分片，存儲到對應的不一樣的redis實例中。
Jedis對Sharded的實現主要是在ShardedJedis.java和ShardedJedisPool.java中。本文主要介紹ShardedJedis的實現，ShardedJedisPool是基於apache的common-pool2的對象池實現。

繼承關係

ShardedJedis--->BinaryShardedJedis--->Sharded <Jedis, JedisShardInfo>

構造函數

查看其構造函數

public ShardedJedis(List<JedisShardInfo> shards, Hashing algo, Pattern keyTagPattern) { super(shards, algo, keyTagPattern); }

構造器參數解釋：

• shards是一個JedisShardInfo的列表，一個JedisShardedInfo類表明一個數據分片的主體。

• algo是用來進行數據分片的算法

• keyTagPattern，自定義分片算法所依據的key的形式。例如，能夠不針對整個key的字符串作哈希計算，而是相似對thisisa{key}中包含在大括號內的字符串進行哈希計算。

JedisShardInfo是什麼樣的？

public class JedisShardInfo extends ShardInfo<Jedis> { public String toString() { return host + ":" + port + "*" + getWeight(); } private int connectionTimeout; private int soTimeout; private String host; private int port; private String password = null; private String name = null; // Default Redis DB private int db = 0; public String getHost() { return host; } public int getPort() { return port; } public JedisShardInfo(String host) { super(Sharded.DEFAULT_WEIGHT); URI uri = URI.create(host); if (JedisURIHelper.isValid(uri)) { this.host = uri.getHost(); this.port = uri.getPort(); this.password = JedisURIHelper.getPassword(uri); this.db = JedisURIHelper.getDBIndex(uri); } else { this.host = host; this.port = Protocol.DEFAULT_PORT; } } public JedisShardInfo(String host, String name) { this(host, Protocol.DEFAULT_PORT, name); } public JedisShardInfo(String host, int port) { this(host, port, 2000); } public JedisShardInfo(String host, int port, String name) { this(host, port, 2000, name); } public JedisShardInfo(String host, int port, int timeout) { this(host, port, timeout, timeout, Sharded.DEFAULT_WEIGHT); } public JedisShardInfo(String host, int port, int timeout, String name) { this(host, port, timeout, timeout, Sharded.DEFAULT_WEIGHT); this.name = name; } public JedisShardInfo(String host, int port, int connectionTimeout, int soTimeout, int weight) { super(weight); this.host = host; this.port = port; this.connectionTimeout = connectionTimeout; this.soTimeout = soTimeout; } public JedisShardInfo(String host, String name, int port, int timeout, int weight) { super(weight); this.host = host; this.name = name; this.port = port; this.connectionTimeout = timeout; this.soTimeout = timeout; } public JedisShardInfo(URI uri) { super(Sharded.DEFAULT_WEIGHT); if (!JedisURIHelper.isValid(uri)) { throw new InvalidURIException(String.format( "Cannot open Redis connection due invalid URI. %s", uri.toString())); } this.host = uri.getHost(); this.port = uri.getPort(); this.password = JedisURIHelper.getPassword(uri); this.db = JedisURIHelper.getDBIndex(uri); } @Override public Jedis createResource() { return new Jedis(this); } /** * 省略setters和getters **/ }

可見JedisShardInfo包含了一個redis節點ip地址，端口號，name，密碼等等相關信息。要構造一個ShardedJedis，提供一個或多個JedisShardInfo。

最終構造函數的實如今其父類Sharded裏面

public Sharded(List<S> shards, Hashing algo, Pattern tagPattern) { this.algo = algo; this.tagPattern = tagPattern; initialize(shards); }

哈希環的初始化

Sharded類裏面維護了一個TreeMap，基於紅黑樹實現，用來盛放通過一致性哈希計算後的redis節點，另外維護了一個LinkedHashMap，用來保存ShardInfo與Jedis實例的對應關係。
定位的流程以下
先在TreeMap中找到對應key所對應的ShardInfo，而後經過ShardInfo在LinkedHashMap中找到對應的Jedis實例。

Sharded類對這些實例變量的定義以下所示：

public static final int DEFAULT_WEIGHT = 1; private TreeMap<Long, S> nodes; private final Hashing algo; private final Map<ShardInfo<R>, R> resources = new LinkedHashMap<ShardInfo<R>, R>(); /** * The default pattern used for extracting a key tag. The pattern must have * a group (between parenthesis), which delimits the tag to be hashed. A * null pattern avoids applying the regular expression for each lookup, * improving performance a little bit is key tags aren't being used. */ private Pattern tagPattern = null; // the tag is anything between {} public static final Pattern DEFAULT_KEY_TAG_PATTERN = Pattern.compile("\\{(.+?)\\}");

接下來看其構造函數中的initialize方法

private void initialize(List<S> shards) { nodes = new TreeMap<Long, S>(); for (int i = 0; i != shards.size(); ++i) { final S shardInfo = shards.get(i); if (shardInfo.getName() == null) for (int n = 0; n < 160 * shardInfo.getWeight(); n++) { nodes.put(this.algo.hash("SHARD-" + i + "-NODE-" + n), shardInfo); } else for (int n = 0; n < 160 * shardInfo.getWeight(); n++) { nodes.put( this.algo.hash(shardInfo.getName() + "*" + shardInfo.getWeight() + n), shardInfo); } resources.put(shardInfo, shardInfo.createResource()); } }

能夠看到，它對每個ShardInfo經過必定規則計算其哈希值，而後存到TreeMap中，這裏它實現了一致性哈希算法中虛擬節點的概念，由於咱們能夠看到同一個ShardInfo不止一次被放到TreeMap中，數量是，權重*160。
增長了虛擬節點的一致性哈希有不少好處，能避免數據在redis節點間分佈不均勻。

而後，在LinkedHashMap中放入ShardInfo以及其對應的Jedis實例，經過調用其自身的createSource()來獲得jedis實例。

數據定位

從ShardedJedis的代碼中能夠看到，不管進行什麼操做，都要先根據key來找到對應的Redis，而後返回一個可供操做的Jedis實例。

例如其set方法：

public String set(String key, String value) { Jedis j = getShard(key); return j.set(key, value); }

而getShard方法則在Sharded.java中實現，其源代碼以下所示：

public R getShard(byte[] key) { return resources.get(getShardInfo(key)); } public R getShard(String key) { return resources.get(getShardInfo(key)); } public S getShardInfo(byte[] key) { SortedMap<Long, S> tail = nodes.tailMap(algo.hash(key)); if (tail.isEmpty()) { return nodes.get(nodes.firstKey()); } return tail.get(tail.firstKey()); } public S getShardInfo(String key) { return getShardInfo(SafeEncoder.encode(getKeyTag(key))); } 

能夠看到，先經過getShardInfo方法從TreeMap中得到對應的ShardInfo，而後根據這個ShardInfo就可以再LinkedHashMap中得到對應的Jedis實例了。