Redis從入門到放棄系列(三) List

Redis從入門到放棄系列(三) List

本文例子基於：5.0.4 List是Redis中一種比較常見的數據結構,其實現爲quicklist,quicklist是一個ziplist的雙向鏈表

Redis從入門到放棄系列(一) String

Redis從入門到放棄系列(二) Hash

首先讓咱們來看一下該如何在redis裏面使用List類型

//設置key的列表爲value
lpush key value [value...]

代碼示例：

//棧的用法，rpush rpop同樣~ 經過rpush,lpop至關於堆的用法
> lpush books java python c
(integer) 3
> lpop books
"c"
> lpop books
"python"
> lpop books
"java"
----------------------------------
//返回列表key指定區間的元素,區間偏移量start跟stop指定
//start跟stop的下表都是以0爲底
> lrange books 0 2
1) "c"
2) "python"
3) "java"
----------------------------------
//ltrim能夠做爲一個定長的list，每次均可以獲取到最新的2條數據
> lpush books java python c c++
(integer) 4
> ltrim books 0 1
OK
> lrange books 0 -1
1) "c++"
2) "c"
----------------------------------
//當給定列表內沒有任何元素可供彈出的時候，鏈接將被blpop ,brpop命令阻塞，直到等待超時或發現可彈出元素爲止。
//設置超時 1秒
> BLPOP books 1
1) "books"
2) "c++"
> BLPOP books 1
1) "books"
2) "c"
> BLPOP books 1
(nil)
(1.05s)
----------------------------------

至此,redis list的用法先告一段落.

---

源碼解析

本文開頭的時候講list實現爲quicklist,quicklist是一個ziplist的雙向鏈表,那麼其內部結構是怎樣的呢？

/* quicklist is a 40 byte struct (on 64-bit systems) describing a quicklist.
 * 'count' is the number of total entries.
 * 'len' is the number of quicklist nodes.
 * 'compress' is: -1 if compression disabled, otherwise it's the number
 *                of quicklistNodes to leave uncompressed at ends of quicklist.
 * 'fill' is the user-requested (or default) fill factor. */
typedef struct quicklist {
    quicklistNode *head;
    quicklistNode *tail;
    unsigned long count;        /* total count of all entries in all ziplists */
    unsigned long len;          /* number of quicklistNodes */
    int fill : 16;              /* fill factor for individual nodes */
    unsigned int compress : 16; /* depth of end nodes not to compress;0=off */
} quicklist;

/* quicklistNode is a 32 byte struct describing a ziplist for a quicklist.
 * We use bit fields keep the quicklistNode at 32 bytes.
 * count: 16 bits, max 65536 (max zl bytes is 65k, so max count actually < 32k).
 * encoding: 2 bits, RAW=1, LZF=2.
 * container: 2 bits, NONE=1, ZIPLIST=2.
 * recompress: 1 bit, bool, true if node is temporarry decompressed for usage.
 * attempted_compress: 1 bit, boolean, used for verifying during testing.
 * extra: 10 bits, free for future use; pads out the remainder of 32 bits */
typedef struct quicklistNode {
    struct quicklistNode *prev;
    struct quicklistNode *next;
    unsigned char *zl;
    unsigned int sz;             /* ziplist size in bytes */
    unsigned int count : 16;     /* count of items in ziplist */
    unsigned int encoding : 2;   /* RAW==1 or LZF==2 */
    unsigned int container : 2;  /* NONE==1 or ZIPLIST==2 */
    unsigned int recompress : 1; /* was this node previous compressed? */
    unsigned int attempted_compress : 1; /* node can't compress; too small */
    unsigned int extra : 10; /* more bits to steal for future usage */
} quicklistNode;
/* quicklistLZF is a 4+N byte struct holding 'sz' followed by 'compressed'.
 * 'sz' is byte length of 'compressed' field.
 * 'compressed' is LZF data with total (compressed) length 'sz'
 * NOTE: uncompressed length is stored in quicklistNode->sz.
 * When quicklistNode->zl is compressed, node->zl points to a quicklistLZF */
typedef struct quicklistLZF {
    unsigned int sz; /* LZF size in bytes*/
    char compressed[];
} quicklistLZF;

從上面咱們能夠知道,quicklist是一個的雙向鏈表,因此當咱們使用lpush,rpop等操做是O(1)了。

ziplist自己也是一個可以維持數據前後順序的列表(按照插入位置),並且是一個內存緊湊的列表。 當咱們要表示list擁有12個數據項,這時候就會有可能有多種選擇了,例如3個節點的quicklist，每一個節點ziplist又包含4個數據項.或者2個節點的quicklist,每一個節點ziplist又包含6個數據項 那麼redis是如何選擇的呢？咱們能夠再redis.conf找到蛛絲馬跡~

# Lists are also encoded in a special way to save a lot of space.
# The number of entries allowed per internal list node can be specified
# as a fixed maximum size or a maximum number of elements.
# For a fixed maximum size, use -5 through -1, meaning:
# -5: max size: 64 Kb  <-- not recommended for normal workloads
# -4: max size: 32 Kb  <-- not recommended
# -3: max size: 16 Kb  <-- probably not recommended
# -2: max size: 8 Kb   <-- good
# -1: max size: 4 Kb   <-- good
# Positive numbers mean store up to _exactly_ that number of elements
# per list node.
# The highest performing option is usually -2 (8 Kb size) or -1 (4 Kb size),
# but if your use case is unique, adjust the settings as necessary.
list-max-ziplist-size -2

# Lists may also be compressed.
# Compress depth is the number of quicklist ziplist nodes from *each* side of
# the list to *exclude* from compression.  The head and tail of the list
# are always uncompressed for fast push/pop operations.  Settings are:
# 0: disable all list compression
# 1: depth 1 means "don't start compressing until after 1 node into the list,
#    going from either the head or tail"
#    So: [head]->node->node->...->node->[tail]
#    [head], [tail] will always be uncompressed; inner nodes will compress.
# 2: [head]->[next]->node->node->...->node->[prev]->[tail]
#    2 here means: don't compress head or head->next or tail->prev or tail,
#    but compress all nodes between them.
# 3: [head]->[next]->[next]->node->node->...->node->[prev]->[prev]->[tail]
# etc.
list-compress-depth 0

list-max-ziplist-size

當設置爲正數意味着最多隻能儲存該數量的元素，redis的做者建議設置爲-1 or -2,設置每一個quicklist節點上的ziplist能儲存元素的大小~ 當列表很長的時候,中間的數據被訪問的頻率就有可能很低,那麼在這種狀況下,list提供了一個參數可以將中間的數據壓縮~

list-compress-depth 0

這個參數表示quicklist兩端不被壓縮的節點數.head節點跟tail節點老是不壓縮的，方便在list的兩端進行快速存取

• 0: 是個特殊值，表示都不壓縮。這是Redis的默認值。
• 1: 表示quicklist兩端各有1個節點不壓縮，中間的節點壓縮。
• 2: 表示quicklist兩端各有2個節點不壓縮，中間的節點壓縮。
• 3: 表示quicklist兩端各有3個節點不壓縮，中間的節點壓縮。
• 依此類推…

quickList結構圖以下圖所示：

圖中對應的ziplist的配置大小和節點壓縮深度配置以下:

list-max-ziplist-size 3
list-compress-depth 1

在這裏例子中咱們能夠看到，quickList兩端各有一個節點沒有被壓縮,它們的數據指針指向真正的ziplist(即zl的指向).中間的其餘節點是被壓縮過的，它們的數據指針指向quicklistLZF

應用場景

1.消息隊列(無ack機制)

//生產者使用lpush將消息放入list中，消費者就能夠經過rpop取出該消息,而且能夠保證消息的有序性。
>lpush message "ces"
(integer) 1
>rpop message
"ces"

2.時間軸

//一種場景就是當用戶發送一條微博,經過lpush將它存放到list中,而後經過lrange就能夠取出最近的最新的微博信息了
> lpush weibo "xiaoxi1"
(integer) 1
> lpush weibo "xiaoxi2"
(integer) 2
> lpush weibo "xiaoxi3"
(integer) 3
> lrange weibo 0 9
1) "xiaoxi3"
2) "xiaoxi2"
3) "xiaoxi1"