技術與redis之數據淘汰

前言

數據淘汰，是一個友好的功能，不敢說優秀的功能。帶來一些好處，也帶來一些頭疼的問題。 某天一個同事說：redis的數據總是丟失，不能用。 去環境中看下，發現使用數據淘汰，只是在內存不足的狀況下，數據被淘汰。成了他的數據丟失。 在高峯狀況下，使用 lru策略，常常發生數據淘汰的狀況，大程度的下降併發量。

淘汰的數據集合

redis默認有16個db，每一個db有一個dict和一個expires的屬性

typedef struct redisDb {
    dict *dict;                 /* The keyspace for this DB */
    dict *expires;              /* Timeout of keys with a timeout set */
    dict *blocking_keys;        /* Keys with clients waiting for data (BLPOP) */
    dict *ready_keys;           /* Blocked keys that received a PUSH */
    dict *watched_keys;         /* WATCHED keys for MULTI/EXEC CAS */
    struct evictionPoolEntry *eviction_pool;    /* Eviction pool of keys */
    int id;                     /* Database ID */
    long long avg_ttl;          /* Average TTL, just for stats */
} redisDb;

dict與expire都是一個名爲 dict 結構體，實現了一個hash鏈表的數據結構。 不一樣的是 dict（參數別名爲 allkeys）保存的是真實數據。 expire （參數別名爲 volatile） 的數據只有執行一下命令纔會保存進入：

EXPIRE
EXPIREAT
PERSIST
PEXPIRE
PEXPIREAT
SET key value [EX seconds] [PX milliseconds] [NX|XX]

淘汰方式

1. LRU（最小使用次數）
2. 隨機參數
3. TTL(淘汰最快過時的數據)

依據淘汰數據與淘汰方式得到一下六種策略

1. volatile-lru：依據lru方式選中 expire集合中數據，而後在依據key刪除 dict集合數據
2. volatile-ttl：依據ttl方式選中 expire集合中數據，而後在依據key刪除 dict集合數據
3. volatile-random：依據隨機方式選中 expire集合中數據，而後在依據key刪除 dict集合數據
4. allkeys-lru：依據lru方式淘汰 dict集合數據
5. allkeys-random：依據l隨機方式淘汰 dict集合數據
6. no-enviction：不淘汰

redis數據淘汰相關參數

1. maxmemory

默認爲 0 ， 執行redis使用緩存大小 若是爲 0，那麼不會淘汰數據。

2. maxmemory_policy

淘汰策略 請看 依據淘汰數據與淘汰方式得到一下六種策略

3. maxmemory_samples

樣本次數。對於 lru與ttl策略來講，這個參數相當重要。

淘汰時機，條件，結束條件以及計算方式

1. 計算方式是 當前數據申請內存-全部slaves緩存-aof_buf緩存-aof_rewrite_buf_blocks緩存

slaves 緩存與 aof緩存請看其餘博客。

c文件爲 server.c 方法：freeMemoryIfNeeded

size_t mem_used, mem_tofree, mem_freed;
    int slaves = listLength(server.slaves);
    mstime_t latency, eviction_latency;

    /* Remove the size of slaves output buffers and AOF buffer from the
     * count of used memory. */
	 // 當前分配的使用內存
    mem_used = zmalloc_used_memory();
    if (slaves) {//識別是否有slave
        listIter li;
        listNode *ln;

        listRewind(server.slaves,&li);//得到全部slave
        while((ln = listNext(&li))) {//迭代savel
            client *slave = listNodeValue(ln);
            unsigned long obuf_bytes = getClientOutputBufferMemoryUsage(slave);
            if (obuf_bytes > mem_used)
                mem_used = 0;
            else
                mem_used -= obuf_bytes;
        }
    }
    if (server.aof_state != AOF_OFF) {
        mem_used -= sdslen(server.aof_buf);
        mem_used -= aofRewriteBufferSize();
    }

    /* Check if we are over the memory limit. */
    if (mem_used <= server.maxmemory) return C_OK;

2. 條件

上面公式獲得的內存大小是否大於 參數 maxmemory

if (mem_used <= server.maxmemory) return C_OK;

淘汰策略 不是 no-enviction

if (server.maxmemory_policy == MAXMEMORY_NO_EVICTION)
        return C_ERR; /* We need to free memory, but policy forbids. */

3. 結束條件

須要關注方法裏面的 mem_tofree 變量與mem_freed變量。mem_freed 大於 mem_tofree

while (mem_freed < mem_tofree) {
  .............
}

mem_tofree 變量 關於meme_used 請看計算公式

mem_tofree = mem_used - server.maxmemory;

mem_freed 變量。累加每次刪除數據內存改變大小

delta = (long long) zmalloc_used_memory();
..... 執行刪除數據操做
delta -= (long long) zmalloc_used_memory();
mem_freed += delta;

4. 觸發時機
   1. 執行configSet命令，修改maxmemory值的時候

config_set_memory_field("maxmemory",server.maxmemory) {
        if (server.maxmemory) {
            if (server.maxmemory < zmalloc_used_memory()) {
                serverLog(LL_WARNING,"WARNING: the new maxmemory value set via CONFIG SET is smaller than the current memory usage. This will result in keys eviction and/or inability to accept new write commands depending on the maxmemory-policy.");
            }
            freeMemoryIfNeeded();
        }
    }

2. 執行lua腳本中的call與pcall命令的時候(luaRedisGenericCommand)。若是大量使用lua腳本那麼這個時機須要關注

luaRedisCallCommand,luaRedisPCallCommand
if (server.maxmemory && server.lua_write_dirty == 0 &&
        (cmd->flags & CMD_DENYOOM))
    {
        if (freeMemoryIfNeeded() == C_ERR) {
            luaPushError(lua, shared.oomerr->ptr);
            goto cleanup;
        }
    }

3. 每次執行命令以前

代碼流程走向是 epoll 讀事件方法 readQueryFromClient --> processInputBuffer--> processCommand-->

if (server.maxmemory) {
        int retval = freeMemoryIfNeeded();
        /* freeMemoryIfNeeded may flush slave output buffers. This may result
         * into a slave, that may be the active client, to be freed. */
        if (server.current_client == NULL) return C_ERR;

        /* It was impossible to free enough memory, and the command the client
         * is trying to execute is denied during OOM conditions? Error. */
        if ((c->cmd->flags & CMD_DENYOOM) && retval == C_ERR) {
            flagTransaction(c);
            addReply(c, shared.oomerr);
            return C_OK;
        }
    }

分析每一個方式的實現細節

random策略

隨機的實現很是簡單，就是dict中，隨機抽取一個key，而後刪除 若是隨機參數的數據是熱點數據，那麼十分影響緩衝命中率，沒有命中得會去db去查，在高併發狀況下十分影響性能 使用場景：熱點量小，資源充足，併發低 若是內存中存在重要數據，那麼也不適合使用隨機算法

if (server.maxmemory_policy == MAXMEMORY_ALLKEYS_RANDOM ||
                server.maxmemory_policy == MAXMEMORY_VOLATILE_RANDOM)
            {
                de = dictGetRandomKey(dict);
                bestkey = dictGetKey(de);
            }

在說 LRU，TTL以前，先說一個重點。 samples 中文爲 樣品。做用爲：從dict裏面得到samples個數的樣本。而後從裏面計算除一個最合適的數據。若是隨機淘汰只有一個動做，那麼TTL有samples個隨機淘汰動做。

TTL策略

循環maxmemory_samples次 從expire 中得到數據，選擇其中val值最小的（null 小於任何數）

for (k = 0; k < server.maxmemory_samples; k++) {
                    sds thiskey;
                    long thisval;

                    de = dictGetRandomKey(dict);
                    thiskey = dictGetKey(de);
                    thisval = (long) dictGetVal(de);

                    /* Expire sooner (minor expire unix timestamp) is better
                     * candidate for deletion */
                    if (bestkey == NULL || thisval < bestval) {
                        bestkey = thiskey;
                        bestval = thisval;
                    }
                }

1. LRU策略

LRU是最複雜的策略，redis的算法並非真正的lru，而是隨機maxmemory_samples個數據進行略複雜的識別

else if (server.maxmemory_policy == MAXMEMORY_ALLKEYS_LRU ||
                server.maxmemory_policy == MAXMEMORY_VOLATILE_LRU)
            {
                struct evictionPoolEntry *pool = db->eviction_pool;

                while(bestkey == NULL) {
                    evictionPoolPopulate(dict, db->dict, db->eviction_pool);
                    /* Go backward from best to worst element to evict. */
                    for (k = MAXMEMORY_EVICTION_POOL_SIZE-1; k >= 0; k--) {
                        if (pool[k].key == NULL) continue;
                        de = dictFind(dict,pool[k].key);

                        /* Remove the entry from the pool. */
                        sdsfree(pool[k].key);
                        /* Shift all elements on its right to left. */
                        memmove(pool+k,pool+k+1,
                            sizeof(pool[0])*(MAXMEMORY_EVICTION_POOL_SIZE-k-1));
                        /* Clear the element on the right which is empty
                         * since we shifted one position to the left.  */
                        pool[MAXMEMORY_EVICTION_POOL_SIZE-1].key = NULL;
                        pool[MAXMEMORY_EVICTION_POOL_SIZE-1].idle = 0;

                        /* If the key exists, is our pick. Otherwise it is
                         * a ghost and we need to try the next element. */
                        if (de) {
                            bestkey = dictGetKey(de);
                            break;
                        } else {
                            /* Ghost... */
                            continue;
                        }
                    }
                }
            }

evictionPoolPopulate重要是調用dictGetSomeKeys方法得到maxmemory_samples（ >= 16 ）數量的數據。而後與 pool 中的數據進行對比，得到最少使用的數據。

#define EVICTION_SAMPLES_ARRAY_SIZE 16
void evictionPoolPopulate(dict *sampledict, dict *keydict, struct evictionPoolEntry *pool) {
    int j, k, count;
    dictEntry *_samples[EVICTION_SAMPLES_ARRAY_SIZE];
    dictEntry **samples;

    /* Try to use a static buffer: this function is a big hit...
     * Note: it was actually measured that this helps. */
    if (server.maxmemory_samples <= EVICTION_SAMPLES_ARRAY_SIZE) {
        samples = _samples;
    } else {
        samples = zmalloc(sizeof(samples[0])*server.maxmemory_samples);
    }

    count = dictGetSomeKeys(sampledict,samples,server.maxmemory_samples);
    for (j = 0; j < count; j++) {
        unsigned long long idle;
        sds key;
        robj *o;
        dictEntry *de;

        de = samples[j];
        key = dictGetKey(de);
        /* If the dictionary we are sampling from is not the main
         * dictionary (but the expires one) we need to lookup the key
         * again in the key dictionary to obtain the value object. */
        if (sampledict != keydict) de = dictFind(keydict, key);
        o = dictGetVal(de);
        idle = estimateObjectIdleTime(o);

        /* Insert the element inside the pool.
         * First, find the first empty bucket or the first populated
         * bucket that has an idle time smaller than our idle time. */
        k = 0;
        while (k < MAXMEMORY_EVICTION_POOL_SIZE &&
               pool[k].key &&
               pool[k].idle < idle) k++;
        if (k == 0 && pool[MAXMEMORY_EVICTION_POOL_SIZE-1].key != NULL) {
            /* Can't insert if the element is < the worst element we have
             * and there are no empty buckets. */
            continue;
        } else if (k < MAXMEMORY_EVICTION_POOL_SIZE && pool[k].key == NULL) {
            /* Inserting into empty position. No setup needed before insert. */
        } else {
            /* Inserting in the middle. Now k points to the first element
             * greater than the element to insert.  */
            if (pool[MAXMEMORY_EVICTION_POOL_SIZE-1].key == NULL) {
                /* Free space on the right? Insert at k shifting
                 * all the elements from k to end to the right. */
                memmove(pool+k+1,pool+k,
                    sizeof(pool[0])*(MAXMEMORY_EVICTION_POOL_SIZE-k-1));
            } else {
                /* No free space on right? Insert at k-1 */
                k--;
                /* Shift all elements on the left of k (included) to the
                 * left, so we discard the element with smaller idle time. */
                sdsfree(pool[0].key);
                memmove(pool,pool+1,sizeof(pool[0])*k);
            }
        }
        pool[k].key = sdsdup(key);
        pool[k].idle = idle;
    }
    if (samples != _samples) zfree(samples);
}

當數據淘汰以後，redis會作什麼處理

1. 會觸發 NOTIFY_EVICTED 事件

notifyKeyspaceEvent(NOTIFY_EVICTED, "evicted",
  keyobj, db->id);

2. 強制執行一個數據同步到slave

if (slaves) flushSlavesOutputBuffers();

後記

策略性能排序： LRU > TTL > RANDOM > no-enviction 冷數據淘汰正確性： RANDMON > TTL > LRU 不是全部業務都適合數據淘汰。好比須要分佈式鎖的業務，數據關聯業務（用lru處理邏輯） 在資源有限狀況下。能夠對簡單業務數據進行淘汰。好比用戶數據，等等 注意 LRU 策略在併發的狀況下，性能下降的狀況