OCF介紹

轉譯自 https://open-cas.github.io/cache.html

https://open-cas.github.io/doxygen/ocf/

https://www.sdnlab.com/23341.html

 

相關概念

1.cache

In general a cache (1) is a component that mediates data exchange between an application and a backend storage (2), selectively storing most accessed data on a relatively smaller and faster cache storage (3). Every time the cached data is accessed by the application, an I/O operation is performed on the cache storage, decreasing data access time and improving performance of the whole system.

 

cache line 是cache處理應用數據的基本單位。 每一個cache line 和一個core line對應，可是在必定時間中不是每一個core line有對應的cache line。 若是有，那麼這個core line被mapped 到cache中，不然是unmapped。

當一個應用執行IO操做時，cache檢查被訪問的core line是否已經映射到cache的cache line了。若是沒有，那就是cache miss，cache engine須要從後端的存儲設備訪問數據。而後根據cache mode 決定是否把core line 映射到cache line，若是映射的話，執行的是cache insert的操做。如果cache hit，那麼應用程序從cache獲取數據。

cache object

在OCF中，一個cache object提供了一個和cache 狀態機交互的接口。其API能夠attach cache storage, add and remove cores, modify various configuration parameters and get statistics. 每一個cache操做一個單一的cache storage（ cache volume), 其存儲數據和元數據。

Single cache can handle data from many cores. It’s called a multi-core configuration. The core object can become backend storage of another core or cache storage of some cache, which makes it possible to construct complex multilevel configurations.

2.cache operation

mapping

Mapping is an operation of linking up a core line with a cache line in the cache metadata. 在映射的過程當中，cache storage的一個cache line大小的區域被分配給core line。以後在cache操做中，這塊區域能夠用來存儲core line數據，直到cache line被驅逐evicted了。

mapping 更新cache line metadata中的core id和core line number以及哈希表的信息。

insertion

Insertion is an operation of writing core line data to a cache line. During insertion the data of I/O request is being written to an area of cache storage storing data of the cache line corresponding to the core line being accessed. 元數據中cache line的valid和dirty bit被更新。

insertion不改變cache line mapping metadata，只是改變valid和 dirty bit。Because of that it can be done only after successfull mapping operation. However, unlike mapping, which operates on whole cache lines, insertion is performed with sector granularity. Every change of valid bit from 0 to 1 is considered an insertion.

Insertion may occur for both read and write requests. Additionally, in the Write-Back mode the insertion may introduce a dirty data. In such situation, besides setting a valid bit, it also sets a dirty bit for accessed sectors.

update

Update is an operation of overwriting cache line data in the cache storage. update發生在當寫請求的core line被mapped到cache line了且被訪問的sector是valid。若cache line中一些被寫的sectors是invalid的，那麼這些sector被inserted，這意味着update和insertion可能在一個io 請求中被執行。

Update can be performed only during a write request (a read can never change the data) and similarly to insert it can introduce dirty data, so it may update dirty bit for accessed sectors in the cache line. However it will never change valid bit, as update touches only sectors that are already valid.

invalidation

Invalidation is an operation of marking one or more cache line sectors as invalid.

It can be done for example during handling of discard request, during purge operation, or as result of some internal cache operations. More information about cases when invalidation is performed can be found here.

eviction

Eviction is an operation of removing cache line mappings. 當cache中沒有足夠的空間for mapping data of incoming I/O requests。 被驅逐的cache line是經過eviction policy algorithm選擇的。LRU（least recently used）選擇最長時間沒有被訪問的cache lines。

flushing

Flushing is an operation of synchronizing data in the cache storage and the backend storage. 讀cache storage中被標記爲dirty的sectors，把他們寫道後端存儲設備，而後設置cache line元數據中的dirty bit爲0。

flush是一個cache management操做，必須由用戶觸發。常常在detach core以前執行以防數據丟失。

cleaning

Cleaning is an operation very simillar to flushing, but it’s performed by the cache automatically as a background task. 是由cleaning policy控制的。目前OCF支持三種cleaning policy：

• Approximately Least Recently Used (ALRU),
• Aggressive Cleaning Policy (ACP),
• No Operation (NOP).

 

3.cache configuration

The cache mode parameter determines how incoming I/O requests are handled by the cache engine. Depending on the cache mode some requests may be inserted to the cache or not. The cache mode determines also if data stored on cache should always be coherent with data stored in the backend storage (if there is a possibility of dirty data).

• Write-Through (WT),
• Write-Back (WB),
• Write-Around (WA),
• Write-Invalidate (WI),
• Write-Only (WO).
• Pass-Through (PT).

（1）Write-Through（WT）:
I/O在寫到cache device的同時，也會直接write-through到後端的core device。WT模式保證了數據在core device和cache device老是同步的，所以不用擔憂忽然掉電形成的緩存數據丟失。由於每次數據依然要寫到慢速的後端設備，因此這個模式只可以加速數據讀取的過程.

（2）Write-Back （WB）：
I/O寫時，數據首先寫到cache device，而後就會告訴應用寫完畢。這些寫操做會週期性的寫回到core device。這裏CAS能夠選擇不一樣的cleaning policy。好比ALRU（Approximately Least Recently Used），ACP（Aggressive Cleaning Policy）或者 NOP（No Operation，即不自動寫回，而由用戶主動的作flush操做）。所以，WB模式可以既加速讀操做，又加速寫操做。可是也會有忽然掉電形成緩存數據沒有及時寫回到core device而致使數據丟失的狀況。

（3）Write-Around (WA):
講這個模式以前，先介紹一下緩存污染（cache pollution）。首先咱們要知道cache device和core device實際上是經過cache line和core line的mapping肯定對應關係的。緩存污染表示不少不經常使用的數據被緩存到了cache device中。那麼在後面新的寫I/O的狀況下，沒有空閒的cache line，沒有足夠的空間去作mapping，此時須要逐出（evict）某條對應的cache line mapping，這就形成了額外的開銷。

WA模式有點相似於WT模式，不一樣的是，寫操做只有在cache line有對應的core line的狀況下（即這部分數據已經被讀緩存過）會同時更新cache device和core device。其餘狀況，則直接寫到core device。所以WA模式也只能加速讀操做，而且保證了數據的一致性。並且還能避免緩存污染這樣的狀況。

（4）Write-Invalidate（WI）
在這個模式中，只有讀操做會被map到緩存中。對於寫操做，會直接寫入到core device，同時，若是有對應的cache line在緩存中，就讓這條cache line變爲無效。WI模式對於讀密集型I/O有更好的加速做用。而且可以減小緩存的evict操做。

（5）Write-only (WO)

寫到cache，讀的話從後端讀，不放到cache。In Write-Only mode, the cache engine writes the data exactly like in Write-Back mode so the data is written to cache storage without writing it to backend storage immediatelly. Read operations do not promote data to cache. Write-Only mode will accelerate only write intensive operations, as reads need to be performed only on the backend storage. There is a risk of data loss if the cache storage fails before the data is written to the backend storage.

（6）Pass-Through (PT)
很好理解，PT模式即全部IO直接繞過cache device直接和core device交互。

cache line size

The cache line size parameter determines the size of a data block on which the cache operates (cache line). It’s the minimum portion of data on the backend storage (core line) that can be mapped into the cache.

OCF allows to set cache line size to one of the following values:

• 4 KiB,
• 8 KiB,
• 16 KiB,
• 32 KiB,
• 64 KiB.

4.core

The core object is an abstraction that allows application access to backend storage cached by a cache. It provides API for submitting I/O requests, which are handled according to current cache mode and other cache and core configuration parameters. During normal cache operation the backend storage is exclusively owned by the cache object, and application should never access it directly unless the core is removed from the cache. That’s it, using the core API is the only proper way of accessing the data on the backend storage.

 

 5.volume

A volume is generic representation of a storage, that allows OCF to access different types of storages using common abstract interface. OCF uses a volume interface for accessing both backend storage and cache storage. Storage represented by volume may be any kind of storage that allows for random block access - it may be HDD or SSD drive, ramdisk, network storage or any other kind of non-volatile or volatile memory.

 

 6.cache line

A cache line is the smallest portion of data that can be mapped into a cache. Every mapped cache line is associated with a core line, which is a corresponding region on a backend storage. Both the cache storage and the backend storage are split into blocks of the size of a cache line, and all the cache mappings are aligned to these blocks. The relationship between a cache line and a core line is illustrated on the picture below.

cache line metadata

OCF maintains small portion of metadata per each cache line, which contains the following information:

• core id,
• core line number,
• valid and dirty bits for every sector.

The core id determines on which core is the core line corresponding to given cache line located, whereas the core line number determines precise core line location on the backend storage. Valid and dirty bits are explained below.

Valid and dirty bits

valid是說該cache line是否被mapped，dirty是說數據在cache和backend storage中是否一致。

Valid and dirty bits define current cache line state. When a cache line is valid it means, that it’s mapped to a core line determined by a core id and a core line number. Otherwise all the other cache line metadata information is invalid and should be ignored. When a cache line is in invalid state, it may be used to map core line accessed by I/O request, and then it becomes valid. There are few situations in which a cache line can return to invalid state:

• When the cache line is being evicted.
• When the core pointed to by the core id is being removed.
• When the core pointed to by the core id is being purged.
• When the entire cache is being purged.
• During discard operation being performed on the corresponding core line.
• During I/O request when a cache mode which may perform an invalidation is selected. Currently these cache modes are Write-Invalidate and Pass-Through.

The dirty bit determines if the cache line data stored in the cache is in sync with the corresponding data on the backend storage. If a cache line is dirty, then only data on the cache storage is up to date, and it will need to be flushed at some point in the future (after that it will be marked as clean by zeroing dirty bit). A cache line can become dirty only during write operation in Write-Back mode.

Valid and dirty bits are maintained per sector, which means that not every sector in a valid cache line has to be valid, as well as not every sector in a dirty cache line has to be dirty. An entire cache line is considered valid if at least one of its sectors is valid, and similarly it’s considered dirty if at least one of its sectors is dirty.至少一個sector是valid的，那麼該cache line是valid的，至少一個sector是dirty的，那麼該cache line是dirty的。