Redis持久化方案

Redis能夠實現數據的持久化存儲,即將數據保存到磁盤上。 
Redis的持久化存儲提供兩種方式:RDB與AOF。RDB是默認配置。AOF須要手動開啓。 redis

 

默認redis是會以快照的形式將數據持久化到磁盤(一個二進制文件,dump.rdb,這個文件名字能夠指定),在配置文件中的格式是:save N M表示在N秒以內,redis至少發生M次修改則redis抓快照到磁盤。固然咱們也能夠手動執行save或者bgsave(異步)作快照。當redis須要作持久化時,redis會fork一個子進程;子進程將數據寫到磁盤上一個臨時RDB文件中;當子進程完成寫臨時文件後,將原來的RDB替換掉,這樣的好處就是能夠copy-on-write.promise

# Save the DB on disk:
#
#   save <seconds> <changes>
#
#   Will save the DB if both the given number of seconds and the given
#   number of write operations against the DB occurred.
#
#   In the example below the behaviour will be to save:
#   after 900 sec (15 min) if at least 1 key changed
#   after 300 sec (5 min) if at least 10 keys changed
#   after 60 sec if at least 10000 keys changed
#
#   Note: you can disable saving completely by commenting out all "save" lines.
#
#   It is also possible to remove all the previously configured save
#   points by adding a save directive with a single empty string argument
#   like in the following example:
#
#   save ""

save 900 1
save 300 10
save 60 10000

 

還有一種持久化方法是Append-only:filesnapshotting方法在redis異常宕機時,最近的數據會丟失(丟失數據的多少視你save策略的配置),因此這是它最大的缺點,當數據量越大,丟失的數據就越多(能夠看下文官方說明)。若是要開啓AOF模式,修改Redis的配置文件redis.conf。安全

appendonly yes  
appendfilename "appendonly.aof" 

# appendfsync always
appendfsync everysec  
# appendfsync no

no-appendfsync-on-rewrite no    

auto-aof-rewrite-percentage 100
auto-aof-rewrite-min-size 64mb  

aof-load-truncated yes  

 

官方說明:服務器

By default Redis asynchronously dumps the dataset on disk. This mode is good enough in many applications, but an issue with the Redis process or a power outage may result into a few minutes of writes lost (depending on the configured save points).架構

The Append Only File is an alternative persistence mode that provides much better durability. For instance using the default data fsync policy (see later in the config file) Redis can lose just one second of writes in a dramatic event like a server power outage, or a single write if something wrong with the Redis process itself happens, but the operating system is still running correctly.app

AOF and RDB persistence can be enabled at the same time without problems. If the AOF is enabled on startup Redis will load the AOF, that is the file with the better durability guarantees.異步

Redis默認採用異步的方式將數據存放到磁盤上,這個模式對大部份應用來講是足夠好的,可是在Redis進程或電源發生故障的狀況下,可能會形成小部份的數據丟失,這取決於配置的保存時間點。 
Appendonly是一種可以提供很是好的持久化的模式,例如使用默認的Fsync方案,Redis能在發生服務器電源故障或操做系統仍然正常運行但Redis進程莫名掛掉的狀況下,只丟失1秒的數據。 
AOF與RDB模式能夠同時啓用,這並不衝突。若是AOF是可用的,那Redis啓動時將自動加載AOF,這個文件可以提供更好的持久性保障。async

 

The fsync() call tells the Operating System to actually write data on disk instead of waiting for more data in the output buffer. Some OS will really flush data on disk, some other OS will just try to do it ASAP.ide

Redis supports three different modes:性能

no: don’t fsync, just let the OS flush the data when it wants. Faster. 
always: fsync after every write to the append only log. Slow, Safest. 
everysec: fsync only one time every second. Compromise.

The default is 「everysec」, as that’s usually the right compromise between speed and data safety. It’s up to you to understand if you can relax this to 「no」 that will let the operating system flush the output buffer when it wants, for better performances (but if you can live with the idea of some data loss consider the default persistence mode that’s snapshotting), or on the contrary, use 「always」 that’s very slow but a bit safer than everysec.

If unsure, use 「everysec」.

fsync()調用告訴操做系統將數據真實的寫入磁盤而不是放到緩衝區中,一些操做系統會真實的執行請求,還有一些操做系統只會盡力的嘗試。

Redis支持3種不一樣的模式: 
no:不即時同步,由操做系統控制什麼時候刷寫到磁盤上,這種模式速度最快; 
always:每次只寫日誌,速度較慢,但最安全; 
everysec:每秒鐘同步一次,折中的方案。

默認的模式是「everysec」,它一般是在速度和數據安全之間折中的方法。若是你能夠控制操做系統在Redis須要的時候去刷寫緩衝區那可使用「no」模式,可以提供更好的性能(但若是你能接受一些數據丟失,能夠考慮默認的持久化模式–快照方式),相反,使用「always」模式很慢,可是它比「everysec」模式要安全一點。

若是不肯定,就使用「everysec」。

 

When the AOF fsync policy is set to always or everysec, and a background saving process (a background save or AOF log background rewriting) is performing a lot of I/O against the disk, in some Linux configurations Redis may block too long on the fsync() call. Note that there is no fix for this currently, as even performing fsync in a different thread will block our synchronous write(2) call.

In order to mitigate this problem it’s possible to use the following option that will prevent fsync() from being called in the main process while a BGSAVE or BGREWRITEAOF is in progress.

This means that while another child is saving, the durability of Redis is the same as 「appendfsync none」. In practical terms, this means that it is possible to lose up to 30 seconds of log in the worst scenario (with the default Linux settings).

If you have latency problems turn this to 「yes」. Otherwise leave it as 「no」 that is the safest pick from the point of view of durability.

當使用AOF的fsync方案設置爲「always」或「everysec」時,後臺的存儲進程會執行大量的磁盤I/O操做,在一些Linux架構中,Redis在fsync()調用時可能會阻塞好久。這個問題當前並無修復,即便是在一個不一樣的線程執行fsync也將會阻塞咱們的同步寫調用。

爲了緩解這個問題,可使用如下選項,它將會在有一個BGSAVE或BGREWRITEAOF正在運行時,阻止主進程調用fsync()。

這意味着有另外一個子進程在存儲時,Redis的持久性等同於「appendfsync none」。在實踐中,意味着在最壞的狀況下它可能丟失多達30秒的日誌(默認的Linux設置)。

若是你有潛在的問題須要更改它爲「yes」。不然從持久性的觀點來看「no」是最安全的選擇。

 

Automatic rewrite of the append only file. 
Redis is able to automatically rewrite the log file implicitly calling BGREWRITEAOF when the AOF log size grows by the specified percentage.

This is how it works: Redis remembers the size of the AOF file after the latest rewrite (if no rewrite has happened since the restart, the size of the AOF at startup is used).

This base size is compared to the current size. If the current size is bigger than the specified percentage, the rewrite is triggered. Also you need to specify a minimal size for the AOF file to be rewritten, this is useful to avoid rewriting the AOF file even if the percentage increase is reached but it is still pretty small.

Specify a percentage of zero in order to disable the automatic AOF rewrite feature. 

自動重寫append only文件。 
當AOF日誌的大小根據指定的百分比增加時,Redis會暗中調用BGREWRITEAOF去自動重寫日誌文件。

工做原理:Redis記憶AOF文件最後一次重寫的大小(若是重啓後沒有重寫發生,AOF的大小在啓動時會被使用)。

基本大小對比當前大小。若是當前大小比指定的百分比大,觸發重寫。而且你要爲AOF文件指定一個最小的尺寸去重寫,這對於避免重寫AOF文件是有用的,即便達到了百分比增加率但它仍然是很是小的。

指定百分比爲0以便禁用自動AOF重寫。

 

An AOF file may be found to be truncated at the end during the Redis startup process, when the AOF data gets loaded back into memory. 
This may happen when the system where Redis is running crashes, especially when an ext4 filesystem is mounted without the data=ordered option (however this can’t happen when Redis itself crashes or aborts but the operating system still works correctly).

Redis can either exit with an error when this happens, or load as much data as possible (the default now) and start if the AOF file is found to be truncated at the end. The following option controls this behavior.

If aof-load-truncated is set to yes, a truncated AOF file is loaded and the Redis server starts emitting a log to inform the user of the event. Otherwise if the option is set to no, the server aborts with an error and refuses to start. When the option is set to no, the user requires to fix the AOF file using the 「redis-check-aof」 utility before to restart the server.

Note that if the AOF file will be found to be corrupted in the middle the server will still exit with an error. This option only applies when Redis will try to read more data from the AOF file but not enough bytes will be found.

可能發現一個AOF文件在Redis啓動過程當中被截斷,當AOF數據被加載回內存時。 
這可能發生在系統中Redis運行崩潰,尤爲是掛載一個EXT4文件系統而沒有使用「data=ordered」選項時(可是這不會發生在Redis自身崩潰或中斷而操做系統仍然正常運行的時候)。

當有一個錯誤發生時Redis要麼退出,要麼加載儘量多的數據(當前默認)啓動,若是AOF文件最後發現被截斷。如下選項控制這個行爲。

若是aof-load-truncated被設置爲「yes」,一個被截斷的AOF文件將會被加載,Redis服務啓動發出一個日誌告知用戶這個事件。若是這個選項設置爲「no」,服務器停止一個錯誤並拒絕啓動。當選項被設置爲「no」,用戶須要在重啓服務以前使用「redis-check-aof」功能肯定AOF文件。

須要注意的是若是AOF文件被發現是損壞的,那服務器仍然會以一個錯誤退出。這個選項僅適用於Redis但願讀取更多的數據可是沒有發現足夠的字節時。

 

 

http://zheng-ji.info/blog/2016/03/10/gai-xuan-ze-na-chong-redischi-jiu-hua-pei-zhi/

https://my.oschina.net/davehe/blog/174662

相關文章
相關標籤/搜索