記系統斷電致使文件中出現null符號(^@)問題的解決

咱們的前端數據採集系統是用的樹莓派，上面跑着基於Debian的Linux操做系統，用的是EXT4的文件系統格式。以前發現採集到的日誌文件末尾偶爾會出現大量的^@，通過一番調研後發現是因爲系統忽然斷電致使的。而後再通過一番google，說是因爲文件的metadata修改後變大，可是實際數據因爲斷電沒有寫入硬盤，因此會用null符號（^@）來填充。而後想了不少辦法來修復這種斷電後產生的異常數據（以前由於解決操做系統卡死的問題給咱們的盒子加了自動重啓線，每隔3天就會自動重啓），一直都沒有很好的解決，直到聽極客時間的Linux操做講到文件系統的journal mode，再通過一番查找，終於理解了這個問題原理，也找到了解決辦法。

EXT4是一種journaling文件系統，使用了相似於MySQL中Transaction的概念來保證一致性，把對文件的操做看作是一次事務，而且也使用了先寫redo log的機制。分爲如下三種模式（能夠用dmesg | grep EXT4查看：mounted filesystem with ordered data mode. Opts: (null)）

• journal

將數據及元數據都先寫入日誌文件並落盤，commit日誌文件，而後再將數據和元數據寫入硬盤正確的位置(In-place write)，異步刪除已經寫入硬盤正確位置的日誌文件數據。若是發生系統crash，能夠replay日誌文件中已經commit的數據進行恢復。這種模式最安全但性能較差（除了數據須要同時被寫入和讀取），由於數據被寫入了兩次。流程以下：

• ordered（默認）

只有元數據寫入日誌文件，但保證先將數據寫入硬盤，再commit日誌文件，流程圖以下（第1,2步能夠同時進行，只要保證在第3步前完成就行）：

若是系統crash，能夠經過replay日誌文件恢復元數據，保證文件系統一致性，且若是存儲的數據是複用已有的文件block來存數據，能夠保證數據已經落盤，但因爲存在Delayed allocation機制，若是文件須要分配新的block來寫入，則系統會推遲將近1分鐘才真正刷入硬盤，會致使數據丟失，這也是爲何會看到文件出現^@來填補真實的數據。

"Delayed allocation" means that the filesystem tries to delay the allocation of physical disk blocks for written data for as long as possible. This policy brings some important performance benefits. Many files are short-lived; delayed allocation can keep the system from writing fleeting temporary files to disk at all. And, for longer-lived files, delayed allocation allows the kernel to accumulate more data and to allocate the blocks for data contiguously, speeding up both the write and any subsequent reads of that data. It's an important optimization which is found in most contemporary filesystems.
But, if blocks have not been allocated for a file, there is no need to write them quickly as a security measure. Since the blocks do not yet exist, it is not possible to read somebody else's data from them. So ext4 will not (cannot) write out unallocated blocks as part of the next journal commit cycle. Those blocks will, instead, wait until the kernel decides to flush them out; at that point, physical blocks will be allocated on disk and the data will be made persistent. The kernel doesn't like to let file data sit unwritten for too long, but it can still take a minute or so (with the default settings) for that data to be flushed - far longer than the five seconds normally seen with ext3. And that is why a crash can cause the loss of quite a bit more data when ext4 is being used.

• writeback

只有元數據寫入日誌文件，但不保證數據寫入硬盤何時發生。有可能系統重啓後元數據被恢復，但數據由於並無寫入硬盤而丟失。流程圖以下：

最後將文件系統在mount時設爲journal模式，用性能來換取強一致性。修改/etc/fstab：

/dev/mapper/sd03 /pi/ ext4 defaults,data=journal 0 0

ext4 and data loss