kafka Disks and Filesystem（磁盤和文件系統）

轉載請註明來源地址：http://www.cnblogs.com/dongxiao-yang/p/5206631.html

 

We recommend using multiple drives to get good throughput and not sharing the same drives used for Kafka data with application logs or other OS filesystem activity to ensure good latency. You can either RAID these drives together into a single volume or format and mount each drive as its own directory. Since Kafka has replication the redundancy provided by RAID can also be provided at the application level. This choice has several tradeoffs.

咱們推薦服務器使用多塊硬盤：(1)實現高吞吐 (2)隔離kafka數據文件與應用的日誌文件以及其餘系統相關的磁盤消耗以保證低延遲。多塊硬盤能夠raid成一個卷或者每塊硬盤單獨顯示一個盤符掛載。因爲kakfa在應用級別已經能夠提供raid所提供的數據冗餘備份的功能，能夠經過幾個方面權衡選擇的策略。

 

If you configure multiple data directories partitions will be assigned round-robin to data directories. Each partition will be entirely in one of the data directories. If data is not well balanced among partitions this can lead to load imbalance between disks.

若是配置爲多塊硬盤，分區將會輪詢分佈到硬盤文件下，每一個分區將會徹底落到一塊單獨磁盤上。若是數據裏的分區並非均勻分佈的話會可能致使磁盤之間的負載不均衡。

 

RAID can potentially do better at balancing load between disks (although it doesn't always seem to) because it balances load at a lower level. The primary downside of RAID is that it is usually a big performance hit for write throughput and reduces the available disk space.

raid先天性的在硬盤間數據均衡上表現的更好（雖然並不老是如此），由於raid是在更底層的層面實現的數據均衡。但其主要缺點是raid一般在寫吞吐上會有很高的消耗，而且會減小可用的磁盤空間。

 

Another potential benefit of RAID is the ability to tolerate disk failures. However our experience has been that rebuilding the RAID array is so I/O intensive that it effectively disables the server, so this does not provide much real availability improvement.

raid 的另外一個潛在的好處是可以容忍磁盤故障。然而，咱們的經驗是，重建raid隊列的動做是一個過於io密集的工做，顯著地使服務器工做失能，所以這不提供不少實際的可用性改進。