（9）ceph pool 性能測試

Ceph包含在rados bench RADOS存儲集羣上執行性能基準測試的命令。該命令將執行寫測試和兩種類型的讀測試。--no-cleanup在測試讀寫性能時，該選項很重要。默認狀況下，該rados bench命令將刪除已寫入存儲池的對象。留在這些對象後面容許兩個讀取測試來測量順序和隨機讀取性能。

（1）準備工做，清除全部的文件系統緩存
[root@node141 mnt]# echo 3 > /proc/sys/vm/drop_caches
[root@node141 mnt]# sync

（2）建立一個新的存儲池
[root@node141 mnt]# ceph osd pool create rbd 128 128

（3）測試一下rbd的寫性能，添加 --no-cleanup參數

[root@node141 mnt]# rados bench -p rbd 10 write  --no-cleanup 
hints = 1
Maintaining 16 concurrent writes of 4194304 bytes to objects of size 4194304 for up to 10 seconds or 0 objects
Object prefix: benchmark_data_node141_28363
  sec Cur ops   started  finished  avg MB/s  cur MB/s last lat(s)  avg lat(s)
    0       0         0         0         0         0           -           0
    1      16        81        65   259.999       260    0.107901    0.200157
    2      16       161       145   289.976       320    0.190659    0.215092
    3      16       252       236   314.632       364   0.0807696    0.193549
    4      16       351       335   334.958       396    0.191582    0.185652
    5      16       445       429   343.157       376    0.158618    0.178393
    6      16       548       532   354.621       412    0.162274    0.178178
    7      16       652       636   363.378       416    0.190742    0.173784
    8      16       753       737   368.448       404     0.15392    0.170866
    9      16       863       847   376.391       440    0.103038    0.169129
   10      16       964       948   379.144       404    0.138741    0.167306
Total time run:         10.1023
Total writes made:      965
Write size:             4194304
Object size:            4194304
Bandwidth (MB/sec):     382.09
Stddev Bandwidth:       53.3267
Max bandwidth (MB/sec): 440
Min bandwidth (MB/sec): 260
Average IOPS:           95
Stddev IOPS:            13.3317
Max IOPS:               110
Min IOPS:               65
Average Latency(s):     0.167422
Stddev Latency(s):      0.0765119
Max latency(s):         0.668499
Min latency(s):         0.0569634

（4）測試一下，10秒的順序讀取：

[root@node141 mnt]# rados bench -p rbd 10 seq
hints = 1
  sec Cur ops   started  finished  avg MB/s  cur MB/s last lat(s)  avg lat(s)
    0       0         0         0         0         0           -           0
    1      16       257       241   963.679       964   0.0607106   0.0623843
    2      16       549       533   1065.73      1168   0.0450988   0.0583824
    3      16       835       819   1091.76      1144    0.044447    0.057111
Total time run:       3.47417
Total reads made:     965
Read size:            4194304
Object size:          4194304
Bandwidth (MB/sec):   1111.06
Average IOPS:         277
Stddev IOPS:          27.8747
Max IOPS:             292
Min IOPS:             241
Average Latency(s):   0.056581
Max latency(s):       0.388605
Min latency(s):       0.015351

（5）測試10秒隨機讀寫性能

[root@node141 mnt]# rados bench -p rbd 10 rand
hints = 1
  sec Cur ops   started  finished  avg MB/s  cur MB/s last lat(s)  avg lat(s)
    0       0         0         0         0         0           -           0
    1      16       408       392   1566.56      1568   0.0715088    0.038812
    2      16       794       778   1554.08      1544   0.0219942   0.0396005
    3      16      1211      1195   1591.93      1668    0.125987   0.0389289
    4      15      1604      1589    1587.9      1576   0.0497586   0.0392412
    5      16      2013      1997   1596.67      1632  0.00512184   0.0388821
    6      16      2415      2399   1598.52      1608    0.146983   0.0389842
    7      16      2806      2790   1593.56      1564    0.110503   0.0391283
    8      16      3186      3170   1584.34      1520   0.0226104   0.0393014
    9      15      3598      3583   1591.83      1652   0.0117805   0.0392624
   10      16      3965      3949   1579.03      1464   0.0752829   0.0395739
Total time run:       10.0536
Total reads made:     3965
Read size:            4194304
Object size:          4194304
Bandwidth (MB/sec):   1577.54
Average IOPS:         394
Stddev IOPS:          15.6308
Max IOPS:             417
Min IOPS:             366
Average Latency(s):   0.0397174
Max latency(s):       0.353027
Min latency(s):       0.00414538

（6）要增長併發讀取和寫入的數量，請使用-t 默認爲16個現場的選項，-b參數能夠調整正在寫入的對象的大小，默認對象大小爲4MB。安全的最大對象爲16MB.
添加 --run-name <label> 選項以控制在基準測試期間寫入的對象的名稱。多個rados bench命令能夠同時經過改變跑出--run-name標籤每一個運行命令的實例。這能夠防止在多個客戶端嘗試訪問同一對象並容許不一樣客戶端訪問不一樣對象時可能發生的潛在I / O錯誤。--run-name在嘗試模擬真實世界的工做負載時，該選項也頗有用。例如：

[root@node141 mnt]# rados bench -p rbd 10 write  -t 4 --run-name zabbix71
hints = 1
Maintaining 4 concurrent writes of 4194304 bytes to objects of size 4194304 for up to 10 seconds or 0 objects
Object prefix: benchmark_data_node141_28450
  sec Cur ops   started  finished  avg MB/s  cur MB/s last lat(s)  avg lat(s)
    0       0         0         0         0         0           -           0
    1       4        40        36   143.992       144    0.065704    0.103059
    2       4        86        82   163.981       184   0.0763531   0.0955013
    3       4       126       122   162.641       160    0.106746   0.0953542
    4       4       173       169   168.972       188   0.0872597   0.0934176
    5       4       218       214   171.171       180    0.170851   0.0927783
    6       4       255       251   167.305       148   0.0943133    0.093864
    7       4       294       290   165.686       156   0.0976048   0.0959294
    8       4       340       336   167.971       184   0.0882485   0.0946923
    9       4       386       382   169.748       184   0.0703278   0.0933611
   10       4       430       426    170.37       176   0.0623777   0.0933391
Total time run:         10.1109
Total writes made:      431
Write size:             4194304
Object size:            4194304
Bandwidth (MB/sec):     170.51
Stddev Bandwidth:       16.6747
Max bandwidth (MB/sec): 188
Min bandwidth (MB/sec): 144
Average IOPS:           42
Stddev IOPS:            4.16867
Max IOPS:               47
Min IOPS:               36
Average Latency(s):     0.0935851
Stddev Latency(s):      0.0433054
Max latency(s):         0.579434
Min latency(s):         0.0533891
Cleaning up (deleting benchmark objects)
Removed 431 objects
Clean up completed and total clean up time :1.62203

(7) 刪除rados bench 命令建立的數據
[root@node141 mnt]# rados -p rbd cleanup
Removed 965 objects