FastDFS介紹和配置過程

因爲網站使用nfs共享方式保存用戶上傳的圖片，附件等資料，而後經過apache下載的方式供用戶訪問，在網站架構初期，使用這種簡單的方式實現了靜態資源的讀寫分離，但隨着網站數據量的增長，圖片服務器漸漸成爲整個網站的短板，緣次催生了使用fastfds的想法，故而先進行一番簡單的測試！在開始以前仍是先來看看fastfds的介紹信息：

FastDFS is an open source high performance distributed file system (DFS). It's major functions include: file storing, file syncing and file accessing, and design for high capacity and load balancing. FastDFS is an open source high performance distributed file system. It's major functions include: file storing, file syncing and file accessing (file uploading and file downloading), and it can resolve the high capacity and load balancing problem. FastDFS should meet the requirement of the website whose service based on files such as photo sharing site and vidio sharing site. FastDFS has two roles: tracker and storage. The tracker takes charge of scheduling and load balancing for file access. The storage store files and it's function is file management including: file storing, file syncing, providing file access interface. It also manage the meta data which are attributes representing as key value pair of the file. For example: width=1024, the key is "width" and the value is "1024". The tracker and storage contain one or more servers. The servers in the tracker or storage cluster can be added to or removed from the cluster by any time without affecting the online services. The servers in the tracker cluster are peer to peer. The storarge servers organizing by the file volume/group to obtain high capacity. The storage system contains one or more volumes whose files are independent among these volumes. The capacity of the whole storage system equals to the sum of all volumes' capacity. A file volume contains one or more storage servers whose files are same among these servers. The servers in a file volume backup each other, and all these servers are load balancing. When adding a storage server to a volume, files already existing in this volume are replicated to this new server automatically, and when this replication done, system will switch this server online to providing storage services. When the whole storage capacity is insufficiency, you can add one or more volumes to expand the storage capacity. To do this, you need to add one or more storage servers. The identification of a file is composed of two parts: the volume name and the file name.

大意爲：
fastdfs是一個開源的，高性能的的分佈式文件系統，他主要的功能包括：文件存儲，同步和訪問，設計基於高可用和負載均衡，fastfd很是適用於基於文件服務的站點，例如圖片分享和視頻分享網站

fastfds有兩個角色：跟蹤服務和存儲服務，跟蹤服務控制，調度文件以負載均衡的方式訪問；存儲服務包括：文件存儲，文件同步，提供文件訪問接口，同時以key value的方式管理文件的元數據

跟蹤和存儲服務能夠由1臺或者多臺服務器組成，同時能夠動態的添加，刪除跟蹤和存儲服務而不會對在線的服務產生影響，在集羣中，tracker服務是對等的

存儲系統由一個或多個卷組成，卷與卷之間的文件是相互獨立的，全部卷的文件容量累加就是整個存儲系統中的文件容量。一個卷能夠由一臺或多臺存儲服務器組成，一個卷下的存儲服務器中的文件都是相同的，卷中的多臺存儲服務器起到了冗餘備份和負載均衡的做用。在卷中增長服務器時，同步已有的文件由系統自動完成，同步完成後，系統自動將新增服務器切換到線上提供服務。當存儲空間不足或即將耗盡時，能夠動態添加捲。只須要增長一臺或多臺服務器，並將它們配置爲一個新的卷，這樣就擴大了存儲系統的容量。

下面幾張圖能夠清楚的說明fastfds的架構和文件上傳和下載流程等：

下面將介紹下fastdfs在rhel上的部署過程
tracker服務器：192.168.123.110/24
storage服務器：192.168.123.20/24

一：編譯安裝

[root@db1 ~]# wget http://cloud.github.com/downloads/libevent/libevent/libevent-2.0.14
-stable.tar.gz
[root@db1 ~]# tar -zxvpf libevent-2.0.14-stable.tar.gz
[root@db1 ~]# cd libevent-2.0.14-stable
[root@db1 libevent-2.0.14-stable]# ./configure --prefix=/usr/local/libevent-2.0.14 &&
make && make install
[root@db1 ~]# wget http://fastdfs.googlecode.com/files/FastDFS_v3.02.tar.gz
[root@db1 ~]# tar -zxvf FastDFS_v3.02.tar.gz
[root@db1 ~]# cd FastDFS
[root@db1 FastDFS]# grep -A 4 '/usr/local/FastDFS' make.sh
TARGET_PREFIX=/usr/local/FastDFS
TARGET_CONF_PATH=/etc/fdfs
WITH_HTTPD=1
WITH_LINUX_SERVICE=1
[root@db1 FastDFS]# ./make.sh C_INCLUDE_PATH=/usr/local/libevent-2.0.14/include
LIBRARY_PATH=/usr/local/libevent-2.0.14/lib
[root@db1 FastDFS]# ./make.sh install
[root@db1 FastDFS]# ls /etc/fdfs/
client.conf  http.conf  mime.types  storage.conf  tracker.conf

二：tracker配置文件

[root@db1 ~]# grep -v '^#' /etc/fdfs/tracker.conf |grep -v '^$'
disabled=false
bind_addr=192.168.123.110
port=22122
connect_timeout=30
network_timeout=60
base_path=/home/data/fastdfs
max_connections=256
work_threads=4
store_lookup=2
store_group=group2
store_server=0
store_path=0
download_server=0
reserved_storage_space = 4GB
log_level=info
run_by_group=
run_by_user=
allow_hosts=*
sync_log_buff_interval = 10
check_active_interval = 120
thread_stack_size = 64KB
storage_ip_changed_auto_adjust = true
storage_sync_file_max_delay = 86400
storage_sync_file_max_time = 300
use_trunk_file = false
slot_min_size = 256
slot_max_size = 16MB
trunk_file_size = 64MB
http.disabled=false
http.server_port=8080
http.check_alive_interval=30
http.check_alive_type=tcp
http.check_alive_uri=/status.html
http.need_find_content_type=true

[root@db1 ~]# grep -v '^#' /etc/fdfs/http.conf  |grep -v '^$'
http.default_content_type = application/octet-stream
http.mime_types_filename=/etc/fdfs/mime.types
http.anti_steal.check_token=false
http.anti_steal.token_ttl=900
http.anti_steal.secret_key=FastDFS1234567890
http.anti_steal.token_check_fail=/home/data/fastdfs/conf/anti-steal.jpg

三：啓動tracker服務，須要注意tracker.conf文件最後一行爲#include httpd.conf

[root@db1 ~]# /usr/local/FastDFS/bin/fdfs_trackerd /etc/fdfs/tracker.conf
/usr/local/FastDFS/bin/fdfs_trackerd: error while loading shared libraries: libevent-
2.0.so.5: cannot open shared object file: No such file or directory
[root@db1 ~]# echo '/usr/local/libevent-2.0.14/include/' >> /etc/ld.so.conf
[root@db1 ~]# echo '/usr/local/libevent-2.0.14/lib/' >> /etc/ld.so.conf
[root@db1 ~]# ldconfig
[root@db1 ~]# /usr/local/FastDFS/bin/fdfs_trackerd /etc/fdfs/tracker.conf
[2012-07-04 17:52:25] ERROR - file: tracker_func.c, line: 160, "/home/data/fastdfs"
can't be accessed, error info: No such file or directory
[root@db1 ~]# mkdir -p /home/data/fastdfs
[root@db1 ~]# /usr/local/FastDFS/bin/fdfs_trackerd /etc/fdfs/tracker.conf
[root@db1 ~]# echo $?
22
[root@db1 ~]# cat  /home/data/fastdfs/logs/trackerd.log
[2012-07-04 17:52:50] ERROR - file: ../common/fdfs_http_shared.c, line: 128, param
"http.mime_types_filename" not exist or is empty
[root@db1 ~]# tail -1 /etc/fdfs/tracker.conf
#include http.conf
[root@db1 ~]# /usr/local/FastDFS/bin/fdfs_trackerd /etc/fdfs/tracker.conf
[root@db1 ~]# echo $?
0
[root@db1 ~]# ps -ef |grep track
root      3535     1  0 15:47 ?        00:00:00 /usr/local/FastDFS/bin/fdfs_trackerd
/etc/fdfs/tracker.conf

[root@db1 ~]# netstat -ntpl |grep fdfs
tcp        0      0 192.168.123.110:22122       0.0.0.0:*                   LISTEN
3535/fdfs_trackerd
tcp        0      0 192.168.123.110:8080        0.0.0.0:*                   LISTEN
3535/fdfs_trackerd

四：storage配置文件

[root@db2 ~]# grep -v '^#' /etc/fdfs/storage.conf |grep -v '^$'
disabled=false
group_name=group1
bind_addr=192.168.123.20
client_bind=true
port=23000
connect_timeout=30
network_timeout=60
heart_beat_interval=30
stat_report_interval=60
base_path=/home/data/fastdfs
max_connections=256
buff_size = 256KB
work_threads=4
disk_rw_separated = true
disk_rw_direct = false
disk_reader_threads = 1
disk_writer_threads = 1
sync_wait_msec=50
sync_interval=0
sync_start_time=00:00
sync_end_time=23:59
write_mark_file_freq=500
store_path_count=1
store_path0=/home/data/fastdfs
subdir_count_per_path=256
tracker_server=192.168.123.110:22122
log_level=info
run_by_group=
run_by_user=
allow_hosts=*
file_distribute_path_mode=0
file_distribute_rotate_count=100
fsync_after_written_bytes=0
sync_log_buff_interval=10
sync_binlog_buff_interval=10
sync_stat_file_interval=300
thread_stack_size=512KB
upload_priority=10
if_alias_prefix=
check_file_duplicate=0
key_namespace=FastDFS
keep_alive=0
http.disabled=false
httphttp.domain_name=
http.server_port=8888
http.trunk_size=256KB
http.need_find_content_type=true

[root@db2 ~]# grep -v '^#' /etc/fdfs/client.conf  |grep -v '^$'
connect_timeout=30
network_timeout=60
base_path=/home/data/fastdfs
tracker_server=192.168.123.110:22122
log_level=info
http.tracker_server_port=8080

[root@db2 ~]#  grep -v '^#' /etc/fdfs/http.conf |grep -v '^$'
http.default_content_type = application/octet-stream
http.mime_types_filename=mime.types
http.anti_steal.check_token=false
http.anti_steal.token_ttl=900
http.anti_steal.secret_key=FastDFS1234567890
http.anti_steal.token_check_fail=/home/data/fastdfs/conf/anti-steal.jpg

五：啓動storage，須要注意storage.conf文件最後一行爲#include httpd.conf

[root@db2 ~]# mkdir -p /home/data/fastdfs
[root@db2 ~]# echo '/usr/local/libevent-2.0.14/include/' >> /etc/ld.so.conf
[root@db2 ~]# echo '/usr/local/libevent-2.0.14/lib/' >> /etc/ld.so.conf
[root@db2 ~]# ldconfig

[root@db2 ~]# tail -2 /etc/fdfs/storage.conf
#use "#include" directive to include HTTP other settings
#include http.conf

[root@db2 ~]# /usr/local/FastFDS/bin/fdfs_storaged /etc/fdfs/storage.conf
mkdir data path: 00 ...
mkdir data path: 01 ...
mkdir data path: 02 ...
———輸出省略————
data path: /home/data/fastdfs/data, mkdir sub dir done.

[root@db2 ~]# ps -ef |grep fdfs
root     14451     1  0 16:15 ?        00:00:00 /usr/local/FastFDS/bin/fdfs_storaged
/etc/fdfs/storage.conf
root     14468  8238  0 16:16 pts/1    00:00:00 grep fdfs

[root@db2 ~]# netstat -ntpl |grep fdfs
tcp        0      0 192.168.123.20:8888         0.0.0.0:*                   LISTEN
14451/fdfs_storaged
tcp        0      0 192.168.123.20:23000        0.0.0.0:*                   LISTEN
14451/fdfs_storaged

五：測試文件上傳

[root@db2 ~]# /usr/local/FastFDS/bin/fdfs_test /etc/fdfs/client.conf upload /etc/passwd This is FastDFS client test program v3.02 Copyright (C) 2008, Happy Fish / YuQing FastDFS may be copied only under the terms of the GNU General Public License V3, which may be found in the FastDFS source kit. Please visit the FastDFS Home Page http://www.csource.org/ for more detail. [2012-07-30 16:25:59] INFO - base_path=/home/data/fastdfs, connect_timeout=30, network_timeout=60, tracker_server_count=1, anti_steal_token=0, anti_steal_secret_key length=0 tracker_query_storage_store_list_without_group:  server 1. group_name=group1, ip_addr=192.168.123.20, port=23000 group_name=group1, ip_addr=192.168.123.20, port=23000 storage_upload_by_filename group_name=group1, remote_filename=M00/00/00/wKh7FFAWRRfcMOGtAAAHKBGhZhE8065783 source ip address: 192.168.123.20 file timestamp=2012-07-30 16:25:59 file size=1832 file crc32=295790097 file url: http://192.168.123.110:8080/group1/M00/00/00/wKh7FFAWRRfcMOGtAAAHKBGhZhE8065783 storage_upload_slave_by_filename group_name=group1, remote_filename=M00/00/00/wKh7FFAWRRfcMOGtAAAHKBGhZhE8065783_big source ip address: 192.168.123.20 file timestamp=2012-07-30 16:25:59 file size=1832 file crc32=295790097 file url: http://192.168.123.110:8080/group1/M00/00/00/wKh7FFAWRRfcMOGtAAAHKBGhZhE8065783_big [root@db2 ~]# /usr/local/FastFDS/bin/fdfs_test /etc/fdfs/client.conf upload 2.jpg This is FastDFS client test program v3.02 Copyright (C) 2008, Happy Fish / YuQing FastDFS may be copied only under the terms of the GNU General Public License V3, which may be found in the FastDFS source kit. Please visit the FastDFS Home Page http://www.csource.org/ for more detail. [2012-07-30 16:29:22] INFO - base_path=/home/data/fastdfs, connect_timeout=30, network_timeout=60, tracker_server_count=1, anti_steal_token=0, anti_steal_secret_key length=0 tracker_query_storage_store_list_without_group:  server 1. group_name=group1, ip_addr=192.168.123.20, port=23000 group_name=group1, ip_addr=192.168.123.20, port=23000 storage_upload_by_filename group_name=group1, remote_filename=M00/00/00/wKh7FFAWReLzPs4sAAEYquSDGwc472.jpg source ip address: 192.168.123.20 file timestamp=2012-07-30 16:29:22 file size=71850 file crc32=3833797383 file url: http://192.168.123.110:8080/group1/M00/00/00/wKh7FFAWReLzPs4sAAEYquSDGwc472.jpg storage_upload_slave_by_filename group_name=group1, remote_filename=M00/00/00/wKh7FFAWReLzPs4sAAEYquSDGwc472_big.jpg source ip address: 192.168.123.20 file timestamp=2012-07-30 16:29:22 file size=71850 file crc32=3833797383 file url: http://192.168.123.110:8080/group1/M00/00/00/wKh7FFAWReLzPs4sAAEYquSDGwc472_big.jpg