FastDFS存儲-集羣配置

Nginx相關軟件

nginx-1.4.7 下載地址: http://nginx.org/en/download.html#    能夠升級選擇高版本

nginx清除緩存模塊 ngx_cache_purge-2.1 http://labs.frickle.com/nginx_ngx_cache_purge/

pcre-8.36 ftp://ftp.csx.cam.ac.uk/pub/software/programming/pcre/pcre-8.36.tar.gz

zlib庫 http://zlib.net/zlib-1.2.8.tar.gz

FastDFS相關軟件

FastDFS_v5.04
http://sourceforge.net/projects/fastdfs/files
FastDFS Nginx Module 1.16
http://sourceforge.net/projects/fastdfs/files/FastDFS%20Nginx%20Module%20Source%20Code/
libfastcommon
https://github.com/happyfish100/libfastcommon.git

拓撲圖

 

二：tracker的安裝與配置

1.安裝

如下操做是在 tracker 10.10.10.124上操做
經過rz工具，把軟件上傳到/usr/local/src目錄下，若是沒有rz工具，可使用 yum -y install lrzsz 安裝

[root@tracker src]# ls
FastDFS_v5.04.tar.gz
[root@tracker src]# tar xf FastDFS_v5.04.tar.gz
[root@tracker src]# cd FastDFS
[root@tracker FastDFS]# ll
總用量 132
drwxr-xr-x. 3 8980 users 4096 9月 21 22:14 client
drwxr-xr-x. 2 8980 users 4096 9月 21 22:14 common
drwxr-xr-x. 2 8980 users 4096 9月 21 22:14 conf
-rw-r--r--. 1 8980 users 35067 9月 21 22:14 COPYING-3_0.txt
-rw-r--r--. 1 8980 users 2802 9月 21 22:14 fastdfs.spec
-rw-r--r--. 1 8980 users 31224 9月 21 22:14 HISTORY
drwxr-xr-x. 2 8980 users 4096 9月 21 22:14 init.d
-rw-r--r--. 1 8980 users 7755 9月 21 22:14 INSTALL
-rwxr-xr-x. 1 8980 users 5735 9月 21 22:14 make.sh
drwxr-xr-x. 2 8980 users 4096 9月 21 22:14 php_client
-rw-r--r--. 1 8980 users 2380 9月 21 22:14 README.md
-rwxr-xr-x. 1 8980 users 1768 9月 21 22:14 restart.sh
-rwxr-xr-x. 1 8980 users 1680 9月 21 22:14 stop.sh
drwxr-xr-x. 4 8980 users 4096 9月 21 22:14 storage
drwxr-xr-x. 2 8980 users 4096 9月 21 22:14 test
drwxr-xr-x. 2 8980 users 4096 9月 21 22:14 tracker
[root@tracker FastDFS]# ./make.sh
執行make.sh 遇到的錯誤

[root@tracker FastDFS]# ./make.sh
cc -Wall -D_FILE_OFFSET_BITS=64 -D_GNU_SOURCE -g -O -DDEBUG_FLAG -DOS_LINUX -DIOEVENT_USE_EPOLL -c -o ../common/fdfs_global.o ../common/fdfs_global.c -I../common -I/usr/include/fastcommon
../common/fdfs_global.c:20:20: 錯誤：logger.h：沒有那個文件或目錄
在包含自 ../common/fdfs_global.c：21 的文件中:
../common/fdfs_global.h:14:27: 錯誤：common_define.h：沒有那個文件或目錄
../common/fdfs_global.h:16:29: 錯誤：connection_pool.h：沒有那個文件或目錄
In file included from ../common/fdfs_global.c:21:
../common/fdfs_global.h:26: 錯誤：‘MAX_PATH_SIZE’未聲明(不在函數內)
../common/fdfs_global.h:27: 錯誤：expected ‘=’, ‘,’, ‘;’, ‘asm’ or ‘__attribute__’ before ‘g_fdfs_version’
../common/fdfs_global.h:28: 錯誤：expected ‘=’, ‘,’, ‘;’, ‘asm’ or ‘__attribute__’ before ‘g_use_connection_pool’
../common/fdfs_global.h:29: 錯誤：expected ‘=’, ‘,’, ‘;’, ‘asm’ or ‘__attribute__’ before ‘g_connection_pool’
../common/fdfs_global.c:23: 錯誤：‘DEFAULT_CONNECT_TIMEOUT’未聲明(不在函數內)
../common/fdfs_global.c:24: 錯誤：‘DEFAULT_NETWORK_TIMEOUT’未聲明(不在函數內)
../common/fdfs_global.c:26: 錯誤：expected ‘=’, ‘,’, ‘;’, ‘asm’ or ‘__attribute__’ before ‘g_fdfs_version’
../common/fdfs_global.c:27: 錯誤：expected ‘=’, ‘,’, ‘;’, ‘asm’ or ‘__attribute__’ before ‘g_use_connection_pool’
../common/fdfs_global.c:28: 錯誤：expected ‘=’, ‘,’, ‘;’, ‘asm’ or ‘__attribute__’ before ‘g_connection_pool’
../common/fdfs_global.c: 在函數‘fdfs_check_data_filename’中:
../common/fdfs_global.c:39: 警告：隱式聲明函數‘logError’
../common/fdfs_global.c:45: 警告：隱式聲明函數‘IS_UPPER_HEX’
make: *** [../common/fdfs_global.o] 錯誤 1
cc -Wall -D_FILE_OFFSET_BITS=64 -D_GNU_SOURCE -g -O -DDEBUG_FLAG -DOS_LINUX -DIOEVENT_USE_EPOLL -c -o ../common/fdfs_global.o ../common/fdfs_global.c -I. -Itrunk_mgr -I../common -I../tracker -I../client -Ifdht_client -I/usr/include/fastcommon
../common/fdfs_global.c:20:20: 錯誤：logger.h：沒有那個文件或目錄
在包含自 ../common/fdfs_global.c：21 的文件中:
../common/fdfs_global.h:14:27: 錯誤：common_define.h：沒有那個文件或目錄
../common/fdfs_global.h:16:29: 錯誤：connection_pool.h：沒有那個文件或目錄
In file included from ../common/fdfs_global.c:21:
../common/fdfs_global.h:26: 錯誤：‘MAX_PATH_SIZE’未聲明(不在函數內)
../common/fdfs_global.h:27: 錯誤：expected ‘=’, ‘,’, ‘;’, ‘asm’ or ‘__attribute__’ before ‘g_fdfs_version’
../common/fdfs_global.h:28: 錯誤：expected ‘=’, ‘,’, ‘;’, ‘asm’ or ‘__attribute__’ before ‘g_use_connection_pool’
../common/fdfs_global.h:29: 錯誤：expected ‘=’, ‘,’, ‘;’, ‘asm’ or ‘__attribute__’ before ‘g_connection_pool’
../common/fdfs_global.c:23: 錯誤：‘DEFAULT_CONNECT_TIMEOUT’未聲明(不在函數內)
../common/fdfs_global.c:24: 錯誤：‘DEFAULT_NETWORK_TIMEOUT’未聲明(不在函數內)
../common/fdfs_global.c:26: 錯誤：expected ‘=’, ‘,’, ‘;’, ‘asm’ or ‘__attribute__’ before ‘g_fdfs_version’
../common/fdfs_global.c:27: 錯誤：expected ‘=’, ‘,’, ‘;’, ‘asm’ or ‘__attribute__’ before ‘g_use_connection_pool’
../common/fdfs_global.c:28: 錯誤：expected ‘=’, ‘,’, ‘;’, ‘asm’ or ‘__attribute__’ before ‘g_connection_pool’
../common/fdfs_global.c: 在函數‘fdfs_check_data_filename’中:
../common/fdfs_global.c:39: 警告：隱式聲明函數‘logError’
../common/fdfs_global.c:45: 警告：隱式聲明函數‘IS_UPPER_HEX’
make: *** [../common/fdfs_global.o] 錯誤 1
cc -Wall -D_FILE_OFFSET_BITS=64 -D_GNU_SOURCE -g -O -DDEBUG_FLAG -DOS_LINUX -DIOEVENT_USE_EPOLL -c -o ../common/fdfs_global.o ../common/fdfs_global.c -I../common -I../tracker -I/usr/include/fastcommon
../common/fdfs_global.c:20:20: 錯誤：logger.h：沒有那個文件或目錄
在包含自 ../common/fdfs_global.c：21 的文件中:
../common/fdfs_global.h:14:27: 錯誤：common_define.h：沒有那個文件或目錄
../common/fdfs_global.h:16:29: 錯誤：connection_pool.h：沒有那個文件或目錄
In file included from ../common/fdfs_global.c:21:
../common/fdfs_global.h:26: 錯誤：‘MAX_PATH_SIZE’未聲明(不在函數內)
../common/fdfs_global.h:27: 錯誤：expected ‘=’, ‘,’, ‘;’, ‘asm’ or ‘__attribute__’ before ‘g_fdfs_version’
../common/fdfs_global.h:28: 錯誤：expected ‘=’, ‘,’, ‘;’, ‘asm’ or ‘__attribute__’ before ‘g_use_connection_pool’
../common/fdfs_global.h:29: 錯誤：expected ‘=’, ‘,’, ‘;’, ‘asm’ or ‘__attribute__’ before ‘g_connection_pool’
../common/fdfs_global.c:23: 錯誤：‘DEFAULT_CONNECT_TIMEOUT’未聲明(不在函數內)
../common/fdfs_global.c:24: 錯誤：‘DEFAULT_NETWORK_TIMEOUT’未聲明(不在函數內)
../common/fdfs_global.c:26: 錯誤：expected ‘=’, ‘,’, ‘;’, ‘asm’ or ‘__attribute__’ before ‘g_fdfs_version’
../common/fdfs_global.c:27: 錯誤：expected ‘=’, ‘,’, ‘;’, ‘asm’ or ‘__attribute__’ before ‘g_use_connection_pool’
../common/fdfs_global.c:28: 錯誤：expected ‘=’, ‘,’, ‘;’, ‘asm’ or ‘__attribute__’ before ‘g_connection_pool’
../common/fdfs_global.c: 在函數‘fdfs_check_data_filename’中:
../common/fdfs_global.c:39: 警告：隱式聲明函數‘logError’
../common/fdfs_global.c:45: 警告：隱式聲明函數‘IS_UPPER_HEX’
make: *** [../common/fdfs_global.o] 錯誤 1
看 源碼目錄下的INSTALL文件發現，須要安裝 libfastcommon

#step 1. download libfastcommon source package from github and install it,
the github address:
https://github.com/happyfish100/libfastcommon.git
到如上的網址下載 libfastcommon進行安裝，上傳至服務器的src目錄下

[root@tracker src]# unzip libfastcommon-master.zip
[root@tracker src]# cd libfastcommon-master
[root@tracker libfastcommon-master]# ls
HISTORY INSTALL libfastcommon.spec make.sh README src
[root@tracker libfastcommon-master]# ./make.sh
[root@tracker libfastcommon-master]# ./make.sh install
mkdir -p /usr/lib64
install -m 755 libfastcommon.so /usr/lib64
mkdir -p /usr/include/fastcommon
install -m 644 common_define.h hash.h chain.h logger.h base64.h shared_func.h pthread_func.h ini_file_reader.h _os_bits.h sockopt.h sched_thread.h http_func.h md5.h local_ip_func.h avl_tree.h ioevent.h ioevent_loop.h fast_task_queue.h fast_timer.h process_ctrl.h fast_mblock.h connection_pool.h /usr/include/fastcommon
至此libfastcommon安裝完成。
再回到 FastDFS目錄下來安裝FastDFS

[root@tracker libfastcommon-master]# cd ../FastDFS
[root@tracker FastDFS]# ./make.sh
cc -Wall -D_FILE_OFFSET_BITS=64 -D_GNU_SOURCE -g -O -DDEBUG_FLAG -DOS_LINUX -DIOEVENT_USE_EPOLL -c -o ../common/fdfs_global.o ../common/fdfs_global.c -I../common -I/usr/include/fastcommon
cc -Wall -D_FILE_OFFSET_BITS=64 -D_GNU_SOURCE -g -O -DDEBUG_FLAG -DOS_LINUX -DIOEVENT_USE_EPOLL -c -o tracker_proto.o tracker_proto.c -I../common -I/usr/include/fastcommon
cc -Wall -D_FILE_OFFSET_BITS=64 -D_GNU_SOURCE -g -O -DDEBUG_FLAG -DOS_LINUX -DIOEVENT_USE_EPOLL -c -o tracker_mem.o tracker_mem.c -I../common -I/usr/include/fastcommon
cc -Wall -D_FILE_OFFSET_BITS=64 -D_GNU_SOURCE -g -O -DDEBUG_FLAG -DOS_LINUX -DIOEVENT_USE_EPOLL -c -o tracker_service.o tracker_service.c -I../common -I/usr/include/fastcommon
cc -Wall -D_FILE_OFFSET_BITS=64 -D_GNU_SOURCE -g -O -DDEBUG_FLAG -DOS_LINUX -DIOEVENT_USE_EPOLL -c -o tracker_status.o tracker_status.c -I../common -I/usr/include/fastcommon
cc -Wall -D_FILE_OFFSET_BITS=64 -D_GNU_SOURCE -g -O -DDEBUG_FLAG -DOS_LINUX -DIOEVENT_USE_EPOLL -c -o tracker_global.o tracker_global.c -I../common -I/usr/include/fastcommon
cc -Wall -D_FILE_OFFSET_BITS=64 -D_GNU_SOURCE -g -O -DDEBUG_FLAG -DOS_LINUX -DIOEVENT_USE_EPOLL -c -o tracker_func.o tracker_func.c -I../common -I/usr/include/fastcommon
cc -Wall -D_FILE_OFFSET_BITS=64 -D_GNU_SOURCE -g -O -DDEBUG_FLAG -DOS_LINUX -DIOEVENT_USE_EPOLL -c -o fdfs_shared_func.o fdfs_shared_func.c -I../common -I/usr/include/fastcommon
cc -Wall -D_FILE_OFFSET_BITS=64 -D_GNU_SOURCE -g -O -DDEBUG_FLAG -DOS_LINUX -DIOEVENT_USE_EPOLL -c -o tracker_nio.o tracker_nio.c -I../common -I/usr/include/fastcommon
cc -Wall -D_FILE_OFFSET_BITS=64 -D_GNU_SOURCE -g -O -DDEBUG_FLAG -DOS_LINUX -DIOEVENT_USE_EPOLL -c -o tracker_relationship.o tracker_relationship.c -I../common -I/usr/include/fastcommon
cc -Wall -D_FILE_OFFSET_BITS=64 -D_GNU_SOURCE -g -O -DDEBUG_FLAG -DOS_LINUX -DIOEVENT_USE_EPOLL -c -o tracker_dump.o tracker_dump.c -I../common -I/usr/include/fastcommon
cc -Wall -D_FILE_OFFSET_BITS=64 -D_GNU_SOURCE -g -O -DDEBUG_FLAG -DOS_LINUX -DIOEVENT_USE_EPOLL -c -o ../common/linux_stack_trace.o ../common/linux_stack_trace.c -I../common -I/usr/include/fastcommon
cc -Wall -D_FILE_OFFSET_BITS=64 -D_GNU_SOURCE -g -O -DDEBUG_FLAG -DOS_LINUX -DIOEVENT_USE_EPOLL -o fdfs_trackerd fdfs_trackerd.c ../common/fdfs_global.o tracker_proto.o tracker_mem.o tracker_service.o tracker_status.o tracker_global.o tracker_func.o fdfs_shared_func.o tracker_nio.o tracker_relationship.o tracker_dump.o ../common/linux_stack_trace.o -L/usr/local/lib -lpthread -ldl -rdynamic -lfastcommon -I../common -I/usr/include/fastcommon
cc -Wall -D_FILE_OFFSET_BITS=64 -D_GNU_SOURCE -g -O -DDEBUG_FLAG -DOS_LINUX -DIOEVENT_USE_EPOLL -c -o tracker_client_thread.o tracker_client_thread.c -I. -Itrunk_mgr -I../common -I../tracker -I../client -Ifdht_client -I/usr/include/fastcommon
cc -Wall -D_FILE_OFFSET_BITS=64 -D_GNU_SOURCE -g -O -DDEBUG_FLAG -DOS_LINUX -DIOEVENT_USE_EPOLL -c -o storage_global.o storage_global.c -I. -Itrunk_mgr -I../common -I../tracker -I../client -Ifdht_client -I/usr/include/fastcommon
cc -Wall -D_FILE_OFFSET_BITS=64 -D_GNU_SOURCE -g -O -DDEBUG_FLAG -DOS_LINUX -DIOEVENT_USE_EPOLL -c -o storage_func.o storage_func.c -I. -Itrunk_mgr -I../common -I../tracker -I../client -Ifdht_client -I/usr/include/fastcommon
cc -Wall -D_FILE_OFFSET_BITS=64 -D_GNU_SOURCE -g -O -DDEBUG_FLAG -DOS_LINUX -DIOEVENT_USE_EPOLL -c -o storage_service.o storage_service.c -I. -Itrunk_mgr -I../common -I../tracker -I../client -Ifdht_client -I/usr/include/fastcommon
cc -Wall -D_FILE_OFFSET_BITS=64 -D_GNU_SOURCE -g -O -DDEBUG_FLAG -DOS_LINUX -DIOEVENT_USE_EPOLL -c -o storage_sync.o storage_sync.c -I. -Itrunk_mgr -I../common -I../tracker -I../client -Ifdht_client -I/usr/include/fastcommon
...
[root@tracker FastDFS]# ./make.sh install
mkdir -p /usr/bin
mkdir -p /etc/fdfs
cp -f fdfs_trackerd /usr/bin
if [ ! -f /etc/fdfs/tracker.conf.sample ]; then cp -f ../conf/tracker.conf /etc/fdfs/tracker.conf.sample; fi
mkdir -p /usr/bin
mkdir -p /etc/fdfs
cp -f fdfs_storaged /usr/bin
if [ ! -f /etc/fdfs/storage.conf.sample ]; then cp -f ../conf/storage.conf /etc/fdfs/storage.conf.sample; fi
mkdir -p /usr/bin
mkdir -p /etc/fdfs
mkdir -p /usr/lib64
cp -f fdfs_monitor fdfs_test fdfs_test1 fdfs_crc32 fdfs_upload_file fdfs_download_file fdfs_delete_file fdfs_file_info fdfs_appender_test fdfs_appender_test1 fdfs_append_file fdfs_upload_appender /usr/bin
if [ 0 -eq 1 ]; then cp -f libfdfsclient.a /usr/lib64; fi
if [ 1 -eq 1 ]; then cp -f libfdfsclient.so /usr/lib64; fi
mkdir -p /usr/include/fastdfs
cp -f ../common/fdfs_define.h ../common/fdfs_global.h ../common/mime_file_parser.h ../common/fdfs_http_shared.h ../tracker/tracker_types.h ../tracker/tracker_proto.h ../tracker/fdfs_shared_func.h ../storage/trunk_mgr/trunk_shared.h tracker_client.h storage_client.h storage_client1.h client_func.h client_global.h fdfs_client.h /usr/include/fastdfs
if [ ! -f /etc/fdfs/client.conf.sample ]; then cp -f ../conf/client.conf /etc/fdfs/client.conf.sample; fi
至此FastDFS在 tracker上安裝完成，全部的可執行文件位於 /usr/bin 目錄下，以fdfs開頭的文件

[root@tracker FastDFS]# ll /usr/bin/fd*
-rwxr-xr-x. 1 root root 257631 10月 28 11:29 /usr/bin/fdfs_appender_test
-rwxr-xr-x. 1 root root 257424 10月 28 11:29 /usr/bin/fdfs_appender_test1
-rwxr-xr-x. 1 root root 247664 10月 28 11:29 /usr/bin/fdfs_append_file
-rwxr-xr-x. 1 root root 247340 10月 28 11:29 /usr/bin/fdfs_crc32
-rwxr-xr-x. 1 root root 247723 10月 28 11:29 /usr/bin/fdfs_delete_file
-rwxr-xr-x. 1 root root 248850 10月 28 11:29 /usr/bin/fdfs_download_file
-rwxr-xr-x. 1 root root 248584 10月 28 11:29 /usr/bin/fdfs_file_info
-rwxr-xr-x. 1 root root 260848 10月 28 11:29 /usr/bin/fdfs_monitor
-rwxr-xr-x. 1 root root 886610 10月 28 11:29 /usr/bin/fdfs_storaged
-rwxr-xr-x. 1 root root 264031 10月 28 11:29 /usr/bin/fdfs_test
-rwxr-xr-x. 1 root root 263208 10月 28 11:29 /usr/bin/fdfs_test1
-rwxr-xr-x. 1 root root 374872 10月 28 11:29 /usr/bin/fdfs_trackerd
-rwxr-xr-x. 1 root root 248730 10月 28 11:29 /usr/bin/fdfs_upload_appender
-rwxr-xr-x. 1 root root 249676 10月 28 11:29 /usr/bin/fdfs_upload_file
[root@tracker FastDFS]#
全部的配置文件在 /etc/fdfs/ 目錄下

[root@tracker FastDFS]# ll /etc/fdfs/
總用量 20
-rw-r--r--. 1 root root 1461 10月 28 11:29 client.conf.sample
-rw-r--r--. 1 root root 7647 10月 28 11:29 storage.conf.sample
-rw-r--r--. 1 root root 7102 10月 28 11:29 tracker.conf.sample
[root@tracker FastDFS]#

至此 FastDFS在tracker 上安裝完成，如下進行配置

2.配置：

進入到 /etc/fdfs 目錄下，複製模版文件到 tracker.conf

[root@tracker fdfs]# cp tracker.conf.sample tracker.conf
建立數據文件和日誌文件目錄

[root@tracker fdfs]# mkdir -pv /data/fastdfs/tracker
mkdir: 已建立目錄 "/data"
mkdir: 已建立目錄 "/data/fastdfs"
mkdir: 已建立目錄 "/data/fastdfs/tracker"
編輯 tracker.conf 文件，測試的時候只須要修改如下參數便可

disabled=false #啓用配置文件
port=22122 #設置 tracker 的端口號
base_path=/data/fastdfs/tracker #設置 tracker 的數據文件和日誌目錄（需預先建立）
http.server_port=8080 #設置 http 端口號
這個http.server_port=8080 指的是在tracker服務器上啓動http服務進程，如:apache或者nginx 啓動時所監聽的端口，這個彷佛是能夠不用管的，由於tracker自己就沒有安裝http服務

3.運行：

直接使用 fdfs_trackerd 來啓動tracker進程，而後使用netstat 查看端口是否起來。

[root@tracker fdfs]# fdfs_trackerd /etc/fdfs/tracker.conf restart
[root@tracker fdfs]# netstat -antp | grep trackerd
tcp 0 0 0.0.0.0:22122 0.0.0.0:* LISTEN 14520/fdfs_trackerd
[root@tracker fdfs]#
也能夠查看日誌看服務是否正常運行
[root@tracker tracker]# cat /data/fastdfs/tracker/logs/trackerd.log
[2014-10-28 13:53:43] INFO - FastDFS v5.04, base_path=/data/fastdfs/tracker, run_by_group=, run_by_user=, connect_timeout=30s, network_timeout=60s, port=22122, bind_addr=, max_connections=256, accept_threads=1, work_threads=4, store_lookup=2, store_group=, store_server=0, store_path=0, reserved_storage_space=10.00%, download_server=0, allow_ip_count=-1, sync_log_buff_interval=10s, check_active_interval=120s, thread_stack_size=64 KB, storage_ip_changed_auto_adjust=1, storage_sync_file_max_delay=86400s, storage_sync_file_max_time=300s, use_trunk_file=0, slot_min_size=256, slot_max_size=16 MB, trunk_file_size=64 MB, trunk_create_file_advance=0, trunk_create_file_time_base=02:00, trunk_create_file_interval=86400, trunk_create_file_space_threshold=20 GB, trunk_init_check_occupying=0, trunk_init_reload_from_binlog=0, trunk_compress_binlog_min_interval=0, use_storage_id=0, id_type_in_filename=ip, storage_id_count=0, rotate_error_log=0, error_log_rotate_time=00:00, rotate_error_log_size=0, log_file_keep_days=0, store_slave_file_use_link=0, use_connection_pool=0, g_connection_pool_max_idle_time=3600s
[2014-10-28 13:53:43] INFO - local_host_ip_count: 2, 127.0.0.1 10.10.10.124

4.設置開機自動啓動：

[root@tracker tracker]# echo "/usr/bin/fdfs_trackerd /etc/fdfs/tracker.conf restart" >> /etc/rc.local
[root@tracker tracker]# cat /etc/rc.local
#!/bin/sh
#
# This script will be executed *after* all the other init scripts.
# You can put your own initialization stuff in here if you don't
# want to do the full Sys V style init stuff.
touch /var/lock/subsys/local
/usr/bin/fdfs_trackerd /etc/fdfs/tracker.conf restart
[root@tracker tracker]#
至此tracker上的安裝與配置結束。

三：storagede 安裝與配置：

1.安裝：

如下操做是在 10.10.10.120 storage1上操做
利用上傳 FastDFS_v5.04.tar.gz 和 libfastcommon-master.zip 至 storage服務器的/usr/local/src 目錄下

[root@server1 src]# rz
rz waiting to receive.
Starting zmodem transfer. Press Ctrl+C to cancel.
Transferring FastDFS_v5.04.tar.gz...
100% 335 KB 335 KB/sec 00:00:01 0 Errors
[root@server1 src]# ls
FastDFS_v5.04.tar.gz
[root@server1 src]# rz
rz waiting to receive.
Starting zmodem transfer. Press Ctrl+C to cancel.
Transferring libfastcommon-master.zip...
100% 95 KB 95 KB/sec 00:00:01 0 Errors
[root@server1 src]# ll
總用量 432
-rw-r--r-- 1 root root 343477 10月 27 09:26 FastDFS_v5.04.tar.gz
-rw-r--r-- 1 root root 97548 10月 28 11:14 libfastcommon-master.zip
[root@server1 src]#
解壓安裝 libfastcommon

[root@server1 src]# unzip libfastcommon-master.zip
[root@server1 libfastcommon-master]# ll
總用量 24
-rw-r--r-- 1 root root 2397 10月 26 11:53 HISTORY
-rw-r--r-- 1 root root 582 10月 26 11:53 INSTALL
-rw-r--r-- 1 root root 1341 10月 26 11:53 libfastcommon.spec
-rwxr-xr-x 1 root root 2151 10月 26 11:53 make.sh
-rw-r--r-- 1 root root 617 10月 26 11:53 README
drwxr-xr-x 2 root root 4096 10月 26 11:53 src
[root@server1 libfastcommon-master]# ./make.sh
cc -Wall -D_FILE_OFFSET_BITS=64 -g -DDEBUG_FLAG -DOS_LINUX -DIOEVENT_USE_EPOLL -c -fPIC -o hash.lo hash.c
cc -Wall -D_FILE_OFFSET_BITS=64 -g -DDEBUG_FLAG -DOS_LINUX -DIOEVENT_USE_EPOLL -c -fPIC -o chain.lo chain.c
cc -Wall -D_FILE_OFFSET_BITS=64 -g -DDEBUG_FLAG -DOS_LINUX -DIOEVENT_USE_EPOLL -c -fPIC -o shared_func.lo shared_func.c
cc -Wall -D_FILE_OFFSET_BITS=64 -g -DDEBUG_FLAG -DOS_LINUX -DIOEVENT_USE_EPOLL -c -fPIC -o ini_file_reader.lo ini_file_reader.c
cc -Wall -D_FILE_OFFSET_BITS=64 -g -DDEBUG_FLAG -DOS_LINUX -DIOEVENT_USE_EPOLL -c -fPIC -o logger.lo logger.c
cc -Wall -D_FILE_OFFSET_BITS=64 -g -DDEBUG_FLAG -DOS_LINUX -DIOEVENT_USE_EPOLL -c -fPIC -o sockopt.lo sockopt.c
cc -Wall -D_FILE_OFFSET_BITS=64 -g -DDEBUG_FLAG -DOS_LINUX -DIOEVENT_USE_EPOLL -c -fPIC -o base64.lo base64.c
...

r1 libfastcommon-master]# ./make.sh install
mkdir -p /usr/lib64
install -m 755 libfastcommon.so /usr/lib64
mkdir -p /usr/include/fastcommon
install -m 644 common_define.h hash.h chain.h logger.h base64.h shared_func.h pthread_func.h ini_file_reader.h _os_bits.h sockopt.h sched_thread.h http_func.h md5.h local_ip_func.h avl_tree.h ioevent.h ioevent_loop.h fast_task_queue.h fast_timer.h process_ctrl.h fast_mblock.h connection_pool.h /usr/include/fastcommon
解壓安裝 FastDFS_v5.04.tar.gz

[root@server1 src]# tar xf FastDFS_v5.04.tar.gz
[root@server1 src]# cd FastDFS
[root@server1 FastDFS]# ll
總用量 132
drwxr-xr-x 3 8980 users 4096 9月 21 22:14 client
drwxr-xr-x 2 8980 users 4096 9月 21 22:14 common
drwxr-xr-x 2 8980 users 4096 9月 21 22:14 conf
-rw-r--r-- 1 8980 users 35067 9月 21 22:14 COPYING-3_0.txt
-rw-r--r-- 1 8980 users 2802 9月 21 22:14 fastdfs.spec
-rw-r--r-- 1 8980 users 31224 9月 21 22:14 HISTORY
drwxr-xr-x 2 8980 users 4096 9月 21 22:14 init.d
-rw-r--r-- 1 8980 users 7755 9月 21 22:14 INSTALL
-rwxr-xr-x 1 8980 users 5735 9月 21 22:14 make.sh
drwxr-xr-x 2 8980 users 4096 9月 21 22:14 php_client
-rw-r--r-- 1 8980 users 2380 9月 21 22:14 README.md
-rwxr-xr-x 1 8980 users 1768 9月 21 22:14 restart.sh
-rwxr-xr-x 1 8980 users 1680 9月 21 22:14 stop.sh
drwxr-xr-x 4 8980 users 4096 9月 21 22:14 storage
drwxr-xr-x 2 8980 users 4096 9月 21 22:14 test
drwxr-xr-x 2 8980 users 4096 9月 21 22:14 tracker
[root@server1 FastDFS]#
[root@server1 FastDFS]# ./make.sh
cc -Wall -D_FILE_OFFSET_BITS=64 -D_GNU_SOURCE -g -O -DDEBUG_FLAG -DOS_LINUX -DIOEVENT_USE_EPOLL -c -o ../common/fdfs_global.o ../common/fdfs_global.c -I../common -I/usr/include/fastcommon
cc -Wall -D_FILE_OFFSET_BITS=64 -D_GNU_SOURCE -g -O -DDEBUG_FLAG -DOS_LINUX -DIOEVENT_USE_EPOLL -c -o tracker_proto.o tracker_proto.c -I../common -I/usr/include/fastcommon
cc -Wall -D_FILE_OFFSET_BITS=64 -D_GNU_SOURCE -g -O -DDEBUG_FLAG -DOS_LINUX -DIOEVENT_USE_EPOLL -c -o tracker_mem.o tracker_mem.c -I../common -I/usr/include/fastcommon
cc -Wall -D_FILE_OFFSET_BITS=64 -D_GNU_SOURCE -g -O -DDEBUG_FLAG -DOS_LINUX -DIOEVENT_USE_EPOLL -c -o tracker_service.o tracker_service.c -I../common -I/usr/include/fastcommon
cc -Wall -D_FILE_OFFSET_BITS=64 -D_GNU_SOURCE -g -O -DDEBUG_FLAG -DOS_LINUX -DIOEVENT_USE_EPOLL -c -o tracker_status.o tracker_status.c -I../common -I/usr/include/fastcommon
...
[root@server1 FastDFS]# ./make.sh install
mkdir -p /usr/bin
mkdir -p /etc/fdfs
cp -f fdfs_trackerd /usr/bin
if [ ! -f /etc/fdfs/tracker.conf.sample ]; then cp -f ../conf/tracker.conf /etc/fdfs/tracker.conf.sample; fi
mkdir -p /usr/bin
mkdir -p /etc/fdfs
cp -f fdfs_storaged /usr/bin
if [ ! -f /etc/fdfs/storage.conf.sample ]; then cp -f ../conf/storage.conf /etc/fdfs/storage.conf.sample; fi
mkdir -p /usr/bin
mkdir -p /etc/fdfs
mkdir -p /usr/lib64
cp -f fdfs_monitor fdfs_test fdfs_test1 fdfs_crc32 fdfs_upload_file fdfs_download_file fdfs_delete_file fdfs_file_info fdfs_appender_test fdfs_appender_test1 fdfs_append_file fdfs_upload_appender /usr/bin
if [ 0 -eq 1 ]; then cp -f libfdfsclient.a /usr/lib64; fi
if [ 1 -eq 1 ]; then cp -f libfdfsclient.so /usr/lib64; fi
mkdir -p /usr/include/fastdfs
cp -f ../common/fdfs_define.h ../common/fdfs_global.h ../common/mime_file_parser.h ../common/fdfs_http_shared.h ../tracker/tracker_types.h ../tracker/tracker_proto.h ../tracker/fdfs_shared_func.h ../storage/trunk_mgr/trunk_shared.h tracker_client.h storage_client.h storage_client1.h client_func.h client_global.h fdfs_client.h /usr/include/fastdfs
if [ ! -f /etc/fdfs/client.conf.sample ]; then cp -f ../conf/client.conf /etc/fdfs/client.conf.sample; fi
安裝完成後，可執行文件在/usr/bin/下以 fdfs 開頭

[root@server1 FastDFS]# ll /usr/bin/fdfs_*
-rwxr-xr-x 1 root root 257631 10月 28 14:18 /usr/bin/fdfs_appender_test
-rwxr-xr-x 1 root root 257424 10月 28 14:18 /usr/bin/fdfs_appender_test1
-rwxr-xr-x 1 root root 247664 10月 28 14:18 /usr/bin/fdfs_append_file
-rwxr-xr-x 1 root root 247340 10月 28 14:18 /usr/bin/fdfs_crc32
-rwxr-xr-x 1 root root 247723 10月 28 14:18 /usr/bin/fdfs_delete_file
-rwxr-xr-x 1 root root 248850 10月 28 14:18 /usr/bin/fdfs_download_file
-rwxr-xr-x 1 root root 248584 10月 28 14:18 /usr/bin/fdfs_file_info
-rwxr-xr-x 1 root root 260848 10月 28 14:18 /usr/bin/fdfs_monitor
-rwxr-xr-x 1 root root 886610 10月 28 14:18 /usr/bin/fdfs_storaged
-rwxr-xr-x 1 root root 264031 10月 28 14:18 /usr/bin/fdfs_test
-rwxr-xr-x 1 root root 263208 10月 28 14:18 /usr/bin/fdfs_test1
-rwxr-xr-x 1 root root 374872 10月 28 14:18 /usr/bin/fdfs_trackerd
-rwxr-xr-x 1 root root 248730 10月 28 14:18 /usr/bin/fdfs_upload_appender
-rwxr-xr-x 1 root root 249676 10月 28 14:18 /usr/bin/fdfs_upload_file
[root@server1 FastDFS]#
配置文件在 /etc/fdfs 目錄下

[root@server1 FastDFS]# ll /etc/fdfs/

總用量 20
-rw-r--r-- 1 root root 1461 10月 28 14:18 client.conf.sample
-rw-r--r-- 1 root root 7647 10月 28 14:18 storage.conf.sample
-rw-r--r-- 1 root root 7102 10月 28 14:18 tracker.conf.sample
[root@server1 FastDFS]#

2.配置：

進入到 /etc/fdfs 目錄下，複製/usr/local/src/FastDFS/conf/* 下面的全部文件到當前目錄下(/etc/fdfs)

[root@server1 FastDFS]# cd /etc/fdfs/
[root@server1 fdfs]# cp /usr/local/src/FastDFS/conf/* .
[root@server1 fdfs]# ll
總用量 64
-rw-r--r-- 1 root root 1461 10月 29 14:33 client.conf
-rw-r--r-- 1 root root 858 10月 29 14:33 http.conf
-rw-r--r-- 1 root root 31172 10月 29 14:33 mime.types
-rw-r--r-- 1 root root 3697 10月 29 14:34 mod_fastdfs.conf
-rw-r--r-- 1 root root 7646 10月 28 14:34 storage.conf
-rw-r--r-- 1 root root 105 10月 29 14:33 storage_ids.conf
-rw-r--r-- 1 root root 7102 10月 29 14:33 tracker.conf
[root@server1 fdfs]#
編輯配置文件 storage.conf 測試的時候，只需修改如下內容便可

[root@server1 fdfs]# vim storage.conf
disabled=false #啓用配置文件
group_name=group1 #組名，根據實際狀況修改
port=23000 #設置 storage 的端口號
base_path=/data/fastdfs/storage #設置 storage 的日誌目錄（需預先建立）
store_path_count=1 #存儲路徑個數，須要和 store_path 個數匹配
store_path0=/data/fastdfs/storage #存儲路徑
tracker_server=10.10.10.124:22122 #tracker 服務器的 IP 地址和端口號
http.server_port=8080 #設置storage上啓動的http服務的端口號，如安裝的nginx的端口號

3.運行：

[root@server1 fdfs]# fdfs_storaged /etc/fdfs/storage.conf restart
查看端口是否起來

[root@server1 fdfs]# netstat -antp | grep storage
tcp 0 0 0.0.0.0:23000 0.0.0.0:* LISTEN 10316/fdfs_storaged
[root@server1 fdfs]#
觀察日誌 /data/fastdfs/storage/logs/storaged.log 看運行狀況

mkdir data path: FB ...
mkdir data path: FC ...
mkdir data path: FD ...
mkdir data path: FE ...
mkdir data path: FF ...
data path: /data/fastdfs/storage/data, mkdir sub dir done.
[2014-10-28 14:42:55] INFO - file: storage_param_getter.c, line: 191, use_storage_id=0, id_type_in_filename=ip, storage_ip_changed_auto_adjust=1, store_path=0, reserved_storage_space=10.00%, use_trunk_file=0, slot_min_size=256, slot_max_size=16 MB, trunk_file_size=64 MB, trunk_create_file_advance=0, trunk_create_file_time_base=02:00, trunk_create_file_interval=86400, trunk_create_file_space_threshold=20 GB, trunk_init_check_occupying=0, trunk_init_reload_from_binlog=0, trunk_compress_binlog_min_interval=0, store_slave_file_use_link=0
[2014-10-28 14:42:55] INFO - file: storage_func.c, line: 175, tracker_client_ip: 10.10.10.120, my_server_id_str: 10.10.10.120, g_server_id_in_filename: 2013923850
[2014-10-28 14:42:55] INFO - local_host_ip_count: 2, 127.0.0.1 10.10.10.120
[2014-10-28 14:42:55] INFO - file: tracker_client_thread.c, line: 310, successfully connect to tracker server 10.10.10.124:22122, as a tracker client, my ip is 10.10.10.120
[2014-10-28 14:43:25] INFO - file: tracker_client_thread.c, line: 1235, tracker server 10.10.10.124:22122, set tracker leader: 10.10.10.124:22122
看到先建立二級目錄，而後已經成功鏈接到tracker
可使用 fdfs_monitor 來查看一下storage的狀態，看是否已經成功註冊到了tracker

[root@server2 logs]# fdfs_monitor /etc/fdfs/storage.conf
也能夠如下命令來監控服務器的狀態

[root@tracker ~]# fdfs_monitor /etc/fdfs/client.conf
結果以下:

[2014-10-28 15:57:54] DEBUG - base_path=/data/fastdfs/storage, connect_timeout=30, network_timeout=60, tracker_server_count=1, anti_steal_token=0, anti_steal_secret_key length=0, use_connection_pool=0, g_connection_pool_max_idle_time=3600s, use_storage_id=0, storage server id count: 0
server_count=1, server_index=0
tracker server is 10.10.10.124:22122
group count: 1
Group 1:
group name = group1
disk total space = 14020 MB
disk free space = 10267 MB
trunk free space = 0 MB
storage server count = 2
active server count = 2
storage server port = 23000
storage HTTP port = 8080
store path count = 1
subdir count per path = 256
current write server index = 0
current trunk file id = 0
Storage 1:
id = 10.10.10.120
ip_addr = 10.10.10.120 ACTIVE
http domain =
version = 5.04
join time = 2014-10-28 14:42:43
up time = 2014-10-28 14:42:43
total storage = 14020 MB
free storage = 10385 MB
upload priority = 10
store_path_count = 1
subdir_count_per_path = 256
storage_port = 23000
storage_http_port = 8080
current_write_path = 0
source storage id =
if_trunk_server = 0
connection.alloc_count = 256
connection.current_count = 1
connection.max_count = 1
total_upload_count = 0
success_upload_count = 0
total_append_count = 0
success_append_count = 0
total_modify_count = 0
success_modify_count = 0
total_truncate_count = 0
success_truncate_count = 0
total_set_meta_count = 0
success_set_meta_count = 0
total_delete_count = 0
success_delete_count = 0
total_download_count = 0
success_download_count = 0
total_get_meta_count = 0
success_get_meta_count = 0
total_create_link_count = 0
success_create_link_count = 0
total_delete_link_count = 0
success_delete_link_count = 0
total_upload_bytes = 0
success_upload_bytes = 0
total_append_bytes = 0
success_append_bytes = 0
total_modify_bytes = 0
success_modify_bytes = 0
stotal_download_bytes = 0
success_download_bytes = 0
total_sync_in_bytes = 0
success_sync_in_bytes = 0
total_sync_out_bytes = 0
success_sync_out_bytes = 0
total_file_open_count = 0
success_file_open_count = 0
total_file_read_count = 0
success_file_read_count = 0
total_file_write_count = 0
success_file_write_count = 0
last_heart_beat_time = 2014-10-28 15:56:42
last_source_update = 1970-01-01 08:00:00
last_sync_update = 1970-01-01 08:00:00
last_synced_timestamp = 1970-01-01 08:00:00
Storage 2:
id = 10.10.10.122
ip_addr = 10.10.10.122 (shizhouliang.mobimtech.local) ACTIVE
http domain =
version = 5.04
join time = 2014-10-28 15:53:51
up time = 2014-10-28 15:53:51
total storage = 14020 MB
free storage = 10267 MB
upload priority = 10
store_path_count = 1
subdir_count_per_path = 256
storage_port = 23000
storage_http_port = 8080
current_write_path = 0
source storage id = 10.10.10.120
if_trunk_server = 0
connection.alloc_count = 256
connection.current_count = 1
connection.max_count = 1
total_upload_count = 0
success_upload_count = 0
total_append_count = 0
success_append_count = 0
total_modify_count = 0
success_modify_count = 0
total_truncate_count = 0
success_truncate_count = 0
total_set_meta_count = 0
success_set_meta_count = 0
total_delete_count = 0
success_delete_count = 0
total_download_count = 0
success_download_count = 0
total_get_meta_count = 0
success_get_meta_count = 0
total_create_link_count = 0
success_create_link_count = 0
total_delete_link_count = 0
success_delete_link_count = 0
total_upload_bytes = 0
success_upload_bytes = 0
total_append_bytes = 0
success_append_bytes = 0
total_modify_bytes = 0
success_modify_bytes = 0
stotal_download_bytes = 0
success_download_bytes = 0
total_sync_in_bytes = 0
success_sync_in_bytes = 0
total_sync_out_bytes = 0
success_sync_out_bytes = 0
total_file_open_count = 0
success_file_open_count = 0
total_file_read_count = 0
success_file_read_count = 0
total_file_write_count = 0
success_file_write_count = 0
last_heart_beat_time = 2014-10-28 15:56:53
last_source_update = 1970-01-01 08:00:00
last_sync_update = 1970-01-01 08:00:00
last_synced_timestamp = 1970-01-01 08:00:00
看到ACTIVE,就說明已經成功註冊到了tracker。

4.設置開機啓動：

[root@server1 fdfs]# echo "/usr/bin/fdfs_storaged /etc/fdfs/storage.conf restart" >> /etc/rc.local
[root@server1 fdfs]# cat /etc/rc.local
#!/bin/sh
#
# This script will be executed *after* all the other init scripts.
# You can put your own initialization stuff in here if you don't
# want to do the full Sys V style init stuff.
touch /var/lock/subsys/local
/usr/bin/fdfs_storaged /etc/fdfs/storage.conf restart
至此 storage 已經成功安裝，再把另一臺服務器也一樣的步驟進行安裝。
等第二臺storage2(10.10.10.122)安裝完成併成功啓動後，看日誌會發現

[2014-10-28 15:54:05] INFO - file: storage_sync.c, line: 2698, successfully connect to storage server 10.10.10.120:23000
說明已經成功鏈接到另一臺storage了。
也可使用

[root@server2 logs]# fdfs_monitor /etc/fdfs/storage.conf
來查看一下storage2的狀態

四.storage上安裝配置nginx

如下操做是在 storage1 10.10.10.120 上操做

1.安裝nginx

上傳 fastdfs-nginx-module_v1.16.tar.gz nginx-1.4.7.tar.gz pcre-8.36.tar.gz zlib-1.2.8.tar.gz 到storage服務器上的/usr/local/src 目錄下

[root@server1 src]# rz
rz waiting to receive.
Starting zmodem transfer. Press Ctrl+C to cancel.
Transferring fastdfs-nginx-module_v1.16.tar.gz...
100% 17 KB 17 KB/sec 00:00:01 0 Errors
Transferring nginx-1.4.7.tar.gz...
100% 751 KB 751 KB/sec 00:00:01 0 Errors
Transferring pcre-8.36.tar.gz...
100% 1962 KB 1962 KB/sec 00:00:01 0 Errors
Transferring zlib-1.2.8.tar.gz...
100% 557 KB 557 KB/sec 00:00:01 0 Errors
解壓各個文件，而後編譯安裝nginx

[root@server1 src]# tar xf nginx-1.4.7.tar.gz
[root@server1 src]# tar xf pcre-8.36.tar.gz
[root@server1 src]# tar xf zlib-1.2.8.tar.gz
[root@server1 src]# tar xf fastdfs-nginx-module_v1.16.tar.gz
[root@server1 src]# cd nginx-1.4.7
[root@server1 nginx-1.4.7]# ls
auto CHANGES CHANGES.ru conf configure contrib html LICENSE man README src
[root@server1 nginx-1.4.7]# ./configure --prefix=/usr/local/nginx --add-module=/usr/local/src/fastdfs-nginx-module/src --with-pcre=/usr/local/src/pcre-8.36 --with-zlib=/usr/local/src/zlib-1.2.8

出現如下錯誤:

cc -c -pipe -O -W -Wall -Wpointer-arith -Wno-unused-parameter -Werror -g -D_FILE_OFFSET_BITS=64 -DFDFS_OUTPUT_CHUNK_SIZE='256*1024' -DFDFS_MOD_CONF_FILENAME='"/etc/fdfs/mod_fastdfs.conf"' -I src/core -I src/event -I src/event/modules -I src/os/unix -I /usr/local/include/fastdfs -I /usr/local/include/fastcommon/ -I /usr/local/src/pcre-8.36 -I /usr/local/src/zlib-1.2.8 -I objs -I src/http -I src/http/modules -I src/mail \
-o objs/addon/src/ngx_http_fastdfs_module.o \
/usr/local/src/fastdfs-nginx-module/src/ngx_http_fastdfs_module.c
在包含自 /usr/local/src/fastdfs-nginx-module/src/ngx_http_fastdfs_module.c：6 的文件中:
/usr/local/src/fastdfs-nginx-module/src/common.c:21:25: 錯誤：fdfs_define.h：沒有那個文件或目錄
/usr/local/src/fastdfs-nginx-module/src/common.c:22:20: 錯誤：logger.h：沒有那個文件或目錄
/usr/local/src/fastdfs-nginx-module/src/common.c:23:25: 錯誤：shared_func.h：沒有那個文件或目錄
/usr/local/src/fastdfs-nginx-module/src/common.c:24:25: 錯誤：fdfs_global.h：沒有那個文件或目錄
/usr/local/src/fastdfs-nginx-module/src/common.c:25:21: 錯誤：sockopt.h：沒有那個文件或目錄
/usr/local/src/fastdfs-nginx-module/src/common.c:26:23: 錯誤：http_func.h：沒有那個文件或目錄
/usr/local/src/fastdfs-nginx-module/src/common.c:27:30: 錯誤：fdfs_http_shared.h：沒有那個文件或目錄
/usr/local/src/fastdfs-nginx-module/src/common.c:28:25: 錯誤：fdfs_client.h：沒有那個文件或目錄
/usr/local/src/fastdfs-nginx-module/src/common.c:29:27: 錯誤：local_ip_func.h：沒有那個文件或目錄
/usr/local/src/fastdfs-nginx-module/src/common.c:30:30: 錯誤：fdfs_shared_func.h：沒有那個文件或目錄
/usr/local/src/fastdfs-nginx-module/src/common.c:31:26: 錯誤：trunk_shared.h：沒有那個文件或目錄
以上的這個錯誤是因爲沒找到 頭文件致使的，找了找緣由,在編譯fastdf模塊的時候會找這個地方/usr/local/include尋找fastdfs主程序的 fastcommon和fastdfs這2個目錄下的頭文件，可是在編譯安裝的fastdfs主程序的時候，卻把這2個目錄放到了/usr /include下面了，因而複製這2個目錄到/usr/local/include下

[root@server1 src]# cp -rf /usr/include/fast* /usr/local/include/
[root@server1 src]# ls /usr/local/include/
fastcommon fastdfs
[root@server1 src]#
因而再次編譯

[root@server1 src]# cd nginx-1.4.7
[root@server1 nginx-1.4.7]# ./configure --prefix=/usr/local/nginx --add-module=/usr/local/src/fastdfs-nginx-module/src --with-pcre=/usr/local/src/pcre-8.36 --with-zlib=/usr/local/src/zlib-1.2.8
[root@server1 nginx-1.4.7]# make && make install
此次終於編譯安裝經過。

2.準備啓動腳本：

編輯 /etc/init.d/nginx，以下內容:

#!/bin/bash
# nginx Startup script for the Nginx HTTP Server
# it is v.1.4.7 version.
# chkconfig: - 85 15
# description: Nginx is a high-performance web and proxy server.
# It has a lot of features, but it's not for everyone.
# processname: nginx

# pidfile: /usr/local/nginx/logs/nginx.pid
# config: /usr/local/nginx/conf/nginx.conf

nginxd=/usr/local/nginx/sbin/nginx
nginx_config=/usr/local/nginx/conf/nginx.conf
nginx_pid=/usr/local/nginx/logs/nginx.pid
nginx_lock=/var/lock/subsys/nginx
RETVAL=0
prog="nginx"

# Source function library.
. /etc/rc.d/init.d/functions

# Source networking configuration.
. /etc/sysconfig/network

# Check that networking is up.
[ ${NETWORKING} = "no" ] && exit 0
[ -x $nginxd ] || exit 0

# Start nginx daemons functions.
start() {
nginx_is_run=`ps -ef | egrep 'nginx:\s*(worker|master)\s*process' | wc -l`
if [ ${nginx_is_run} -gt 0 ];then
echo "nginx already running...."
exit 1
fi
echo -n $"Starting $prog: "
daemon $nginxd -c ${nginx_config}
RETVAL=$?
echo
[ $RETVAL = 0 ] && touch ${nginx_lock}
return $RETVAL
}

# Stop nginx daemons functions.
stop() {
echo -n $"Stopping $prog: "
killproc $nginxd
RETVAL=$?
echo
[ $RETVAL = 0 ] && rm -f ${nginx_lock} ${nginx_pid}
}

# Reload nginx config file
reload() {
echo -n $"Reloading $prog: "
#kill -HUP `cat ${nginx_pid}`
killproc $nginxd -HUP
RETVAL=$?
echo
}

# See how we were called.
case "$1" in
start)
start
;;
stop)
stop
;;
reload)
reload
;;
restart)
stop
start
;;
status)
status $prog
RETVAL=$?
;;
*)
echo $"Usage: $prog {start|stop|restart|reload|status|help}"
exit 1
esac

3.將FastDFS的nginx插件模塊的配置文件

copy 到 FastDFS 配置文件目錄

[root@server1 src]# cp fastdfs-nginx-module/src/mod_fastdfs.conf /etc/fdfs/

4.修改nginx的配置文件

4.1 修改nginx的監聽端口

將 server 段中的 listen 端口號改成 8080：

[root@server1 nginx]# vim /usr/local/nginx/conf/nginx.conf
listen 8080;

4.2 在 server 段中添加fastdfs的配置
這裏的group[1-3] 能夠匹配 group1,group2,group3，固然group也能夠是其餘的名字

location ~ /group[1-3]/M00 {
root /data/fastdfs/storage/data;
ngx_fastdfs_module;
}
/data/fastdfs/storage/data 這個是指向 真正存儲文件的地方，該目錄下有這些個二級目錄

[root@server1 src]# cd /data/fastdfs/storage/data/
[root@server1 data]# ll
總用量 1036
drwxr-xr-x 258 root root 4096 10月 28 14:42 00
drwxr-xr-x 258 root root 4096 10月 28 14:42 01
drwxr-xr-x 258 root root 4096 10月 28 14:42 02
drwxr-xr-x 258 root root 4096 10月 28 14:42 03
drwxr-xr-x 258 root root 4096 10月 28 14:42 04
drwxr-xr-x 258 root root 4096 10月 28 14:42 05
drwxr-xr-x 258 root root 4096 10月 28 14:42 06
drwxr-xr-x 258 root root 4096 10月 28 14:42 07
drwxr-xr-x 258 root root 4096 10月 28 14:42 08
drwxr-xr-x 258 root root 4096 10月 28 14:42 09
drwxr-xr-x 258 root root 4096 10月 28 14:42 0A
drwxr-xr-x 258 root root 4096 10月 28 14:42 0B
drwxr-xr-x 258 root root 4096 10月 28 14:42 0C
drwxr-xr-x 258 root root 4096 10月 28 14:42 0D
drwxr-xr-x 258 root root 4096 10月 28 14:42 0E
drwxr-xr-x 258 root root 4096 10月 28 14:42 0F
drwxr-xr-x 258 root root 4096 10月 28 14:42 10

5.修改fastdfs的nginx模塊的配置文件mod_fastdfs.conf

通常只需改動如下幾個參數便可：

base_path=/data/fastdfs/storage #保存日誌目錄
tracker_server=10.10.10.124:22122 #tracker 服務器的 IP 地址以及端口號
storage_server_port=23000 #storage 服務器的端口號
group_name=group1 #當前服務器的 group 名
url_have_group_name = true #文件 url 中是否有 group 名
store_path_count=1 #存儲路徑個數，須要和 store_path 個數匹配
store_path0=/data/fastdfs/storage #存儲路徑
http.need_find_content_type=true # 從文件 擴展 名查 找 文件 類型 （ nginx 時 爲true）
group_count = 1 #設置組的個數
而後在末尾添加分組信息，目前只有一個分組，就只寫一個

[group1]
group_name=group1
storage_server_port=23000
store_path_count=1
store_path0=/data/fastdfs/storage

6.創建M00至存儲目錄的符號連接

[root@server1 fdfs]# ln -s /data/fastdfs/storage/data /data/fastdfs/storage/data/M00
[root@server1 fdfs]# ll /data/fastdfs/storage/data/M00
lrwxrwxrwx 1 root root 26 10月 29 11:22 /data/fastdfs/storage/data/M00 -> /data/fastdfs/storage/data
至此nginx和FastDFS 插件模塊設置完成

7.啓動nginx

[root@server1 nginx]# chmod u+x /etc/init.d/nginx
[root@server1 nginx]# chkconfig --add nginx
[root@server1 nginx]# chkconfig nginx on
[root@server1 nginx]# service nginx start
正在啓動 nginx： [肯定]
[root@server1 nginx]# service nginx status
nginx (pid 26500) 正在運行...
查看nginx的日誌 錯誤日誌logs/error.log 看是否有問題

[2014-10-29 14:35:29] INFO - local_host_ip_count: 2, 127.0.0.1 10.10.10.120
[2014-10-29 14:35:29] INFO - fastdfs apache / nginx module v1.15, response_mode=proxy, base_path=/tmp, url_have_group_name=1, group_count=1, connect_timeout=2, network_timeout=30, tracker_server_count=1, if_alias_prefix=, local_host_ip_count=2, anti_steal_token=0, token_ttl=0s, anti_steal_secret_key length=0, token_check_fail content_type=, token_check_fail buff length=0, load_fdfs_parameters_from_tracker=1, storage_sync_file_max_delay=86400s, use_storage_id=0, storage server id count=0, flv_support=1, flv_extension=flv
[2014-10-29 14:35:29] INFO - group 1. group_name=group1, storage_server_port=23000, path_count=1, store_path0=/data/fastdfs/storage
也能夠訪問 http://10.10.10.120:8080

看到沒什麼錯誤，至此在storage1上安裝配置nginx結束。
再在storage2上以相同的步驟來安裝nginx。

五：安裝nginx反向代理及緩存服務器

如下操做在 10.10.10.123 上操做

1.安裝nginx：

將所需的軟件包上傳至服務器/usr/local/src下

[root@fastdfs-nginx ~]# cd /usr/local/src/
[root@fastdfs-nginx src]# ls
[root@fastdfs-nginx src]# rz
rz waiting to receive.
Starting zmodem transfer. Press Ctrl+C to cancel.
Transferring nginx-1.4.7.tar.gz...
100% 751 KB 751 KB/sec 00:00:01 0 Errors
Transferring ngx_cache_purge-2.1.tar.gz...
100% 10 KB 10 KB/sec 00:00:01 0 Errors
Transferring pcre-8.36.tar.gz...
100% 1962 KB 1962 KB/sec 00:00:01 0 Errors
Transferring zlib-1.2.8.tar.gz...
100% 557 KB 557 KB/sec 00:00:01 0 Errors
解壓各個軟件包

[root@fastdfs-nginx src]# tar xf nginx-1.4.7.tar.gz
[root@fastdfs-nginx src]# tar xf ngx_cache_purge-2.1.tar.gz
[root@fastdfs-nginx src]# tar xf pcre-8.36.tar.gz
[root@fastdfs-nginx src]# tar xf zlib-1.2.8.tar.gz
[root@fastdfs-nginx src]# cd nginx-1.4.7
編譯安裝nginx

[root@fastdfs-nginx nginx-1.4.7]# ./configure --prefix=/usr/local/nginx --add-module=/usr/local/src/ngx_cache_purge-2.1 --with-pcre=/usr/local/src/pcre-8.36/ --with-zlib=/usr/local/src/zlib-1.2.8
checking for OS
+ Linux 2.6.32-431.el6.x86_64 x86_64
checking for C compiler ... found
+ using GNU C compiler
+ gcc version: 4.4.7 20120313 (Red Hat 4.4.7-4) (GCC)
checking for gcc -pipe switch ... found
checking for gcc builtin atomic operations ... found
checking for C99 variadic macros ... found
checking for gcc variadic macros ... found
checking for unistd.h ... found
checking for inttypes.h ... found
...
Configuration summary
+ using PCRE library: /usr/local/src/pcre-8.36/
+ OpenSSL library is not used
+ md5: using system crypto library
+ sha1: using system crypto library
+ using zlib library: /usr/local/src/zlib-1.2.8
nginx path prefix: "/usr/local/nginx"
nginx binary file: "/usr/local/nginx/sbin/nginx"
nginx configuration prefix: "/usr/local/nginx/conf"
nginx configuration file: "/usr/local/nginx/conf/nginx.conf"
nginx pid file: "/usr/local/nginx/logs/nginx.pid"
nginx error log file: "/usr/local/nginx/logs/error.log"
nginx http access log file: "/usr/local/nginx/logs/access.log"
nginx http client request body temporary files: "client_body_temp"
nginx http proxy temporary files: "proxy_temp"
nginx http fastcgi temporary files: "fastcgi_temp"
nginx http uwsgi temporary files: "uwsgi_temp"
nginx http scgi temporary files: "scgi_temp"
[root@fastdfs-nginx nginx-1.4.7]# make && make install
至此 nginx 以及 nginx cache purge 插件模塊安裝完成。

2.配置：

準備nginx的啓動腳本，內容同上面的storage安裝時的腳本，並加入到系統的服務，開機啓動
建立nginx的緩存目錄

[root@fastdfs-nginx nginx]# mkdir -pv /var/cache/nginx/proxy_cache/tmp
mkdir: 已建立目錄 "/var/cache/nginx"
mkdir: 已建立目錄 "/var/cache/nginx/proxy_cache"
mkdir: 已建立目錄 "/var/cache/nginx/proxy_cache/tmp"
增長系統打開文件數的限制

[root@fastdfs-nginx nginx]# ulimit -SHn 102400
[root@fastdfs-nginx nginx]# echo "ulimit -SHn 102400" >> /etc/rc.local
編輯 vim /usr/local/nginx/conf/nginx.conf 文件修改如下內容:

user nobody;
worker_processes 1;
error_log logs/error.log info;
pid logs/nginx.pid;
events {
worker_connections 65535;
use epoll;
}
http {
include mime.types;
default_type application/octet-stream;
sendfile on;
tcp_nopush on;
keepalive_timeout 65;
server_names_hash_bucket_size 128;
client_header_buffer_size 32k;
large_client_header_buffers 4 32k;
client_max_body_size 300m;
proxy_redirect off;
proxy_set_header Host $http_host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_connect_timeout 90;
proxy_send_timeout 90;
proxy_read_timeout 90;
proxy_buffer_size 16k;
proxy_buffers 4 64k;
proxy_busy_buffers_size 128k;
proxy_temp_file_write_size 128k;
#設置緩存存儲路徑、存儲方式、分配內存大小、磁盤最大空間、緩存期限
proxy_cache_path /var/cache/nginx/proxy_cache levels=1:2
keys_zone=http-cache:500m max_size=10g inactive=30d;
proxy_temp_path /var/cache/nginx/proxy_cache/tmp;
#設置 group1 的服務器
upstream fdfs_group1 {
server 10.10.10.120:8080 weight=1 max_fails=2 fail_timeout=30s;
server 10.10.10.122:8080 weight=1 max_fails=2 fail_timeout=30s;
}
log_format main '$remote_addr - $remote_user [$time_local] "$request" '
'$status $body_bytes_sent "$http_referer" '
'"$http_user_agent" "$http_x_forwarded_for"';
server {
listen 80;
server_name localhost;
access_log logs/host.access.log main;
#設置 group1 的反向代理參數
location /group1/M00 {
proxy_next_upstream http_502 http_504 error timeout invalid_header;
proxy_cache http-cache;
proxy_cache_valid 200 304 12h;
proxy_cache_key $uri$is_args$args;
proxy_pass http://fdfs_group1;
expires 30d;
}
#設置清除緩存的訪問權限
location ~ /purge(/.*) {
allow 127.0.0.1;
allow 10.10.10.0/24;
deny all;
proxy_cache_purge http-cache $1$is_args$args;
}
}
}
使用 /usr/local/nginx/sbin/nginx -t 來測試配置文件語法的正確性

[root@fastdfs-nginx nginx]# ./sbin/nginx -t
nginx: the configuration file /usr/local/nginx/conf/nginx.conf syntax is ok
nginx: configuration file /usr/local/nginx/conf/nginx.conf test is successful

3.啓動nginx

[root@fastdfs-nginx nginx]# service nginx start
正在啓動 nginx： [肯定]
查看監聽的端口

[root@fastdfs-nginx conf]# netstat -antp | grep nginx
tcp 0 0 0.0.0.0:80 0.0.0.0:* LISTEN 20526/nginx
[root@fastdfs-nginx conf]#

六，測試上傳文件

如下操做在tracker服務器（10.10.10.124）上進行

1.建立client使用的日誌目錄

[root@tracker tracker]# mkdir -pv /data/fastdfs/client
mkdir: 已建立目錄 "/data/fastdfs/client"

2.修改客戶端的配置文件/etc/fdfs/client.conf

修改如下參數便可：

base_path=/data/fastdfs/client #日誌存放路徑
tracker_server=10.10.10.124:22122 #tracker 服務器 IP 地址和端口號
http.tracker_server_port=8080 #tracker 服務器的 http 端口號 ，這一項感受沒什麼用處，tracker都沒有啓用http服務

3.上傳一張圖片

[root@tracker ~]# fdfs_upload_file /etc/fdfs/client.conf 1.jpg
group1/M00/00/00/CgoKeFRQqaaASsgNAAEgRqu1UiU712.jpg
[root@tracker ~]#
獲得了一個如上的一個地址 group1/M00/00/00/CgoKeFRQqaaASsgNAAEgRqu1UiU712.jpg
可使用fdfs_file_info 來查看上傳文件的信息

[root@tracker ~]# fdfs_file_info /etc/fdfs/client.conf group1/M00/00/00/CgoKeFRQqaaASsgNAAEgRqu1UiU712.jpg
source storage id: 0
source ip address: 10.10.10.120
file create timestamp: 2014-10-29 16:47:34
file size: 73798
file crc32: 2880786981 (0xABB55225)
這個信息裏面能夠看到 這個文件上傳的時候 首先上傳到了10.10.10.120這臺服務器，文件的建立時間、文件的大小

4.使用瀏覽器訪問一下地址

完整的URL是這樣的 http://10.10.10.123/group1/M00/00/00/CgoKeFRQqaaASsgNAAEgRqu1UiU712.jpg

10.10.10.123 就是nginx反向代理緩存服務器

成功訪問到圖片
查看一下nginx的訪問日誌，能夠看到狀態碼是 200OK.

[root@fastdfs-nginx ~]# tail -1 /usr/local/nginx/logs/host.access.log
10.10.10.46 - - [29/Oct/2014:16:48:43 +0800] "GET /group1/M00/00/00/CgoKeFRQqaaASsgNAAEgRqu1UiU712.jpg HTTP/1.1" 200 73798 "-" "Mozilla/5.0 (Windows NT 6.1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/35.0.1916.153 Safari/537.36" "-"
[root@fastdfs-nginx ~]#
查看nginx的緩存目錄，看是否已經緩存成功

[root@fastdfs-nginx ~]# ll -R /var/cache/nginx/proxy_cache/
/var/cache/nginx/proxy_cache/:
總用量 8
drwx------. 3 nobody nobody 4096 10月 29 16:47 e
drwxr-xr-x. 2 nobody root 4096 10月 29 16:47 tmp
/var/cache/nginx/proxy_cache/e:
總用量 4
drwx------. 2 nobody nobody 4096 10月 29 16:47 a8
/var/cache/nginx/proxy_cache/e/a8:
總用量 76
-rw-------. 1 nobody nobody 74110 10月 29 16:47 54408edb44f74ab4da965506f54a6a8e 在此已經緩存成功
/var/cache/nginx/proxy_cache/tmp:
總用量 0
[root@fastdfs-nginx ~]#

5.清楚緩存：

若要清除該圖片的緩存，能夠在文件 URL 以前加上 purge 便可，以下圖
http://10.10.10.123/purge/group1/M00/00/00/CgoKeFRQqaaASsgNAAEgRqu1UiU712.jpg

6.配置瀏覽器端查看是否命中緩存

可使用Chrome瀏覽器的開發工具，查看響應的頭部信息來查看是否命中緩存，要啓用此功能，修改nginx的主配置文件以下:
在location /group1/M00 段增長這2個行

add_header X-Via $server_addr;
add_header X-Cache-Status $upstream_cache_status;
最終效果

location /group1/M00 {
proxy_next_upstream http_502 http_504 error timeout invalid_header;
proxy_cache http-cache;
proxy_cache_valid 200 304 12h;
proxy_cache_key $uri$is_args$args;
proxy_pass http://fdfs_group1;
expires 30d;
add_header X-Via $server_addr;
add_header X-Cache-Status $upstream_cache_status;
}
重啓nginx

[root@fastdfs-nginx ~]# service nginx restart
中止 nginx： [肯定]
正在啓動 nginx： [肯定]
[root@fastdfs-nginx ~]#
清空chrome瀏覽器的緩存，清除那張圖片在nginx中的緩存
第一次訪問

顯示MISS,表明緩存中沒有
第二次 ctrl+F5 強制刷新訪問

已經顯示HIT，表明已經命中緩存
至此 FastDFS + nginx 反向代理緩存搭建完畢。

七.優化部分

1.Tracker優化：

1) max_connections=10000或更大響應參數增大內存消耗增長該參數需修改單一進程同時可打開文件數量（* soft nofile 65536 * hard nofile 65536）的限制後生效。
2) work_threads=12 cpu數量此處設置cpu核心數量
3) thread_stack_size=256
測試配置以下

work_threads=12
thread_stack_size=256
max_connections=65535

2.storage優化

work_threads= work_threads + 1 + (disk_reader_threads + disk_writer_threads) * store_path_count= 15

disk_rw_separated：磁盤讀寫是否分離
disk_reader_threads：單個磁盤讀線程數
disk_writer_threads：單個磁盤寫線程數
若是磁盤讀寫混合，單個磁盤讀寫線程數爲讀線程數和寫線程數以後
對於單盤掛載方式，磁盤讀寫線程分別設置爲1便可
若是磁盤作了RAID，那麼須要酌情加大讀寫線程數，這樣才能最大程度地發揮磁盤性能
buff_size=512增大提高磁盤IO的另一個技巧，一次儘量多寫入或多讀取。也就是說，將程序的讀寫buffer設置得儘量大一些。例如日誌或者redo log的寫入，不是每次調用都直接寫磁盤，而是先緩存到內存中，等buffer滿了再寫入磁盤，也能夠定時寫入磁盤。
測試配置以下

thread_stack_size=512KB
disk_reader_threads = 4
disk_writer_threads = 4
buff_size=512
max_connections=65535
accept_threads=10
work_threads=16a
以上配置壓力測試上傳併發2w 事務平均響應時間0.46秒，4w事務平均響應時間2.2秒。

八：FastDFS使用流程：

1.上傳文件

FastDFS 提供了多種方式上傳文件：
使用 fdfs_upload_file 上傳
使用 C 語言客戶端接口上傳
使用 PHP 客戶端接口上傳
使用 Java 客戶端接口上傳
使用 Python 客戶端接口上傳
使用.NET 客戶端接口上傳
根據具體狀況選擇使用。上傳均支持斷點續傳。
若是Java的客戶端，在API的配置文件中配置 tracker服務器，若是有多個tracker服務器，就配置多臺
WEB-INF/classes/fdfs_client.conf

connect_timeout = 2
network_timeout = 30
charset = UTF-8
http.tracker_http_port = 8080
http.anti_steal_token = no
http.secret_key = FastDFS1234567890

tracker_server =192.168.10.27:22122
tracker_server = 192.168.10.28:22122

2.下載文件

使用 fdfs_download_file 經過 tracker 服務器下載
使用 nginx 經過 http 方式直接從 storage 下載（支持斷點續傳，做者推薦）

3.檢測服務器資源

使用 fdfs_monitor 查看 tracker 和全部 group 的運行狀況

[root@tracker ~]# fdfs_monitor /etc/fdfs/client.conf
[2014-10-31 14:09:20] DEBUG - base_path=/data/fastdfs/client, connect_timeout=30, network_timeout=60, tracker_server_count=1, anti_steal_token=0, anti_steal_secret_key length=0, use_connection_pool=0, g_connection_pool_max_idle_time=3600s, use_storage_id=0, storage server id count: 0

server_count=1, server_index=0

tracker server is 10.10.10.124:22122

group count: 1

Group 1:
group name = group1
disk total space = 14020 MB
disk free space = 10224 MB
trunk free space = 0 MB
storage server count = 2
active server count = 2
storage server port = 23000
storage HTTP port = 8080
store path count = 1
subdir count per path = 256
current write server index = 1
current trunk file id = 0

Storage 1:
id = 10.10.10.120
ip_addr = 10.10.10.120 ACTIVE
http domain =
version = 5.04
join time = 2014-10-28 14:42:43
up time = 2014-10-29 14:14:44
total storage = 14020 MB
....
Storage 2:
id = 10.10.10.122
ip_addr = 10.10.10.122 ACTIVE
http domain =
version = 5.04
join time = 2014-10-28 15:53:51
up time = 2014-10-28 15:53:51
total storage = 14020 MB
....

4.其它功能

使用 fdfs_crc32 獲取文件 CRC使用 fdfs_delete_file 刪除文件使用 fdfs_file_info 查看文件屬性信息