Nginx+Fastdfs

注：
在配置時，使用非root用戶配置

fdfs/fdfs

 

1.    集羣部署

 

1.1.    準備

建立目錄：本文檔中全部內容安裝到/fdfs目錄

[fdfs@5861be93b5b0 /]$mkdir -p /fdfs/fastdfs/data /fdfs/nginx/nginx_temp /fdfs/soft && ln -s /fdfs/fastdfs/data /fdfs/fastdfs/data/M00

[fdfs@5861be93b5b0 /]$sudo yum install gcc make gcc-c++ -y

上傳文件：
上傳【Nginx+Fastdfs.zip】包到安裝目錄/fdfs/soft，執行解壓到soft目錄

[fdfs@5861be93b5b0 /]$cd /fdfs/soft && unzip Nginx+Fastdfs.zip

 

1.2.    libfastcommon , fastdfs
 

1.2.1安裝libfastcommon 

[fdfs@5861be93b5b0 /]$cd /fdfs/soft && tar -xvf libfastcommon.tar&&cd libfastcommon

[fdfs@5861be93b5b0 /]$./make.sh && ./make.sh install

 

1.2.2安裝 fastdfs

[fdfs@5861be93b5b0 /]$cd /fdfs/soft && tar -xvf FastDFS_v5.08.tar.gz && cd FastDFS

 

1.2.3拷貝配置文件到/fdfs/fastdfs/config
 

[fdfs@5861be93b5b0 /]$TARGET_CONF_PATH=/fdfs/fastdfs/config
     

[fdfs@5861be93b5b0 /]$

    cat > ./1.sh <<EOF
    TARGET_CONF_PATH=/fdfs/fastdfs/config
    if [ ! -d \$TARGET_CONF_PATH ]; then
            mkdir -p \$TARGET_CONF_PATH

    fi
            cp -f conf/tracker.conf \$TARGET_CONF_PATH
            cp -f conf/storage.conf \$TARGET_CONF_PATH
            cp -f conf/client.conf \$TARGET_CONF_PATH
            cp -f conf/http.conf \$TARGET_CONF_PATH
            cp -f conf/mime.types \$TARGET_CONF_PATH

   
    EOF
    [fdfs@5861be93b5b0 /]$sh 1.sh&&rm -f 1.sh

1.2.4修改make文件配置路徑爲/fdfs/fastdfs

[fdfs@5861be93b5b0 /]$sed -i 's#$DESTDIR/usr$#/fdfs/fastdfs#' make.sh
[fdfs@5861be93b5b0 /]$sed -i 's#$DESTDIR/etc/fdfs#/fdfs/fastdfs/config#' make.sh
[fdfs@5861be93b5b0 /]$sed -i 's#$DESTDIR/etc/init.d#/fdfs/fastdfs/init.d#' make.sh

1.2.5安裝

[fdfs@5861be93b5b0 /]$./make.sh && ./make.sh install

 

1.2.2.    驗證

進入/fdfs/fastdfs，出現如下5個目錄bin、config、data（手工建立）、include、lib、init.d

[fdfs@5861be93b5b0 /]$ll /fdfs/fastdfs
total 312
drwxr-xr-x. 2 root root   4096 Nov  5 06:20 bin
drwxr-xr-x. 2 root root    204 Nov  5 06:46 config
drwxr-xr-x. 2 root root     17 Nov  5 06:17 data
drwxr-xr-x. 3 root root     21 Nov  5 06:20 include
drwxr-xr-x. 2 root root     48 Nov  5 06:20 init.d
drwxr-xr-x. 2 root root     54 Nov  5 06:56 lib64

 

把libfastcommon 的動態庫關聯到fdfs

 ln -s /usr/lib64/libfastcommon.so /fdfs/fastdfs/lib64/libfastcommon.so
 ln -s /usr/lib64/libfastcommon.so /usr/lib/libfastcommon.so
 ln -s /fdfs/fastdfs/lib64/libfdfsclient.so /usr/lib64/libfdfsclient.so
 ln -s /fdfs/fastdfs/lib64/libfdfsclient.so /usr/lib/libfdfsclient.so
 ln -s /usr/include/fastcommon/* /fdfs/fastdfs/include/fastdfs/

1.3.    nginx

1.3.1.    解壓依賴庫

[fdfs@5861be93b5b0 /]$cd /fdfs/soft && tar -zxvf pcre-8.42.tar.gz && tar -zxvf zlib-1.2.11.tar.gz && tar -zxvf openssl-1.0.2n.tar.gz

1.3.2.    配置fastdfs-nginx-module模塊

[fdfs@5861be93b5b0 /]$cd /fdfs/soft && tar -zxvf fastdfs-nginx-module_v1.16.tar.gz && cd fastdfs-nginx-module/src && cp mod_fastdfs.conf /fdfs/fastdfs/config

修改對應的庫路徑
sed -i 's#/usr/local/#/fdfs/fastdfs/#g' config
sed -i 's#/etc/fdfs/#/fdfs/fastdfs/config/#' config
sed -i 's#lib#lib64#' config

1.3.3.    安裝nginx

[fdfs@5861be93b5b0 /]$cd /fdfs/soft && tar -xvf nginx-1.15.5.tar.gz && cd nginx-1.15.5 && ./configure --prefix=/fdfs/nginx --with-pcre=/fdfs/soft/pcre-8.42 --with-zlib=/fdfs/soft/zlib-1.2.11 --with-openssl=/fdfs/soft/openssl-1.0.2n --add-module=/fdfs/soft/fastdfs-nginx-module/src && make && make install

1.3.4.    驗證
進入/nginx/nginx，出現如下5個目錄conf、html、logs、nginx_temp（手工建立）、sbin

[root@72fcbfa4c397 libfastcommon-master]$ ll /fdfs/nginx/
total 4
drwxr-xr-x. 2 root root 4096 Dec 20 08:54 conf
drwxr-xr-x. 2 root root   40 Dec 20 08:54 html
drwxr-xr-x. 2 root root    6 Dec 20 08:54 logs
drwxr-xr-x. 2 root root    6 Dec 20 07:33 nginx_temp
drwxr-xr-x. 2 root root   19 Dec 20 08:54 sbin

 

2.    配置

2.1 dfs的storage

[fdfs@5861be93b5b0 /]$cd /fdfs/fastdfs/config
[fdfs@5861be93b5b0 /]$vi storage.conf

port=23000                                  /* storage的端口，默認爲23000*/

group_name=group1                           /*分組，默認爲group1*/

base_path=/fdfs/fastdfs                   /*放置data和log的目錄*/

store_path0=/fdfs/fastdfs                  /*放置文件的目錄*/

tracker_server= 192.168.1.1:22122              /*tracker server的ip和端口，
tracker_server= 192.168.1.2:22122              能夠寫多個tracker server，每行一個*/

http.server_port=8080                          /*web server的端口改爲8080*/

 

 

2.2.    dfs的tracker

2.2.1.    修改配置tracker.conf文件

[fdfs@5861be93b5b0 /]$cd /fdfs/fastdfs/config
[fdfs@5861be93b5b0 /]$vi tracker.conf

port=22122                                   /* tracker的端口，默認爲22122*/

base_path=/fdfs/fastdfs                    /*放置data和log的目錄*/

store_group=group1                             /*分組，改成group1*/

reserved_storage_space = 10%                     /*磁盤小於10%不容許上傳*/

http.server_port=8080                           /*web server的端口改爲8080*/

 

2.3.    配置nginx的插件

2.3.1.    修改配置mod_fastdfs.conf文件

[fdfs@5861be93b5b0 /]$cp /fdfs/soft/fastdfs-nginx-module/src/mod_fastdfs.conf /fdfs/fastdfs/config
[fdfs@5861be93b5b0 /]$cd /fdfs/fastdfs/config
[fdfs@5861be93b5b0 /]$vi mod_fastdfs.conf

connect_timeout=30

network_timeout=60

base_path=/fdfs/fastdfs               /*放置log的目錄*/

tracker_server= 192.168.1.1:22122           /*tracker server的ip和端口，
tracker_server= 192.168.1.2:22122            能夠寫多個tracker server，每行一個*/

storage_server_port=23000                   /* storage的端口*/

group_name=group1                          /*分組，默認爲group1*/

url_have_group_name=true                   /*是否在URL中包含group名稱*/

store_path0=/fdfs/fastdfs               /*放置文件的目錄*/

http.need_find_content_type=true

/*增長內容*/
http.mime_types_filename=/fdfs/fastdfs/config/mime.types
http.default_content_type=application/octet-stream
include /fdfs/fastdfs/config/http.conf

 

2.3.2.    修改配置http.conf文件

[fdfs@5861be93b5b0 /]$cd /fdfs/fastdfs/config
[fdfs@5861be93b5b0 /]$vi http.conf

http.mime_types_filename=/fdfs/fastdfs/config/mime.types

http.anti_steal.token_check_fail=/fdfs/fastdfs/config/anti-steal.jpg

 

2.3.3.    修改mime.types

[fdfs@5861be93b5b0 /]$cd /fdfs/fastdfs/config
[fdfs@5861be93b5b0 /]$vi mime.types

/*增長內容*/
#--add next contents
application/vnd.ms-word.document.macroEnabled.12 docm
application/vnd.openxmlformats docx pptx xlsx
application/vnd.ms-word.template.macroEnabled.12 dotm
application/vnd.openxmlformats-officedocument.wordprocessingml.template dotx
application/vnd.ms-powerpoint.template.macroEnabled.12 potm
application/vnd.openxmlformats-officedocument.presentationml.template potx
application/vnd.ms-powerpoint.addin.macroEnabled.12 ppam
application/vnd.ms-powerpoint.slideshow.macroEnabled.12 ppsm
application/vnd.openxmlformats-officedocument.presentationml.slideshow ppsx
application/vnd.ms-powerpoint.presentation.macroEnabled.12 pptm
application/vnd.ms-excel.addin.macroEnabled.12 xlam
application/vnd.ms-excel.sheet.binary.macroEnabled.12 xlsb
application/vnd.ms-excel.sheet.macroEnabled.12 xlsm
application/vnd.openxmlformats-officedocument.spreadsheetml.template xltx
application/vnd.ms-excel.template.macroEnabled.12 xltm

 

2.3.4.    修改client.conf

[fdfs@5861be93b5b0 /]$cd /fdfs/fastdfs/config
[fdfs@5861be93b5b0 /]$vi client.conf

base_path=/fdfs/fastdfs             /*放置文件的目錄*/

tracker_server=192.168.1.1:22122      /*tracker server的ip和端口，
tracker_server=192.168.1.2:22122        能夠寫多個tracker server，每行一個*/

http.tracker_server_port=8080

 

2.4.    配置nginx

2.4.1.    修改配置nginx.conf文件

[fdfs@5861be93b5b0 /]$cd /fdfs/nginx/conf
[fdfs@5861be93b5b0 /]$vi nginx.conf

使用如下內容：
worker_processes  4;
error_log logs/error.log;
events {
worker_connections  1024;
use epoll;
}
http {
    include       mime.types;
    default_type  application/octet-stream;
    sendfile        on;
    keepalive_timeout  65;
    server {
          listen       8080;
          server_name  localhost;
          access_log logs/access.log;

          location ~ /group1/M00/* {
               root /fdfs/fastdfs/data;
               ngx_fastdfs_module;
               client_max_body_size 10m;
               client_body_temp_path /fdfs/nginx/nginx_temp;
               }
           }

   upstream  group1 {
                server   192.168.1.1:8080 ;
                server   192.168.1.2:8080 ;
            }

       server {
            listen       8081;
            server_name  localhost;
            location /group1/M00 {
                 proxy_pass http://group1;
                 proxy_redirect off;
                 proxy_set_header Host $host;
                 proxy_set_header X-Real-IP $remote_addr;
                 proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
                 }
            }
}

2.5.    Nginx 啓動

啓動：/fdfs/nginx/sbin/nginx -c /fdfs/nginx/conf/nginx.conf
中止：/fdfs/nginx/sbin/nginx -s stop
重載：/fdfs/nginx/sbin/nginx -s reload
日誌：/fdfs/nginx/logs/error.log
/fdfs/nginx/logs/access.log

首次啓動可能出現如下錯誤
/fdfs/nginx/sbin/nginx: error while loading shared libraries: libfastcommon.so: cannot open shared object file: No such file or directory
解決辦法：

echo "export LD_LIBRARY_PATH=/fdfs/fastdfs/lib:$LD_LIBRARY_PATH" >> ~/.bash_profile
source ~/.bash_profile

2.6.    Tracker命令

啓動：/fdfs/fastdfs/bin/fdfs_trackerd /fdfs/fastdfs/config/tracker.conf
中止：killall fdfs_trackerd
日誌：/fdfs/fastdfs/logs/trackerd.log

2.7.    Storage命令

啓動：

/fdfs/fastdfs/bin/fdfs_storaged /fdfs/fastdfs/config/storage.conf

data path: /home/yuqing/fastdfs/data, mkdir sub dir...
mkdir data path: 00 ...
mkdir data path: 01 ...
mkdir data path: 02 ...

中止：killall fdfs_storaged
日誌：/fdfs/fastdfs/logs/storaged.log

2.8.    測試

集羣測試首次須要兩臺都啓動，不然會報錯

2.8.1.    測試分佈式文件系統上傳圖片

上傳命令格式：
/fdfs/fastdfs/bin/fdfs_upload_file  /fdfs/fastdfs/config/storage.conf    目標文件

[root@72fcbfa4c397 /]# /fdfs/fastdfs/bin/fdfs_upload_file /fdfs/fastdfs/config/storage.conf skr.jpg
group1/M00/00/00/rBEAAlwbcieANq7nAAL_SObQPXc730.jpg

在兩臺服務器的/fdfs/fastdfs/data/00/00目錄能夠找到此文件

2.8.2.    測試負載均衡系統訪問

如下地址都可訪問到內容
http://192.168.4.1:8080/group1/M00/00/00/rBEAAlwbcieANq7nAAL_SObQPXc730.jpg
http://192.168.1.2:8080/group1/M00/00/00/rBEAAlwbcieANq7nAAL_SObQPXc730.jpg
http://192.168.1.1:8081/group1/M00/00/00/rBEAAlwbcieANq7nAAL_SObQPXc730.jpg
http://192.168.1.2:8081/group1/M00/00/00/rBEAAlwbcieANq7nAAL_SObQPXc730.jpg