fastdfs 一個group內實現按照不一樣的項目,指定路徑存儲.

時間 2019-12-06

標籤 fastdfs 一個 group 實現按照不一樣項目指定路徑存儲简体版

原文原文鏈接

環境介紹:javascript

1: 公司目前有5個項目 A B C D E 往後可能會有所增長.css

2: 使用fastdfs存儲這5各項目的文件,要求各各項目的文件分開存儲,也就是每一個項目的文件存儲到一個固定的位置.html

3: 三臺機器ip地址分配以下java

tracker角色 IP:192.168.1.219 nginx

storageA角色 IP: 192.168.1.215app

storageB角色 IP:192.168.1.216dom

4: 這裏主要記錄下實現思路,具體的配置方法都很簡單.tcp

---------tracker這臺機器主要有兩個服務 fdfs_tracker 和 nginx---------------url

tracker.conf的配置文件很簡單這裏不貼出來了 spa

nginx的配置文件以下: nginx負責把訪問請求轉發給後臺的兩臺storage.

[root@trackerB ~]# cat /usr/local/nginx/conf/nginx.conf | grep -v "#" | grep -v "^$"

user www;

worker_processes 1;

error_log logs/error.log;

events {

worker_connections 1024;

}

http {

include mime.types;

default_type application/octet-stream;

server_names_hash_bucket_size 128;

client_header_buffer_size 32k;

large_client_header_buffers 4 32k;

client_max_body_size 300m;

sendfile on;

tcp_nopush on;

keepalive_timeout 120;

gzip on;

gzip_min_length 1k;

gzip_buffers 4 16k;

gzip_http_version 1.1;

gzip_comp_level 2;

gzip_types text/plain application/x-javascript text/css application/xml;

gzip_vary on;

gzip_disable "MSIE[1-6].";

proxy_redirect off;

proxy_set_header Host $http_host;

proxy_set_header X-Real-IP $remote_addr;

proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;

proxy_connect_timeout 90;

proxy_send_timeout 90;

proxy_read_timeout 90;

proxy_buffer_size 16k;

proxy_buffers 4 64k;

proxy_busy_buffers_size 128k;

proxy_temp_file_write_size 128k;

log_format access '$remote_addr - $remote_user [$time_local] "$request" '

'$status $body_bytes_sent "$http_referer" '

'"$http_user_agent" "$upstream_addr" "$upstream_response_time" "$host"';

proxy_cache_path /var/cache/nginx/proxy_cache levels=1:2 keys_zone=http-cache:500m max_size=10g inactive=30d;

proxy_temp_path /var/cache/nginx/proxy_cache/tmp;

upstream group1 {

server 192.168.1.215:80;

server 192.168.1.216:80;

}

server {

listen 80;

server_name image.ty.com;

location ~* /group1/(M00|M01|M02|M03|M04|M05) { #這裏我使用了6個存儲路徑,每一個項目一個

proxy_next_upstream http_502 http_504 error timeout invalid_header;

proxy_cache http-cache;

proxy_cache_valid 200 304 12h;

proxy_cache_key $uri$is_args$args;

proxy_pass http://group1;

expires 3d;

}

access_log logs/image.access.log access;

location / {

root html;

index index.html index.htm;

}

error_page 500 502 503 504 /50x.html;

location = /50x.html {

root html;

}

include /usr/local/nginx/conf/vhost/*.conf;

}

---------------storage角色的兩臺機器-------------------

這兩天機器的配置徹底同樣. 都運行了fdfs_storage 和 nginx , 其中nginx添加了fastdfs模塊

fdfs_storage的配置文件storage.conf內容以下:

[root@localhost ~]# cat /etc/fdfs/storage.conf | grep -v "^#" | grep -v "^$"

disabled=false

group_name=group1

bind_addr=

client_bind=true

port=23000

connect_timeout=30

network_timeout=60

heart_beat_interval=30

stat_report_interval=60

base_path=/data/

max_connections=256

buff_size = 256KB

accept_threads=1

work_threads=4

disk_rw_separated = true

disk_reader_threads = 1

disk_writer_threads = 1

sync_wait_msec=50

sync_interval=0

sync_start_time=00:00

sync_end_time=23:59

write_mark_file_freq=500

store_path_count=6 一共 store_path0-5 六個存儲路徑.

store_path0=/data/data1

store_path1=/data/data2

store_path2=/data/data3

store_path3=/data/data4

store_path4=/data/data5

store_path5=/data/data6

subdir_count_per_path=256

tracker_server=192.168.1.219:22122

log_level=info

run_by_group=

run_by_user=

allow_hosts=*

file_distribute_path_mode=0

file_distribute_rotate_count=100

fsync_after_written_bytes=0

sync_log_buff_interval=10

sync_binlog_buff_interval=10

sync_stat_file_interval=300

thread_stack_size=512KB

upload_priority=10

if_alias_prefix=

check_file_duplicate=0

file_signature_method=hash

key_namespace=FastDFS

keep_alive=0

use_access_log = false

rotate_access_log = false

access_log_rotate_time=00:00

rotate_error_log = false

error_log_rotate_time=00:00

rotate_access_log_size = 0

rotate_error_log_size = 0

log_file_keep_days = 0

file_sync_skip_invalid_record=false

use_connection_pool = false

connection_pool_max_idle_time = 3600

http.domain_name=

http.server_port=8888

nginx配置文件以下

[root@localhost ~]# cat /usr/local/nginx/conf/nginx.conf | grep -v "^#" | grep -v "^$"

worker_processes 1;

events {

worker_connections 1024;

}

http {

include mime.types;

default_type application/octet-stream;

log_format main '$remote_addr - $remote_user [$time_local] "$request" '

'$status $body_bytes_sent "$http_referer" '

'"$http_user_agent" "$http_x_forwarded_for"';

access_log logs/access.log main;

sendfile on;

keepalive_timeout 65;

client_max_body_size 2m;

server {

listen 80;

server_name localhost;

#charset koi8-r;

access_log logs/host.access.log main;

location /group1/M00 { #根據請求路徑配置對應的存儲路徑.

root /data/data1;

ngx_fastdfs_module; #使用nginxmodule處理請求

}

location /group1/M01 {

root /data/data2;

ngx_fastdfs_module;

}

location /group1/M02 {

root /data/data3;

ngx_fastdfs_module;

}

location /group1/M03 {

root /data/data4;

ngx_fastdfs_module;

}

location /group1/M04 {

root /data/data5;

ngx_fastdfs_module;

}

location /group1/M05 {

root /data/data6;

ngx_fastdfs_module;

}

error_page 500 502 503 504 /50x.html;

location = /50x.html {

root html;

}

include /usr/local/nginx/conf/vhosts/*.conf;

}

nginx的fastdfs模塊的配置文件以下:

[root@localhost ~]# cat /etc/fdfs/mod_fastdfs.conf | grep -v "^#" | grep -v "^$"

connect_timeout=2

network_timeout=30

base_path=/tmp

load_fdfs_parameters_from_tracker=true

storage_sync_file_max_delay = 86400

use_storage_id = false

storage_ids_filename = storage_ids.conf

tracker_server=192.168.1.219:22122

storage_server_port=23000

group_name=group1

url_have_group_name = true

store_path_count=6

store_path0=/data/data1

store_path1=/data/data2

store_path2=/data/data3

store_path3=/data/data4

store_path4=/data/data5

store_path5=/data/data6

log_level=info

log_filename=/var/log/mod_fastdfs.log

response_mode=proxy

if_alias_prefix=

flv_support = true

flv_extension = flv

group_count = 0

-----------------以上就是用到的全部配置文件----------------------------

想實現按照不通的項目存儲到不一樣的路徑,主要經過代碼來實現以java爲例

首先: 咱們能夠經過tracker獲取一個能夠使用的storage的ip, -------java代碼以下---------

/**
* 得到可用的storage IP
*
* @param trackerClient
* @param trackerServer
* @return 返回storage IP
*/
private static String getStorageServerIp(TrackerClient trackerClient, TrackerServer trackerServer) {
String storageIp = null;
if (trackerClient != null && trackerServer != null) {
try {
StorageServer storageServer = trackerClient.getStoreStorage(trackerServer, STORAGE_SERVER_GROUP);
storageIp = storageServer.getSocket().getInetAddress().getHostAddress();
} catch (IOException e) {
e.printStackTrace();
}
}
log.info("——獲取組中可用的storage IP——" + storageIp);
return storageIp;
}

其次:能夠根據獲取的ip把須要上傳的文件指定到要上傳到的路徑裏面

/**
* 獲得Storage服務
*
* @param storageIp
* @return 返回Storage服務
*/
private static StorageServer getStorageServer(String storageIp) {
StorageServer storageServer = null;
if (storageIp != null && !("").equals(storageIp)) {
try {
storageServer = new StorageServer(storageIp, Integer.parseInt(STORAGE_SERVER_PORT), Integer.parseInt(STORAGE_SERVER_M00));
} catch (IOException e) {
e.printStackTrace();
}
}
log.info("——storage server生成");
return storageServer;
}