nginx-upload-module模塊實現文件斷點續傳

導讀	每當咱們想簡單的實現文件上傳功能，而又不使用其餘的語言（好比PHP、Java），或者想實現文件的斷點續傳。這個時候Nginx的一個模塊nginx-upload-module就能知足咱們的需求。

模塊安裝

下載模塊：

cd /tmp
wget https://codeload.github.com/vkholodkov/nginx-upload-module/zip/2.2
unzip 2.2

安裝模塊：

.configure --add-module=/tmp/nginx-upload-module-2.2/

multipart/form-data表單上傳示例

nginx.conf配置：

server {
  [...]
         location /upload {
                upload_pass @uploadHandler;
                upload_store /usr/local/nginx/upload_temp 1;
                upload_set_form_field $upload_field_name.path "$upload_tmp_path";
        }
     
         location @uploadHandler {
                proxy_pass http://backend-host;
        }
    [...]
    }

這裏在server裏定義了upload location，這個location是上傳的接口，還有@uploadHandler location，是當文件上傳完成後，nginx模塊會對這個location發送一些必要的信息，如文件上傳的路徑，這裏涉及了幾個指令：

upload_pass @uploadHandler：上傳完成後會發送必要的數據到@uploadHandler;
upload_store /usr/local/nginx/upload_temp 1: 文件上傳的臨時目錄;
upload_set_form_field $upload_field_name.path 「$upload_tmp_path」: 設置文件上傳完成後，把文件臨時路徑發送給upload_pass指定的location。

斷點續傳示例

nginx.conf配置

server {
[...]
        location /resumable_upload {
               upload_resumable on;
               upload_state_store /usr/local/nginx/upload_temp ;
               upload_pass @drivers_upload_handler;
               upload_store /usr/local/nginx/upload_temp;
               upload_set_form_field $upload_field_name.path "$upload_tmp_path";
            }
     
         location @resumable_upload_handler {
               proxy_pass http://localhost:8002;
        }
    [...]
    }

與上一步multipart/form-data表單上傳示例配置不一樣的地方有：
upload_resumable on: 開啓斷點續傳功能;
upload_state_store /usr/local/nginx/upload_temp: 設置斷點續傳狀態文件存儲的目錄。

上傳文件第一個片斷

POST /upload HTTP/1.1
Host: example.com
Content-Length: 51201
Content-Type: application/octet-stream
Content-Disposition: attachment; filename="big.TXT"
X-Content-Range: bytes 0-51200/511920
Session-ID: 1111215056
    
<0-51200的字節文件數據>

上傳文件第一個片斷服務器響應

HTTP/1.1 201 Created
Date: Thu, 02 Sep 2010 12:54:40 GMT
Content-Length: 14
Connection: close
Range: 0-51200/511920
     
0-51200/511920

上傳文件最後一個片斷

POST /upload HTTP/1.1
Host: example.com
Content-Length: 51111
Content-Type: application/octet-stream
Content-Disposition: attachment; filename="big.TXT"
X-Content-Range: bytes 460809-511919/511920
Session-ID: 1111215056

<460809-511919字節文件數據>

上傳文件最後一個片斷服務器響應

HTTP/1.1 200 OK
Date: Thu, 02 Sep 2010 12:54:43 GMT
Content-Type: text/html
Connection: close
Content-Length: 2270
     
< 響應的內容>

請求頭說明

  請求頭 	                      說明
Content-Disposition  attachment, filename=「上傳的文件名」
Content-Type 	     待上傳文件的mime type,如application/octet-stream(注:不能爲multipart/form-data)
X-Content-Range      待上傳文件字節範圍,如第一片斷bytes 0-51200/511920,最後一個片斷bytes 460809-511919/511920(注:文件第一個字節標號爲0,最後一個字節標號爲n-1,其中n爲文件字節大小)
X-Session-ID 	     上傳文件的標識,由客戶端隨機指定.由於是斷點續傳,客戶端必須確保同一個文件的全部片斷上傳標識一致
Content-Length 	     上傳片斷的大小

Python上傳demo

#!/usr/bin/python
# -*- coding: utf-8 -*- 
     
import os.path
import requests
import hashlib
     
# 待上傳文件路徑
FILE_UPLOAD = "/tmp/testfile"
# 上傳接口地址
UPLOAD_URL = "http://host/drivers_upload"
# 單個片斷上傳的字節數
SEGMENT_SIZE = 1048576
    
def upload(fp, file_pos, size, file_size):
        session_id = get_session_id()
        fp.seek(file_pos)
        payload = fp.read(size)
        content_range = "bytes {file_pos}-{pos_end}/{file_size}".format(file_pos=file_pos,
                        pos_end=file_pos+size-1,file_size=file_size)
        headers = {'Content-Disposition': 'attachment; filename="big.TXT"','Content-Type': 'application/octet-stream',
                    'X-Content-Range':content_range,'Session-ID': session_id,'Content-Length': size}
        res = requests.post(UPLOAD_URL, data=payload, headers=headers)
        print(res.text)
     
     
# 根據文件名hash得到session id
def get_session_id():
  m = hashlib.md5()
  file_name = os.path.basename(FILE_UPLOAD)
  m.update(file_name)
  return m.hexdigest()
     
def main():
  file_pos = 0
  file_size = os.path.getsize(FILE_UPLOAD)
  fp = open(FILE_UPLOAD,"r")
     
  while True:
   if file_pos + SEGMENT_SIZE >= file_size:
       upload(fp, file_pos, file_size - file_pos, file_size)
       fp.close()
       break
   else:
     upload(fp, file_pos, SEGMENT_SIZE, file_size)
     file_pos = file_pos + SEGMENT_SIZE
     
if __name__ == "__main__":
        main()